Toward Country-led Development A Multi-Partner Evaluation of the Comprehensive Development Framework Synthesis Report This evaluation has been supported and guided by the following organizations and countries, which form its Steering Committee. Bilateral Aid Agencies: Multilateral Aid Agencies: Governments: Canadian International TThe World Bank Bolivia Development Agency ­ Operations Evaluation Department Burkina Faso Danish International ­ Development Research Development Assistance, Group Ghana Ministry of Foreign Affairs African Development Bank Romania Japan Bank for International Cooperation Asian Development Bank Uganda Economic Cooperation European Commission Vietnam Bureau, Ministry of Foreign Affairs, Japan Organization for Economic Cooperation and Ministry of Foreign Affairs, Development The Netherlands ­ OECD Development International Centre Nongovernmental Ministry of Foreign Affairs, ­ OECD DAC Network on Organization: Norway Development Evaluation Oxfam Great Britain Swedish International United Nations Development Authority Development Program Swiss Agency for United Nations Economic Private Sector: Development and Commission for Africa Cooperation Pakistan, Packages Limited United Kingdom Romania, TOFAN Department for International Development United States Agency for International Development Toward Country-led Development A Multi-Partner Evaluation of the Comprehensive Development Framework Synthesis Report © 2003 The International Bank for Reconstruction and Development / The World Bank 1818 H Street, NW Washington, DC 20433 Telephone 202-473-1000 Internet www.worldbank.org E-mail feedback@worldbank.org All rights reserved. 1 2 3 4 05 04 03 The findings, interpretations, and conclusions expressed herein are those of the author(s) and do not necessarily reflect the views of the organizations or institutions they represent, or of the Board of Executive Directors of the World Bank or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of the World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. Rights and Permissions The material in this work is copyrighted. Copying and/or transmitting portions or all of this work without permis- sion may be a violation of applicable law. The World Bank encourages dissemination of its work and will normally grant permission promptly. For permission to photocopy or reprint any part of this work, please send a request with complete information to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, USA, telephone 978-750-8400, fax 978-750-4470, www.copyright.com. All other queries on rights and licenses, including subsidiary rights, should be addressed to the Office of the Publisher, World Bank, 1818 H Street NW, Washington, DC 20433, USA, fax 202-522-2422, e-mail pubrights@worldbank.org. ISBN: 0-8213-5643-7 e-ISBN: 0-8213-5644-5 Library of Congress Cataloging-in-Publication Data has been applied for. Contents iii Foreword vii Preface ix Acknowledgments xi Abbreviations and Acronyms xv Executive Summary xvii 1. Introduction: The CDF as a Development Concept 1 Purpose and Structure of this Report 1 Background: Evolution of the CDF 4 Putting the CDF in Practice 5 The CDF Principles 6 The Relationship Between the CDF and the PRSP 12 2. Implementing the CDF Principles: The Evidence 17 Long-term, Holistic Development Framework 17 Results Orientation 21 Country Ownership 24 Country-led Partnership 29 3. The CDF Principles as a Package: Interaction and Impact 41 Complementarities and Tensions among the CDF Principles 41 The Role of the PRSP in Implementing the CDF Principles 44 The CDF and the Business Environment 48 The Development Impact of the CDF and CDF-like Experiences 50 4. Conclusions and Recommendations 57 Long-term, Holistic Framework 58 Results Orientation 59 Country Ownership 60 iv Country-led Partnership 62 The Road Ahead and an Opportunity 64 Annex 1. The CDF Evaluation at a Glance 67 Annex 2. Approach Paper for a Multi-Partner Evaluation of the 69 Comprehensive Development Framework Annex 3. CDF Principles: Working Definitions 80 Annex 4. The CDF and the PRSP: Key Facts 82 Annex 5. Evaluation Methodology 85 Annex 6. Evaluation Survey Instruments 98 Annex 7. Multi-Stakeholder CDF Evaluation Process: 118 Analysis of Survey Questionnaire Results Endnotes 123 Bibliography 129 Boxes Box 1.1: The CDF Principles: Antecedents and Support 7 in the Development Community Box 1.2: Some Reservations about the Results Principle 10 Box 2.1: The CDF, PRSPs, and Political Change 18 Box 2.2: Results Orientation: Examples of Recent 22 Achievements and Innovations Box 2.3. Quality of the Domestic Dialogue 26 Box 2.4: Perils of Counterproductive Participation 27 Box 2.5: Main Burdens of Donor Practices in 11 Countries 31 Box 2.6: Disadvantages of Project Implementation Units 32 Box 2.7: Recent Harmonization Efforts at the Global and Country Levels 35 Box 2.8: Three Donors, Three Sets of Procedures, No Building 36 Box 2.9: Independent Donor Performance Monitoring: 38 The Tanzanian Experiment Box 3.1: Participatory Budgeting and the PRSP 44 Box 3.2: PRSPs and HIPC Conditionality: Evidence from the Case Studies 46 Box 3.3: Feedback on the Early PRSPs 47 v Box 3.4: The Intersection of the PRSP and Domestic Politics 49 Tables Table 1.1: Development and Aid Intensity Indicators for the 3 CDF Evaluation Case Study Countries Table 1.2: Coordination Is Needed with Many Actors 11 Table 3.1: Complementarities and Tensions among the CDF Principles 43 Table 3.2: CDF and Business Environment in the Six Case Study Countries 50 Table 3.3: Marginal Contribution of CDF-like Development 53 Figures Figure 2.1: Use of Project Implementation Units 33 Figure 2.2: Joint Donor Preparation/Appraisal Missions 34 in the Preceding 12 Months Figure 3.1: Index of Overall CDF-like Development Strategies by Region 51 Figure 3.2: Structural and Socioeconomic Factors and Choice 52 of CDF-like Development Strategies Foreword vii We have had the challenge and the responsi- day-to-day work of the secretariat and the bility of overseeing the implementation of a progress of the evaluation. large, complex, and unique evaluative effort. The undertaking was remarkable in several Ten donors provided financial or in-kind respects: support for the evaluation. The governance structure involved some None of these elements is unprecedented 30 stakeholders from partner countries, by itself, but we believe the combination is donors and development agencies, and unparalleled. The effort generated valuable from an international NGO and a private experience in collaborative evaluation of devel- sector enterprise. opment issues and provided an unusually broad perspective and solid support for the resulting The 11 component studies were conducted conclusions and recommendations. We believe by teams composed of evaluators and that the recommendations, if adopted by donors researchers intentionally drawn from the and countries, will result not only in more wide- South and the North. spread adoption of the principles of the Comprehensive Development Framework, but A multi-partner Management Group drawn also in sustained progress in reducing poverty from the Steering Committee oversaw the on a larger scale. CDF Evaluation Management Group Gregory K. Ingram, Co-Chair Paul Collier, Co-Chair Director-General Director Operations Evaluation, World Bank Development Research Group, World Bank Marcelo Barron Emmanuel Tumusiime-Mutebile External Finance General Directorate Governor Ministry of Finance, Bolivia Bank of Uganda Niels Dabelstein Head, Danida Evaluation Secretariat Ministry of Foreign Affairs, Denmark Preface ix The Comprehensive Development Framework-- Operations Evaluation Department (OED) to launched by World Bank President James D. assess CDF implementation. OED and the Wolfensohn in early 1999--has become an Bank's Development Economics Research important influence on the global development Group (DECRG) joined forces and launched agenda. It provided conceptual underpinnings the evaluation by hosting a conference with rep- for the Poverty Reduction Strategy Papers resentatives from CDF pilot countries, donors, (PRSPs) and fed into such initiatives as the nongovernmental and private sector organiza- Millennium Development Goals (MDGs) and tions, and academia. This group recommended the Monterrey Consensus. The core Compre- that the evaluation should be a multi-stake- hensive Development Framework (CDF) princi- holder partnership. Accordingly, a 30-member ples have gained widespread endorsement. They Steering Committee and 5-member Manage- are not new individually, but bringing them ment Group were established in January 2001 to together as a unified concept and championing guide the evaluation. Multi-disciplinary evalua- the package within the global development tion teams with members from developed and community has been an important innovation. developing countries conducted six country studies and five thematic studies. Nine bilateral This report concludes an extended multi-part- and multilateral donors provided financial and ner effort to evaluate the implementation of the in-kind support, amounting to about 60 percent CDF principles, to identify the factors that have of the evaluation's total cost. facilitated and hindered it, and to assess the extent to which CDF implementation has The evaluation shows that both donors and affected behaviors and outcomes. The breadth recipients have made progress in implementing of the evaluation's multi-partner approach and the CDF principles, particularly in countries governance structure has been virtually where one or more of the principles have been unprecedented, and the process itself generated applied over a number of years. These positive valuable experience on collaborative ways to changes are fragile and could be stalled or evaluate issues of broad mutual interest to the reversed. Implementing the principles requires development community. changes in entrenched behaviors and institu- tional practices--not easily or quickly done. The evaluation has its origins in a December Thus, dedicated and consistent attention is 1999 request by the Bank's Board Committee on needed by top donor leadership and recipient Development Effectiveness (CODE) to the countries to ensure that momentum is sustained. Acknowledgments xi A 30-member, multi-partner steering committee Contributors and a 5-member management group have gov- erned the CDF evaluation. CDF Evaluation Secretariat: Ibrahim Elbadawi, Lead Economist, Development Research Group, World Bank, and John Eriksson, Management Group Task Team Leader, Operations Evaluation Department, World Bank, provided overall Gregory K. Ingram (co-chair), Director-General, coordination for the evaluation. John Eriksson Operations Evaluation, World Bank. and Laura Kullenberg, Senior Evaluation Paul Collier (co-chair), Director, Development Officer, Operations Evaluation Department, Research Group, World Bank. coordinated and edited the final report. Marianela Zeballos, General Director, and They also led or coordinated the country case Marcelo Barron, General Directorate for External studies. Victoria Elliott, Operations Evaluation Finance, Bolivia. Department, World Bank, managed OED's contributions. Niels Dabelstein, Director, Evaluation Secretariat, Danish International Development Agency, Chapter 1: Ministry of Foreign Affairs, Denmark Carol Lancaster, Associate Professor and Director, Emmanuel Tumusiime-Mutebile, Governor, Masters of Science in Foreign Service, Foreign Bank of Uganda. Service Program, Georgetown University; Alison Scott, Senior Social Development Adviser, Department for International Development, UK; Financial Support and Laura Kullenberg. The World Bank and the governments of Chapter 2: Canada, Denmark, the Netherlands, Norway, Paul Collier; Laura Kullenberg; Charles Soludo, Sweden, Switzerland, and the United Kingdom Professor of Economics, University of Nigeria, provided financial support for the evaluation. Nsukka; Mirafe Marcos, Research Analyst, The Asian Development Bank and Japan pro- Operations Evaluation Department; and John vided staff and consultant support for the Eriksson. Vietnam country case study. xii Chapter 3: Mirafe Marcos and Georgia Wallen, Research Paul Collier; Alison Scott; Ibrahim Elbadawi; Analysts, provided research support; in John Randa, Research Analyst, Development addition, Mirafe Marcos participated as a Research Group; and John Eriksson. member of the Ghana and Uganda country case study teams. Chapter 4: CDF Evaluation Secretariat. William B. Hurlbut and Caroline McEuen, Writer/Editors, Operations Evaluation Depart- ment, provided technical support for document production and editing. Advisers to the Report Drafting Team: Julia Ooro, Program Assistant, provided administrative and document production and editing support. Hirohisa Kohama, Professor of Economics, University of Shizuoka, Japan. Tourya Tourougui, Program Assistant, provided Don Pressley, Principal, Booz Allen Hamilton, administrative support. Washington, D.C. Alison Scott, Senior Social Development Adviser, The volume was published in OED's Partner- Department for International Development, UK. ships and Knowledge Group, under the direc- Samuel Wangwe, Executive Director, Economic tion of Osvaldo Feinstein, by the Outreach and Social Research Foundation, Tanzania. and Dissemination staff of the Knowledge Management Unit, including Patrick Grasso, In Addition: Lead Knowledge Management Officer; Vivian Jackson, Publications Officer; Caroline McEuen, Simon Maxwell, Director of the Overseas Writer/Editor; and Juicy Qureishi-Huq, Pro- Development Institute, UK, provided com- gram Assistant. ments on draft chapters of the report. xiii A Development Expert Looks at the CDF Evaluation Boiled down to its essentials, evaluation is about answering three questions: `Does it work?' `Is it worth it?', and `Why?' The third question is often neglected. Simon Maxwell Director, Overseas Development Institute Simon Maxwell applies his three evaluation criteria to the findings of this evaluation of the Comprehensive Development Framework. He identifies donors as most in need of "radical change" and ends with a heretical thought: In the case of the CDF evaluation we learn that the four core principles of the CDF have largely been operationalized through the PRSP process, which similarly emphasizes (a) long-term and holistic vision, (b) country ownership, (c) results orientation, and (d) country-led partnership. We learn that none of these ideas is entirely new, but that the CDF/PRSP initiatives have helped to accelerate and reinforce adoption, albeit unevenly. We are led to conclude that there are practical benefits, especially in terms of more widely-owned and better-targeted public expenditure programs. And we can surmise that all this is worth it because progress towards the Millennium Development Goals will be faster than it would otherwise have been. A PRSP is not a perfect incarnation of the CDF principles, but it is a good start, it is worth it, and it does, up to a point, work. There is still some way to go, however. Long-term visions are not always shared by competings political parties. Participatory processes can be--and sometimes are--aptured by vested interests. Results-based approaches are theoretically contested and hard to implement. And donors find it almost impossible to sustain genuine partnerships based on reciprocal accountability. Asking `why' performance falls short of the ideal--so far--points to what must happen next. First, expectations need to be realistic. For example, a national consensus on economic policy, sustained over time, is unachievable and probably undesirable. Second, implementation needs to be pragmatic. For example, the genuine problems with simplistic, target-driven approaches to the reform of public administration need to be recognized by adopting more flexible guidelines. Third, even pragmatic implementation takes time and needs resources, especially for capacity building. And finally, the incentives need to be right, both for recipient countries and within donor agencies. The most radical change needed at present is by donors. There are some practical things they should do differently in-country, for example to reinforce the role of parliaments in policy-making. But more generally, a political commitment is needed, translated into rules, regulations and decisions about resource allocation, to simplify and harmonize procedures, coordinate policy dialogue, and improve accountability to recipient country stakeholders. A heretical thought: is not one way to do this to increase the share of multilateral rather than bilateral aid? Abbreviations and Acronyms xv ADB Asian Development Bank GDP Gross Domestic Product BPRS Bolivia Poverty Reduction Strategy GNI Gross National Income CAF Corporación Andina de Fomento GoB Government of Bolivia CAS Country Assistance Strategy GoV Government of Vietnam CDF Comprehensive Development GPRA Government Performance and Framework Results Act CG Consultative Group GPRS Ghana Poverty Reduction Strategy CSLP Cadre Stratégique de Lutte HIPC Heavily Indebted Poor Countries contre la Pauvreté HQs Headquarters CODE Committee on Development IDA International Development Effectiveness Association CPIA Country Policy and Institutional IDGs International Development Goals Assessment IFI International Financial Institution CPRGS Comprehensive Poverty Reduction and Growth Strategy IMF International Monetary Fund DAC Development Assistance I-PRSP Interim Poverty Reduction Strategy Committee Paper DEC Development Economics JBIC Japan Bank for International (World Bank) Cooperation DECRG Development Economics Research JSA Joint Staff Assessment Group LCR Latin America and the Caribbean DFID Department for International Region of the World Bank Development LICUS Low-Income Countries under Stress EBRD European Bank for Reconstruction M&E Monitoring and Evaluation and Development MDB Multilateral Development Bank ESW Economic and Sector Work MDGs Millennium Development Goals EC European Commission MENA Middle East and North Africa EU European Union Region (World Bank) xvi MFPED Ministry of Finance, Planning, and PRSC Poverty Reduction Support Credit Economic Development (Uganda) PRSP Poverty Reduction Strategy Paper MIT Massachusetts Institute of RBM Results-based Management Technology SPA Strategic Partnership with Africa MOFA Ministry of Foreign Affairs SSA Sub-Saharan Africa MTEF Medium-term Expenditure Framework SWAp Sectorwide Approach NEPAD New Partnership for Africa's TA Technical Assistance Development UK United Kingdom NGO Nongovernmental Organization UNDAF United Nations Development NSSD National Strategy for Sustainable Assistance Framework Development UNDP United Nations Development ODA Official Development Assistance Program OECD Organization for Economic UNECA United Nations Economic Cooperation and Development Commission for Africa OED Operations Evaluation Department UNICEF United Nations Children's Fund PAF Poverty Action Fund UPEI Universal Primary Education Initiative PEAP Poverty Eradication Action Plan US United States PIU Project Implementation Unit USAID United States Agency for PPA Participatory Poverty Assessment International Development PREM Poverty Reduction and Economic WG Working Group Management PRGF Poverty Reduction and Growth Facility Executive Summary xvii This evaluation report synthesizes the findings In response, the donor community took a number of a multi-partner effort to assess implementa- of steps, from debt relief through the joint World tion of the Comprehensive Development Bank-IMF Heavily Indebted Poor Countries Framework (CDF). The evaluation's primary (HIPC) initiative to shifting the aid focus from objectives are to: inputs to measurable outcomes, as with the International Development Goals promulgated by Identify the factors that have facilitated the OECD Development Assistance Committee. implementation of CDF principles, and Recipient countries also began to develop reme- those that have hindered it. dies: the Vietnamese government urged donors to work together to reduce the administrative bur- Assess the extent to which CDF implemen- dens of aid management, and Uganda adopted a tation has affected intermediate outcomes long-term framework and a results-measurement and, to the extent possible, longer-term approach for its poverty reduction efforts. development outcomes. At the World Bank, these strands came together in 1998 in a Partnerships Group. Its purpose was Background announced by its name: to develop the concept of aid as a partnership between recipient govern- In the mid-1990s, the aid community began a ments and donors. This presaged the introduc- candid self-assessment. Disappointing develop- tion in early 1999 of the Comprehensive ment results--especially in Sub-Saharan Development Framework, which, as described Africa--had raised troubling questions: Does by Bank President James Wolfensohn, was a the emphasis on structural adjustment ignore new way for the Bank to do business with recip- the poor? Do the many agencies and interna- ient countries and other development partners. tional organizations working in developing countries overburden, rather than strengthen, the capacity of recipient governments? Does the New Approaches Informed poor coordination of donors add to the chal- lenge of making development effective? by Past Shortcomings Increasingly, the painful realization of develop- ment agencies, recipient countries, and aid ana- At the heart of the CDF is the belief that the lysts was "yes"--the full potential of way aid is delivered, not just its content, has an international aid to reduce poverty by achieving important influence on its effectiveness, and positive, sustainable development results was that poverty reduction is the fundamental goal not being fulfilled. of international aid. The CDF consists of four xviii principles--a Long-Term, Holistic Develop- In 1999, several countries volunteered to pilot ment Framework; Results Orientation, Country CDF approaches. Many others soon developed Ownership; and Country-led Partnership--each broad-based Poverty Reduction Strategy Papers, of which responds to past development assis- or PRSPs, in order to access debt relief under the tance shortcomings and presents an approach HIPC initiative. All four of the CDF principles for improvement. served as the basis for PRSP development. Other donors and recipients also supported CDF Development strategies should be compre- tenets, which often reinforced their own incli- hensive and holistic, and shaped by a long- nations or policies. And the CDF came to influ- term vision. Past emphasis on short-term ence the way subsequent global initiatives, such macroeconomic stabilization and balance of as the Millennium Development Goals, have payment pressures overwhelmed longer-term been framed. structural and social considerations (for example, expanding and improving educa- tion and health facilities, maintaining infra- structure, and training a new generation of "Why" and "How" public officials). of the Evaluation Development performance should be evalu- ated through measurable, on-the-ground In late 1999, the Bank's Executive Board results. The traditional emphasis on dis- Committee on Development Effectiveness bursement levels and project inputs has asked OED to assess CDF implementation. OED measured resource allocation and consump- joined forces with the Bank's Development tion. What really matters is impact on Research Group to begin designing an evalua- people and their needs. tion approach. Then, in true CDF fashion, rep- resentatives of pilot countries, donor agencies, Development goals and strategies should be and civil society and private sector organizations "owned" by the country, based on citizen par- argued that the evaluation should be conducted ticipation in shaping them. While donor- and governed along multi-partner lines. The driven aid delivered under structural resulting 30-member Steering Committee chose adjustment was sometimes effective, in many 5 members to serve as a Management Group. cases painful and lengthy adjustment meas- ures were eventually undone. When coun- Because the Steering Committee concluded tries have greater say in shaping reforms, early on that it was not yet possible to ascribe governments and their citizens will be more development outcomes directly to the CDF, the committed to seeing them through. evaluation focuses on the extent to which CDF principles or CDF-like principles have been Recipient countries should lead aid manage- implemented in six case study countries-- ment and coordination through stakeholder Bolivia, Burkina Faso, Ghana, Romania, partnerships. Partnerships built on trans- Uganda, and Vietnam--and what the effects parency, mutual trust, and consultation can and implications have been thus far. improve aid coordination and reduce the inef- ficiencies, asymmetrical power relationships, Multi-disciplinary evaluation teams, with mem- and tensions of donor-led aid initiatives. bers from developed and developing countries, conducted the country studies as well as thematic port, as well as a lengthy process of learning by xix studies of each of the CDF principles. In addition, doing. The ideal PRSP will not be produced and a quantitative study of 88 developing countries implemented overnight, but will require wide- analyzed relationships between development out- spread public sector reform and institution come measures and proxy indicators of CDF-like building. These are long-term processes that will development strategies. The evaluation focuses not deliver results within the life of an average primarily on the relationship between recipient PRSP cycle. governments and donors and development agen- cies. The CDF principles cover all development relationships, including those among civil society Long-Term, Holistic and private sector partners. Development Framework Evaluation Conclusions A long-term development framework has opera- tional meaning only when it is translated into and Recommendations affordable priorities through a disciplined cen- tral budget process. Medium-term instruments Four sets of conclusions and corresponding rec- (such as the PRSP) provide the bridge from the ommendations--reflecting the four CDF princi- annual budget to the country's long-term frame- ples--emerged from the evaluation synthesis, work. Of the six countries studied, only Uganda with recommendations for recipient govern- follows such a process of costing and setting pri- ments and the donor community, and, in some orities and linking them to a medium-term cases, additional recommendations addressed expenditure framework. specifically to the World Bank. The evaluation also found that there can be tensions in how the Recommendation for recipient countries: CDF principles are applied, specifically involv- Strengthen the link between medium-term ing ownership and partnership, and the long- frameworks (such as the PRSP) and budgets. term focus and the emphasis on results. These Donors should support such efforts and make have been long-standing tensions in develop- sure their assistance is aligned with national ment practice and are neither novel nor unique development strategies. to the CDF. There is also evidence at the coun- try level that the CDF principles can be comple- Recommendation for all donors: Provide mentary. Country leadership, whether at the long-term assistance for capacity strengthening. highest levels of government, in line ministries, This should include sustained support for or among elected officials, remains a critical public sector reforms and institutional factor for making and sustaining progress on development. each of the principles, and for ensuring comple- mentarity among them as they are implemented. Many donor countries are raising the share of aid that they provide as budget and sector program The PRSP is a powerful tool for implementing support, and are increasingly signing multi-year the CDF principles in low-income countries, assistance agreements in support of national and on those grounds merits continued support development strategies. However, program aid and development. For the PRSP to succeed, it remains vulnerable to sudden shifts in donor deci- will need coherent and sustained external sup- sionmaking. If recipient countries are expected to xx adopt a long-term results focus to development Weak capacity of central and regional public planning, so should donors. Delays or non-dis- service providers makes this the most difficult bursement of committed funds undermines the principle to implement for all the case study integrity of the budget process and reduces the countries. Competing budget priorities, inade- effectiveness of projects and programs. quate incentives, and fragile accountability structures are additional constraints. Many Recommendation for all donors: Provide recipient countries appear to have adopted a predictable and reliable financing with trans- results-oriented approach primarily to satisfy parent, multi-year financing indicators, donors. So far, application has often been lim- based on clear country performance criteria. ited to specific aid-funded projects; a "results culture" has rarely been embedded in the day- Applying the holistic principle demands atten- to-day operations of government. tion to the multisectoral determinants of devel- opment outcomes--yet most donor agencies Recommendation for recipient countries: and recipient governments do not have internal Strengthen results orientation by increasing structures that encourage cross-sectoral dialogue political accountability--including citizens' or easy integration of multi-sector interven- right to demand results and government's tions. In addition, "silo" thinking and inter- ability to respond. Citizens' ability to help sectoral/departmental competition in donor monitor results through increased access to agencies can exacerbate inter-ministerial com- government information has been very lim- petition in client countries. ited. Governments should undertake three key initiatives: Additional recommendation specifically for a) Train and empower public servants to the World Bank: A conscious effort is open up information channels and needed to reform organizational arrangements educate the public. that discourage cross-sectoral collaboration, and b) Strengthen systems for financial manage- to develop more effective institutional ment, performance (value for money) and mechanisms for designing and implementing regulatory audits, and internal and exter- cross-sectoral programs. nal accountability. c) Present development strategies (such as Results Orientation the PRSP) in languages and through con- cepts the broad public will understand; Aid donors should no longer employ disburse- involve the media as a channel for ments as their only measure of success; stakehold- informed dialogue. ers should be held accountable for achieving development outcomes. The PRSPs (with their Donors have done little to harmonize (that is, focus on monitoring and evaluating results) and standardize) their reporting and results monitor- the high visibility of the Millennium ing, continuing to place administrative burdens Development Goals have contributed to this on recipient governments. And donors continue shift. Sectorwide approaches have helped institu- to overtax existing monitoring structures by pro- tionalize a results focus, and medium-term posing complex, special-purpose approaches, expenditure frameworks have introduced a with unwieldy indicators that conform more to results orientation in the budgetary process. donors' reporting requirements than to what is needed to manage national service delivery. The their stewardship and accountability functions, xxi result is unsustainable, donor-established they need better information about the impacts enclave monitoring systems. of their decisions on different groups, particularly those who normally have little or no voice. Recommendation for all donors: Strengthen and use country-led M&E systems. Donors Recommendation for all donors: Work with and governments need to view country the government to devise an approach for devel- development outcomes as a joint product, to opment consultations with elected officials and which donors contribute under country nongovernment representatives. In consulta- leadership. Results monitoring should be tions with interest groups, include the pri- seen as a shared responsibility in which vate sector and marginalized civil society donors and governments use the same infor- organizations. Materials and presentations mation flows for their individual purposes. should be easily understandable to grass- This requires significant investment in roots audiences. Donors should be prepared capacity building and harmonization of to provide assistance to strengthen the donor requirements. And it means donors capacities of these groups to participate in need to downplay some individual interests strategy consultations. for more effective joint action. Recommendation for recipient countries: Additional recommendation specifically To enhance country ownership, governments for the World Bank: Enhance the capacity of and parliaments should consult among diverse the World Bank to continue to track and ana- interest groups. This embraces the full range lyze the implementation of CDF principles and of civil society and the private sector, includ- their impacts. ing those who lack an organized voice, such as the poorest and women. Several consulta- tive approaches will be required, such as the Country Ownership use of Participatory Poverty Assessments. Donors and governments increasingly formulate Some elements of the PRSP process can inhibit strategy through expanded consultation with key the principle of country ownership. For exam- stakeholder groups, in both civil society and the ple, some countries believe that the PRSP private sector, contributing to country owner- requires their development programs to focus ship of reforms. However, ownership is too nar- too heavily on social expenditures. They believe rowly based when consultations are confined to that because the PRSP must pass through the the executive branch of the government or sup- Boards of the Bank and the International plemented only by ad hoc interactions with Monetary Fund, their options are limited. organizations that donors or government choose. Parliaments, local governments, and civil soci- Additional recommendations specifically ety and private sector entities have complained for the World Bank: Clarify the PRSP review of being marginalized in policy discussions. In process and the Bank's openness to alterna- countries with credible representative institu- tive PRSP-consistent development strategies. tions, international development organizations Differentiate more clearly the Board's roles should work through them, not sideline them. in relation to the PRSP and the Country But for parliaments and governments to exercise Assistance Strategy. xxii Country-led Partnership at the country level. Instead, provide the capacity building and resources countries Country-led partnership is intended to make the need to assume aid management. recipient country the leading voice in judging Give the recipient country oversight over aid the kind and quality of aid it receives. But quality. Establish a system of regular country- progress has been highly uneven--across level review panels composed of independent donors, across countries, and across sectors representatives from the recipient country within countries. Effective country leadership of and donors. Have the panels review donor as aid partnerships has been difficult to achieve well as recipient country performance against because of the inherent asymmetries in donor- a mutually agreed code of conduct and tar- recipient relationships. gets. Make results accessible to the public through regular publication and presentation Reform will require both donors and recipient at fora such as Consultative Group meetings. countries to make significant changes in their behavior and processes. Many donors face Decentralize staff and delegate more authority to domestic political resistance to harmonizing the field. Select field staff who have proven procedures or reducing the use of international partnership and relationship-building skills, consultants (tied aid still accounts for 50 per- as well as the requisite subject-area expertise. cent of aid flows). Although donors' field offices have made improvements, the basic parameters Phase out Project Implementation Units for donor behavior are set at headquarters level; (PIUs). Every PIU should be accompanied it is there that most change must take place. by a plan for phase-out over the life of the project or program. The distortions created Country leadership is fostered when donors pro- by salary incentives typically associated with vide financial resources through mechanisms that PIUs need to be addressed in the context of apply common pool principles, such as budget public service reform. support through Poverty Reduction Support Credits and Sectorwide Approaches. But donors Additional recommendations specifically for cannot accede to country leadership if recipient the World Bank: The country case studies country administrations display corruption or highlighted some aspects of these recommen- economic mismanagement. Both donors and dations that apply in particular to the Bank. recipients need to move faster to align external Continue decentralization and delegation of assistance with country strategies; to promote authority to field offices. The Bank has made increased rationalization and selectivity of donor- considerable progress in decentralization over funded activities; and to harmonize donor proce- the last five years. This has involved not only dures and practices. Donor-only coordination the placement of country directors in the exercises may be a first step toward harmoniza- field, but also the fielding of additional head- tion, but they do not by themselves foster recipi- quarters staff and decisionmaking authority. ent country ownership. The ultimate goal should be to harmonize donor procedures with recipient- Select, train, and reward staff--in part--on the country national standards and procedures. basis of their partnership performance. This applies not only to field-based staff, but also Recommendations for all donors: to headquarters staff who service country Step back from micro-managing the aid process programs (such as sector specialists). Practice what the Bank preaches regarding har- and results, public attitudes to aid, and the xxiii monization and simplification, program or budget role of Audit Offices and Treasuries in com- support, and selectivity and "stepping back." pounding the problem. Recommendations for recipient countries: d) How to expand learning between recipient countries--for example, Uganda's experi- Put responsibility for aid coordination at a high ence with the medium-term expenditure level of government and endow this function framework and hard budget constraints. with sufficient resources, authority, and political support to lead the aid management The evaluation concludes that while there is process. Strong leadership is needed for a progress toward implementing CDF principles, government to consider only aid proposals that progress has been uneven. The greatest that are consistent with its national priorities gains have occurred in countries that have and budget (and reject those that are not). applied one or more of the principles for a Implement and enforce procurement and other number of years. This finding is not surprising, rules that will engender the confidence of donors. given that the process of change is still young Promulgate procurement and other rules that and that adopting the CDF principles in full will meet donors' expectations, and enforce requires significant (and often difficult) changes them consistently. This is particularly impor- in norms, behaviors, and institutional practices tant where common pool funding and pro- on the part of both donors and recipient coun- gram support is desired. tries. Transparency and mutual trust are required of all parties. Continuous political leadership and sustained will on the part of all major devel- The Road Ahead opment actors are thus needed if today's dysfunc- tional aid practices are to be transformed under the CDF approach. Some promising opportuni- The evaluation suggests several important ties have recently emerged for donors and recip- areas that would benefit from expanded learn- ients to move ahead. These include the New ing efforts, including research and exchange Partnership for Africa's Development, the of experience: Monterrey Consensus, and the increasing adop- a) How to establish country-owned monitor- tion of Millennium Development Goals in ing and evaluation systems that bring stake- PRSPs and other country development frame- holders together, building from information works. Another relevant initiative is the joint and monitoring initiatives in government declaration that emerged from the February 2003 and among civil society, donors, and the High-Level Forum on Harmonization, particu- private sector. larly the commitment by the parties to utilize the strengthen existing mechanisms to maintain b) How to expand involvement in CDF peer pressure on implementing harmonization processes by marginalized groups in civil soci- agreements. The World Bank, in cooperation ety and the private sector. with other development partners, can and should play a lead role in integrating the CDF c) How to start a debate in donor countries principles into all these global initiatives and in about changing incentives, pooling resources identifying additional avenues for progress. C H A P T E R O N E Introduction: 1 The CDF as a Development Concept Purpose and Structure Objectives of the Evaluation of this Report The objectives of the evaluation were: (1) to assess how the CDF is being implemented on the ground; (2) to identify the factors that have World Bank President James D. Wolfensohn facilitated implementation of CDF principles introduced the Comprehensive Development and those that have hindered it; and (3) to Framework in early 1999. It brings together, in a promote learning and capacity development unified framework, four principles to improve in countries where CDF principles are being the effectiveness of development assistance in implemented. The evaluation was to deal with reducing poverty: a long-term, holistic frame- the relevance, efficacy, and efficiency of the over- work; results orientation; country ownership; and all development assistance system in selected country-led partnership. This report synthesizes countries, both on the ground and at the policy the results of a multi-partner effort launched in level, including linkages to the international January 2001 to assess the implementation of the development assistance architecture and the Comprehensive Development Framework Millennium Development Goals (MDGs). (CDF). This first chapter describes the evalua- tion process and methodology, historical prece- dents and events leading to the introduction of Methodology the CDF, the problems the CDF principles Six countries were selected for in-depth case attempted to address, and the relationship studies: Bolivia, Ghana, Romania, Uganda, and between the CDF and Poverty Reduction Vietnam--all CDF pilot countries at the time-- Strategy Paper (PRSP). Chapter 2 summarizes and Burkina Faso (a non-CDF pilot). Because the evaluation findings and is divided into four the main purpose of the evaluation was to look sections--one for each principle. Chapter 3 dis- at what had happened on the ground since the cusses the CDF as a whole and the interaction, CDF was launched, priority was given to inter- complementarities, and tensions among the ested countries with the longest track record of principles, including how they have played out implementation and highest performance rank- through PRSPs. The chapter concludes with a ings (according to the CDF Secretariat), on summary of the results of a quantitative analysis grounds that these cases would offer the greatest of the relationships between indirect or proxy potential for learning. One non-CDF pilot coun- indicators of CDF principles and development try was chosen as a control. Because the PRSP is outcomes in a wide range of countries, as well as defined as an instrument for implementing the the relationships between CDF proxy indicators CDF principles in low-income countries, all and indices of the business environment. countries selected (except Romania) were also Chapter 4 draws the main conclusions and pres- PRSP countries. Consideration was also given to ents recommendations for the future. Regional balance and avoiding overlap with 2 similar evaluative efforts (such as the Strategic matter to ascribe development outcomes Partnership with Africa, or SPA, study of PRSP directly to the advent of the CDF as a whole, implementation in eight African countries). since a framework for development processes and aid management is only one among many Multi-disciplinary evaluation teams with mem- factors that influence the course of a country's bers from developed and developing countries development. But the individual CDF principles visited each country and carried out intensive have been practiced at various times in a variety surveys, literature reviews, focus group meetings, of ways in different countries. Thus the evalua- structured interviews, and feedback workshops. tion tracks the extent to which CDF or CDF- Surveys of government-donor relations, with a like principles have been implemented in six focus on aid transaction costs, were carried out different countries and what the effects and in the five former CDF pilot countries. At the implications have been thus far. It reports on same time, academics and practitioners from the evidence already available to chart changes in North and the South prepared thematic studies behaviors, processes, and transaction costs, as on each of the CDF principles. A pioneering well as emerging problems, obstacles, and risks. attempt was also made to use quantitative analy- sis to assess the effects of CDF-like practices in a Notwithstanding the sound rationale for the wide range of countries. A detailed description selection of case study countries, as in any of methodology and survey instruments used for study relying heavily on country case study evi- the six country case studies, four thematic stud- dence, it would have been desirable to have ies, and the econometric study is included in covered a larger sample of countries, had Annexes 5 and 6. budget and time permitted. Given that only one middle-income country was included in the case study sample (Romania), the evalua- Table 1.1 provides a snapshot socioeconomic tion is not strongly evidence-based regarding profile of the six case study countries. Gross the application of the CDF principles to these domestic product (GDP) per capita increases countries. In addition, there was little or no sixfold from Burkina Faso to Romania. investigation of the applicability of CDF prin- Vietnam's social indicators, such as illiteracy, ciples in post-conflict and LICUS (Low- life expectancy, and infant mortality, are better Income Countries Under Stress) countries or than would be expected given its per capita to global "vertical" programs, such as for income level. All the countries receive rela- HIV/AIDS. Thus, some findings (e.g., general tively high aid flows per capita, but aid as a per- or sector budget support) might not apply to cent of national income is sharply lower for the these contexts. The evaluation focuses prima- higher-income countries. All the countries have rily on the relationship between partner gov- a high level of debt relative to national income; ernments and development agency external Bolivia, Burkina Faso, Ghana, and Uganda are partners. The CDF principles cover all devel- beneficiaries of the Heavily Indebted Poor opment relationships, including those among Country (HPIC) initiative. civil society and private sector partners. The evaluation was unable to fully address informa- Limitations of the Evaluation tion, policy, and capacity constraints that keep Analysis of the development impact of the CDF the poor and other marginalized groups, such as initiative is not yet possible because it was intro- women, from contributing to, and benefiting duced only in 1999. Nor is it a straightforward from, CDF processes. Understanding of these Table 1.1. Development and Aid Intensity Indicators for the CDF Evaluation Case 3 Study Countries Lower- middle- Low-income Burkina income countries Faso Ghana Uganda Vietnam countries Bolivia Romania Population 2.5 11.6 19.7 22.8 79.5 2.2 8.5 22.4 (2001: million) (billion) (billion) Per capita GDP 477 250 421 355 356 1,366 944 1,570 (constant 1995 US$) Per capita GDP 2 3 2 2 4 3 -1 5 growth rate (2001: annual %) Life expectancy 59 44 57 42 69 69 63 70 at birth (2000: years) Infant mortality rate 76 104 58 83 28 33 57 19 (2000: per thousand) Adult illiteracy rate 38 75 27 32 7 15 14 2 (2000: % of population 15 and above) Population below n.a. 61 45 n.a. n.a. n.a. 14 3 $1 a day (%) (1995) (1999) (1999) (1995) Aid per capita 9 30 32 37 22 7 57 19 (2000: US$) Aid (2000: % of GNI) 2 15 13 13 5 1 6 1 Aid (2000: % of n.a. n.a. n.a. 77 26 n.a. 24 3 government (1999) expenditure) External debt n.a. 60 138 55 41 n.a. 71 28 (2000: % of GNI) n.a. = not available. Note: GNI = Gross national income. Source: World Development Indicators and Global Development Finance. issues and possible lines of action would bene- reviews, individual and group interviews, and fit from further evaluative research. questionnaires. A local reference group helped guide the process in each country. Partly owing Each case study used the same set of evaluation to reference group inputs, and partly to time and questions and employed several instruments budget constraints, the precise content of the to obtain information, including document information gathering instruments as well as the 4 coverage of geographic regions and socio-eco- 1996; Lancaster and Wangwe 2000). Some nomic groups varied from country to country. remedial action was clearly needed. Learning and capacity development was one of three objectives of the evaluation. This objec- In response, the donor community launched tive was pursued indirectly through the refer- several programs to enhance the effectiveness of ence groups and workshops held in each development aid: the Heavily Indebted Poor country, as well as through the employment of Countries (HIPC) initiative; the International local researchers on the case study teams and for Development Goals, included in the DAC 1996 the surveys of government-donor relations that statement, Shaping the 21st Century, and the focused on aid transaction costs. later UN Millennium Development Goals; and related efforts to improve aid coordination through the Development Partnership Forums Background: in the OECD's Development Assistance Committee and through the UN Development Evolution of the CDF Assistance Framework. In the mid-1990s, concerns were growing about Thinking about a new paradigm for aid and how aid was used and managed, and about the development was also gathering force at the disappointing impact it was having. The con- World Bank. In 1998, the Bank formed a cerns were widespread--at the World Bank and Partnerships Group to develop the concept of other multilateral agencies, and among bilateral partnership and how it should be implemented. aid agencies, nongovernmental organizations Ideas for improving the management and (NGOs), and developing country governments. impact of aid aimed at poverty reduction began After a decade and a half of structural adjust- to appear in speeches by President Wolfensohn ment, there seemed to be too few positive and and senior Bank staff. In Wolfensohn's address sustainable results, especially in Sub-Saharan to the World Bank Board of Governors at the Africa. Criticisms were mounting, particularly annual meetings in 1998, he invited countries among NGOs, that aid-supported adjustment to pilot a new way of doing business with programs were at best ignoring the poor--and at the Bank and other development agencies, worst further impoverishing them (see Jolly, and in January 1999 he proposed the concept Correa, and Stewart 1987). formally as the Comprehensive Development Framework (CDF). Others argued that too singular a focus on adjustment and growth neglected the funda- The CDF is based on the assumption that mental goal of poverty reduction, broadly all development actors (government, multi- defined to include participation, freedom, and laterals and bilaterals, civil society, and private empowering the poor and excluded (Sen sector) play a part in poverty reduction 1999).1 It was also becoming clear that the and equitable, sustainable development. The many agencies and international organizations CDF has four cardinal principles--a Long-Term, working in developing countries taxed rather Holistic Development Framework; Results Orientation; than strengthened the capacity of recipient Country Ownership; and Country-led Partner- governments. The poor coordination of donors ship--each of which reflects on past develop- merely added to the challenge of making devel- ment assistance shortcomings and presents an opment effective (van de Walle and Johnson approach for improvement. Development strategies should be comprehensive ments and their citizens will be more com- 5 and holistic, and shaped by a long-term vision. mitted to seeing them through. Past emphasis on short-term macroeconomic stabilization and balance of payment pressures Recipient countries should lead aid manage- overwhelmed longer-term structural and ment and coordination through stakeholder social considerations (such as expanding and partnerships. Donor-recipient relation- improving education and health facilities, ships should be actively managed by maintaining infrastructure, and training a new the recipient country as a partnership and generation of public officials). Development not dominated by donor preferences. frameworks should no longer focus only on Partnerships built on mutual trust and short-term macroeconomic issues but should consultation can improve aid coordina- also embrace social and structural issues in a tion and reduce the inefficiencies, long-term vision for society. asymmetrical power relationships, and tensions of donor-led aid initiatives. After a decade and a At the heart of the CDF are the assumptions that the content of aid-funded activities is half of structural important for poverty reduction and that the adjustment, there way aid is delivered has an important influence on its effectiveness. Governments and interna- seemed to be too few tional aid organizations needed to collaborate positive and sustainable far more effectively if aid were to fully realize its potential in helping reduce poverty in the results, especially in world. Although largely promoted by the Bank, the CDF was not intended to be a Bank product. Sub-Saharan Africa It was not explicitly linked to Bank lending, and there was no conditionality attached to it. Development performance should not be meas- None of these individual elements is new; what ured by inputs and outputs, but assessed by out- is unprecedented is that World Bank leadership comes and impacts, by results on the ground. The brought these four principles together in a traditional emphasis on disbursement levels common, codified framework for poverty reduc- and project inputs has measured resource allo- tion and vigorously promoted that framework as cation and consumption. What really matters an organizing principle to inform its work and to is impact on people and their needs. coordinate with other aid agencies and develop- ing country governments. Development goals and strategies should be "owned" by the country, based on broad citizen participation in shaping them. While donor- Putting the CDF driven aid delivered under structural adjust- ment was sometimes effective, in many cases in Practice painful and lengthy adjustment measures were eventually undone. When countries In 1999, 13 countries volunteered to pilot have greater say in shaping reforms, govern- the CDF approach: Bolivia, Côte d'Ivoire, the 6 Dominican Republic, Eritrea, Ethiopia, Ghana, In turn, it has influenced the way in which sub- Jordan, the Kyrgyz Republic, Morocco, sequent global initiatives such as the Romania, Uganda, the West Bank and Gaza, Millennium Development Goals (MDGs) and and Vietnam.2 Soon thereafter, many other the Monterrey Consensus goals have been countries undertook PRSPs. PRSPs were based framed. A Regional example is the New Part- on CDF principles and were introduced as part nership for Africa's Development (NEPAD), of the Bank-Fund terms enabling access to the launched in October 2001, which aims to "use Enhanced HIPC Initiative. The pilot period joint responsibility, collective action and peer closed in September 2000 and was confirmed in process, and to develop and promote standards, January 2001 by the recommendation of the whether of governance, accountability or Bank's Board that the CDF be mainstreamed to sound economic management" (Elbadawi and all countries. Gelb 2002). The CDF was somewhat controversial (both In 1999 the Bank set up a CDF Secretariat as inside and outside the Bank) when it was first the focal point to promulgate the CDF inside introduced. There was particular concern that the Bank and globally, provide technical sup- it was an attempt by the World Bank to impose port, monitor progress in implementing CDF another framework or set of conditionalities on principles, and disseminate good practice recipient countries and other donors. At the through workshops, field visits, and regular same time, many global initiatives were emerg- Learning Group meetings with country directors ing that were highly compatible with the and the Board. When, in 2001, the Bank's tenets of the CDF. Eventually most donors and Board decided the CDF principles should recipients came to support the CDF, particu- guide the Bank's work in all client countries, larly as it underscored their own inclinations or the Secretariat extended its purview to include policies (see box 1.1). all low-income countries, including those that have opted to produce PRSPs, and a Several developing countries had already begun wide range of middle-income countries.3 It has to formulate their own national visions for also established monitorable indicators for poverty reduction and enhancing the develop- implementation of the CDF principles, which it ment effectiveness of aid, and for making glob- tracks and reports periodically. alization more sensitive to the requirements of dealing with poverty in their countries. A number of these initiatives pre-date the CDF (for example, national consultations in Bolivia, The CDF Principles the Health Sectorwide Approach [SWAp] in Ghana, the Poverty Eradication Action Plan The CDF weaves together four basic principles [PEAP] in Uganda). These initiatives emphasize to pursue the fundamental goal of poverty (1) the need for donors to change their behav- reduction (working definitions of the CDF ior and (2) that development is, above all, the principles are presented in Annex 3). Each of product of the visions and actions of the aid- the CDF principles is a response to a problem recipient countries and societies. In sum, the or "system failure" in the delivery of develop- CDF quickly became part of development dis- ment assistance. A brief discussion of the course; it reflected emerging thinking and problems each principle is meant to address is pioneering efforts in some developing countries. found below. Box 1.1. The CDF Principles--Antecedents and Support in the Development Community In his proposal for the CDF (January 19, 1999), James Wolfensohn lays out the four CDF principles, giving most atten- tion to the long-term, holistic framework; country ownership; and country-led partnership. The CDF matrix is used to link the holistic framework with country-led partnership. His Annual Meeting speeches contain precursors of the CDF principles. In The Challenge of Inclusion: Annual Meetings Address (Hong Kong, September 27, 1997), he insisted that the (recipient) government and people be in the driver's seat and that development "cannot be donor-driven." And in The Other Crisis: Annual Meetings Address (Washington, DC, October 6, 1998), he foreshadowed the CDF by calling for a "new development framework" that would involve balance, ownership, and participation. The following selective list briefly annotates several papers, speeches, and events that anticipate or support CDF principles. Shaping the 21st Century: the Contribution of Development Cooperation. 1996 (OECD Development Assistance Committee). Advances several tenets closely related to CDF principles, including partnership, local ownership, and International Development Goals (IDGs--subsequently absorbed into the MDGs). Eliminating World Poverty: A Challenge for the 21st Century: UK White Paper on International Development, 1997. New policy statement for British bilateral aid set poverty reduction as a goal and proposes partnerships with poorer countries, with other donors and agencies, and with the UK private and voluntary sectors. Commitment made to the IDGs. Partnership for Development: Proposed Actions for the World Bank, 1998. Prepared by the Partnerships Group at the Bank, this paper laid out the key challenges and actions required to transform the World Bank's partnership culture. The paper anticipated this aspect of the CDF. Towards a New Paradigm for Development Strategies, Policies, and Processes. 1998. Joseph Stiglitz (Prebisch Lecture, UNCTAD, Geneva). Conceives development strategy as vision for transforming society over 10-to-20 years, with ownership and participation as key ingredients. Making Partnerships Work on the Ground, Stockholm Workshop, 1999, held in Stockholm in August 1999. Strong statements by the President of Tanzania and the Permanent Secretary of Finance and Planning of Uganda laid out the main requirements for effective country-led partnership. European Commission Policy of Development Cooperation, 2000. The guiding principles of the policy include country ownership of the development process and increased attention to the social dimension of development. The agreement between the European Union and the African, Caribbean, and Pacific States is fully consistent with these principles. "Partnership 2000"--Denmark's Development Policy and Strategy, 2000. States that cooperation will be based on partner country strategies, plans and budget; with an emphasis on sector program support and cooperation between central/local governments and the private sector and civil society. NEPAD--New Partnership for Africa's Development, 2001. NEPAD is supported by African Heads of State com- mitted to a "vision and program of action for the redevelopment of the African continent." It stresses African ownership, initiative, responsibility, and partnership with external partners who accept the NEPAD agenda. The "Monterrey Consensus," 2002. At the Conference on Financing for Development, commitments were made by developing countries to good governance, by developed countries to increased aid, and by all to poverty reduction and mutual accountability and responsibility for results. Canada Making a Difference in the World: A Policy Statement on Strengthening Aid Effectiveness, 2002. This new policy statement of the Canadian International Development Agency stresses comprehensive development approaches, local ownership, improved donor coordination with the recipient country in the lead, and a results focus. High Level Forum on Harmonization, 2003. Rome, February. High-level bilateral, multilateral, and recipient coun- try representatives reviewed progress toward harmonization of donor procedures and practices and agreed on next steps, including a status review in early 2005. consideration. This is to provide the basis for gov- 8 Long-term, Holistic Development Framework ernments to set priorities and sequence inter- ventions according to national budget Poverty reduction is a process of economic, constraints and capacity. In short, starting with social, and political change extending over a comprehensive vision would modernize the decades or longer. As a broad goal, it is shared by planning process, moving away from the "silo" all aid donors. However, the budgetary and bal- thinking and wish lists approach of the past ance-of-payments pressures of the 1980s and (where national development plans were simply beyond tended to overwhelm longer-term con- a collection of unrelated sector plans detached siderations, such as expanding basic services, from any real budget process). It would encour- maintaining infrastructure, and training a new age consideration of intersectoral linkages generation of public officials. The holistic prin- (which, because they cut across the mandates of ciple of the CDF was an attempt to rebalance line ministries, are likely to be missed through the heavy emphasis of the international finan- normal planning). cial institutions (IFIs) on macroeconomic stabi- lization and structural adjustment over the last The holistic principle two decades. While essential for long-term growth, the attention to macroeconomics neg- of the CDF was an lected other critical aspects of development, including the institutional (the governmental, attempt to rebalance legal, and financial systems); the human or social the heavy emphasis... (the education and health services essential to long-term poverty reduction); and the physical on macroeconomic (water, sewerage, energy, transportation, and communications infrastructure, environment). stabilization and Also neglected were the microeconomic ele- structural adjustment ments--growth and productivity in agriculture, industry, and services that are keys to long-term poverty reduction. Sometimes priorities are obvious, as in a macro- economic crisis. But more commonly, effective The long-term holistic principle of the CDF was priority setting requires informed analysis of intended to correct this imbalance. The objec- alternatives and a political process to adjudicate tive is not to provide a long-term blueprint that competing claims. Under the CDF approach, is to be followed, as in traditional development priorities would be set and hard choices would planning. It is to inject appropriate considera- be made against a real budget constraint, and tions of the long term into the business of linked to a multi-year expenditure framework. preparing a practical program with a shorter- And this would provide all aid donors and recip- term (three-five year) horizon. ients with a common vision and structure to shape their strategies, policies, and programs.4 The core idea behind the "holistic and compre- hensive" aspect of the CDF is that all elements Results Orientation affecting a country's development (and the The emphasis on results was born of the concern attendant investment options) should be put on that development planners focused too much on the table, side-by-side, and given equal chance for managing inputs and outputs--to the neglect of outcomes and the real needs and well-being of izations, including the World Bank. Second, the 9 clients. A results orientation moves the spot- tools for evaluating the impact of aid improved light from activities and outputs to goals. Its during the 1990s, with better data sets and more practical manifestation is in setting and moni- sophisticated approaches to evaluation among toring targets, to provide policymakers with a major aid agencies. The findings of aid evalua- tool to measure progress and a framework to tions gave rise to concerns about the impact of aid structure rewards and budget allocations. on development, drawing more attention to the results of aid expenditures. Third, managing for The results focus is in response to experience of results spread throughout the business and devel- the past several decades, when the discourse on opment community in the mid-1990s, including foreign aid and development focused on the developing country governments. And a number size of aid flows. Bilateral donors were urged in of donor countries adopted "results-based man- the Organization for Economic Cooperation agement" systems to assess the performance of and Development's Development Assistance many of their activities, including development Committee (OECD DAC) to provide 0.7 per- spending. All of these trends contributed to cent of their GNP for foreign aid, with aid the prominence of "results" in the CDF. efforts judged against this standard. For bilateral and multi-lateral agencies, which feared that Despite the widespread acceptance that results their budgets would be reduced if they failed to and outcomes are important, the approach is obligate all funds available, rising levels of loans not without its critics (see box 1.2). Researchers or grants were signs of effective support of devel- and practitioners point to a number of ways the opment, falling levels as failures of commitment approach could backfire if misunderstood or and action. A critical report in the early 1990s misapplied (Hatry and Yansane 2002). For described this tendency in the World Bank as example, apart from being expensive, a results "an approval culture" and a problem for aid focus can distort public administration and effectiveness. This problem was not specific to create perverse incentives. It could, for example, the World Bank or bilateral agencies; disburse- lead to a bureaucratic proliferation of scorecards ment pressures also drove regional development or a focus on indicators rather than results. At banks, UN agencies, NGOs, foundations, and an operational level, managers might pay atten- other development institutions. The new focus tion only to indicators that affect their personal on the results of aid-funded activities--rather advancement. This could create a risk-averse than the size of aid expenditures--was initially investment climate in which official develop- seen as a correction of these tendencies. ment assistance (ODA) is used only in areas in which results are easy to achieve and measure This shift toward measurable outcomes was rein- quickly, rather than for higher priority invest- forced by several other factors. First, several UN- ments that are more difficult to implement in a sponsored world summits in the 1990s produced short period. targets for socioeconomic changes. A number of these targets were combined in the DAC report Country Ownership Shaping the 21st Century (OECD DAC 1996), Over much of the 50-year history of foreign aid, accepted by all DAC members in 1996. In and particularly since the mid-1980s (the era of September 2000, the UN adopted these targets, structural adjustment), donor governments and with some additions, as the MDGs, which have international organizations typically decided since been endorsed by other development organ- how aid would be used--designing, implement- 10 ing, and evaluating the projects and programs numerous currency devaluations in Sub- they funded. Structural adjustment programs Saharan Africa--at first strongly resisted by usually conditioned aid on the recipient govern- governments there--often turned out to be both ment changing its policies and/or institutions. effective and sustained by those governments. Recipient governments, eager for the aid But in some cases, especially with complex, resources, usually agreed to donor conditions painful, and time-consuming adjustment meas- without consulting their own people on the ures or complicated projects, this has not been projects or reform programs proposed by donors. so. More often governments agreed to reforms And while many donor agencies recognized the but did not implement them. Or they imple- importance of "ownership" and encouraged par- mented reforms and later rolled them back. Or ticipation, in practice they continued to retain they implemented the projects or programs most of the decisionmaking authority regarding poorly. A key problem was the lack of commit- aid programs and projects. ment to the reforms or projects that govern- ments (and their citizens) felt they had been Sometimes such donor-driven aid can prove coerced to adopt, had little say in shaping, or effective and sustainable. For example, the simply did not want. Box 1.2. Some Reservations about the Results Principle Although the results principle has gained acceptance in the international development community, it is not universally accepted as best practice. There is a debate over the utility of targets and results- based management. A central concern is that knowing only about outcomes is insufficient; also crucial is understanding the results chain and the sequence of inputs and events needed to achieve the outcome. Helleiner takes a stronger view, arguing that performance measurement approaches are irrelevant for low-income countries, and that the continuing effort to measure policy change and performance has been driven essentially by the needs of the donor community. He concludes that despite a strong desire to create new forms of aid partnership, nothing essential has changed in the degree of reporting required of aid recipients or the intensity of monitoring their performance by multilateral and bilateral donors (Helleiner 2000). Others, such as Elliot Berg, argue that that the results focus and the logical framework on which it is based are no longer suitable for today's environment, in which higher priority is being given to institu- tional change, capacity building, and governance--areas of intervention in which blueprint approaches, indicators, and logframes are unsuitable when compared with more flexible, systems-oriented approaches (Berg 1999). Chapman argues that public service organizations are complex, adaptive sys- tems that respond poorly to top-down centralized approaches, epitomized by results-based manage- ment (Chapman 2002). Advocates provide the counter-arguments that donors have to meet important fiduciary expectations, as do borrowers. They also argue that the pressure for results information stimulates accountability. Even if governments do not like an externally driven results orientation, civil society and parliament tend to support it. The fundamental intention of the ownership Table 1.2. Coordination Is Needed 11 principle is that the country and not donors with Many Actors should be in charge of its development; that identification of development goals and formu- Multi- Bilateral lateral lation of strategy should be done by the country Country donors donors Total itself and then supported by donors, and not the other way around. Gaining strong national own- Bolivia 17 10 27 ership for development policies means system- Burkina Faso 18 11 29 atic, broad-based stakeholder participation, Ghana 18 11 29 under government leadership, including civil Romania 14 8 22 society, the private sector, local governments, Uganda 18 11 29 and parliaments. The process and strategies Vietnam 26 5 31 developed are to be implemented with sustained Source: OECD/DAC data; multilateral estimates supplemented public support from top political leadership and by information from case studies. intellectual conviction by key policymakers, and strong links to institutions. own projects, priorities, administrative require- ments, and time horizons, which creates excessive The fundamental administrative burdens for government. intention of the The degree to which these agencies, often together with hundreds of NGOs, share infor- ownership principle mation, collaborate in the planning and execu- is that the country and tion of their activities, and engage the recipient government and peoples in these activities not donors should be varies considerably, but is seldom satisfactory (see Chapter 2 for a discussion of this problem in charge of based on evidence from the case studies). its development Most developing countries, for example, have no overview of their ODA picture--who is working in which sectors/regions, total aid Country-led Partnership flows, expenditures. Nor can they predict how The lack of effective donor coordination has been much donors will commit or disburse from year a long-standing and pervasive dysfunction of the to year. Donors themselves seldom know what international aid system. There are simply too others are doing, even in the sectors or regions many uncoordinated actors implementing pro- where they are working. This state of affairs saps grams and projects within a given country, leading the energies of overburdened recipient govern- to a host of inefficiencies and problems for aid ments and frequently exceeds their capacities. recipients. Table 1.2 shows that 20 to 30 aid agen- And when recipient governments do not effec- cies are active in each of the case study countries.5 tively manage their many sources of aid, donors And in some other African countries, more than tend to set the aid agenda. The CDF was pro- 30 aid agencies are operating--this is not neces- posed as a way to address this problem--it would sarily a problem except that each agency has its be a framework for sharing information, 12 planning and implementing activities more lar development thinking during the 1990s, but effectively, and reducing duplication. It would they emerged from slightly different starting also help recipient governments take a greater points. The CDF was basically concerned with lead in shaping and managing their aid--and poverty reduction and the processes of develop- thus in "owning" it. And the CDF recognizes ment, while the PRSP was primarily intended the important role other development actors-- to provide a mechanism to link debt relief to such as civil society, NGOs, and the private poverty reduction. A series of guidelines and sector--play in securing country ownership. statements issued by the World Bank and the IMF over a six-month period following the launch of the PRSP made it clear that the PRSP The Relationship Between should apply the principles of the CDF and that the two initiatives should be mutually reinforc- the CDF and the PRSP ing. Through increasing alignment of external assistance to the PRSP, both national and inter- As the CDF was being piloted, a new vehicle-- national programs would become subject to the the PRSP--was developed to enhance the influence of CDF-type principles. implementation of the HIPC initiative. Intend- ed as the government's own medium-term devel- All the early PRSP documentation contained opment strategy and expenditure plan, the PRSP specific references to the CDF and was entirely would direct resources released through debt consistent with its principles. Three key papers reduction toward poverty reduction. At the same set out the approach,6 stressing the importance time, the PRSPs would embody CDF principles of country ownership: broad-based participa- and further their implementation. In short, the tion: medium- and long-term goals for poverty PRSP was to be an action plan for the CDF and reduction and appropriate targets and indica- was to provide governments with the incentives tors: and integration of macroeconomic, struc- to adopt CDF principles in their development tural, and sectoral policies.7 planning and cooperation with donors (see Annex 4[A] for a chronology of events leading The guidelines for the Joint Staff Assessments to the CDF and PRSP and Annex 4[C] for key of the full PRSP provide further reference to principles and instruments of the PRSP). the CDF principles. They stipulate that the PRSP is to be prepared by the government The PRSP was officially launched in September through a country-driven process including 1999, barely eight months after the CDF broad participation that promotes country own- launch. The early guidance, endorsed by ership of the strategy and its implementation, as Development Committee ministers at the well as partnerships among the government, September 1999 Annual Meetings of the World domestic stakeholders, and development part- Bank and the IMF, indicated that CDF princi- ners. The importance of comprehensive diag- ples would be embedded in the PRSPs. By nosis, a long-term perspective, and results October 2002, 20 countries had produced full orientation is also highlighted. PRSPs and a further 29 had produced interim PRSPs (I-PRSPs). Undoubtedly, the continuing influence of the CDF on the PRSP was largely the result of the There was initially some confusion over the dif- World Bank President's strong personal com- ference between the CDF and the PRSP. The mitment to the CDF. His close scrutiny of CDF and the PRSP both developed out of simi- progress with the CDF pilots throughout this period, his insistence that the PRSP be moni- donors would align their assistance with the 13 tored for compliance with the CDF principles, PRSP, thereby reducing transaction costs and and his frequent public statements about the the proliferation of donor conditionality.10 importance of the link between the CDF and the PRSP 8 sent a strong message to Bank staff The attachment of resources to the PRSP has to maintain this link. had two consequences for governments: it cre- ated pressure to meet the deadlines associated Differences Between the CDF with these resources and a desire to secure a favorable assessment of the document by Fund and the PRSP and Bank staff members. Although these may Despite having the same guiding principles, the have been self-imposed pressures, they had the CDF and PRSPs have important differences. potential to undermine some of the CDF princi- These differences and the tensions they some- ples. Highly indebted countries have strong times create in practice are discussed below. incentives to complete the PRSP process as quickly as possible because of the built-in time Conditionality and resources lags before debt relief can be obtained.11 The The CDF was not linked to Bank lending, and interim PRSP (I-PRSP) was introduced as a there was no explicit conditionality attached. It device to dilute this time pressure, but interna- was an approach that governments and other tional demands for rapid results from the stakeholders could voluntarily implement as Enhanced HIPC initiative put pressure on Fund they wished (however, some argue that because and Bank staffs to bring the majority of poten- the CDF was a high profile World Bank initia- tially eligible countries to decision point during tive, developing countries assumed an implicit 2000.12 As a result, many I-PRSPs were rushed, link to Bank resources if they adopted the CDF and were based on existing policy documents approach). The CDF did not propose a single rather than re-thought in a holistic manner and model or a standard approach (apart from early prioritized according to real budget constraints. suggestions to use a CDF matrix as a coordinat- ing tool). Governments were invited to offer Principles and action plans their countries for monitoring for a pilot period, The CDF is described as a set of principles, and there were no guidelines or timeframes. The processes, or mechanisms, rather than a pro- operationalization process would depend on gram for action. It is "an approach," a "way of local context and priorities. doing business." But relatively little is said about policy content, mechanisms for imple- The PRSP has resources attached to it, but as a mentation, or links to the budget. The CDF country-owned process, it does not incorporate argues for a long-term holistic framework and conditions as to its content. It is a new form of results-based targets but does not specify crite- social contract with donors for the production of ria for them. It promotes participation but does a "credible framework for concessional lending" not indicate how this should be organized. in return for debt relief, an IMF Poverty It is not, in other words, an action plan. But Reduction and Growth Facility (PRGF) pro- the PRSP is. As an instrument for policy imple- gram and IDA assistance under the Bank's mentation, it must detail policy priorities, Country Assistance Strategy (CAS) and the targets and indicators, costings and financing Poverty Reduction Support Credit (PRSC).9 plans, and procedures for monitoring. Another aspect of this contract is that as the Mechanisms for incorporating the strategy into single framework for government policy, all government structures and procedures as well as 14 institutionalizing sustainable consultation tries are fully adopting the longer 10­15 year processes are also required. planning horizon implied by the CDF. Technical standards Country-led partnership The guidelines for preparing Joint Staff The donor-government relationship is a central Assessments (JSAs) of the PRSP documents sug- concern of the CDF, suggesting the need for gest criteria of good public policy in relevance, changes in the interpersonal and institutional consistency, affordability, consultation, and behavior of donors. It advocates greater open- accountability. They avoid prescribing PRSP ness, trust, transparency, and flexibility; better content in order to avoid undermining country information sharing; and more collaborative ownership. However, the detailed action plans of work. It argues for government leadership of the the PRSP take it into technical areas where it is policymaking process, and of donor coordina- difficult to distinguish generic principles of good tion and alignment of programs to government public policy from judgments about policy con- policy. The CDF provides both norms for donor tent. There is also ambiguity over whether JSA behavior and a simple framework for donor advice is directed to governments as well as to coordination (the CDF matrix). the Fund and Bank Boards. The tension between ownership and technical standards is never far from the surface in PRSP dialogue, despite inter- The CDF provides both nal guidance to minimize it. norms for donor behavior The PRSP can be credited for a real shift toward and a simple framework country ownership of policy and planning, with for donor coordination donors relatively disengaged during the prepara- tory process. But the informal advisory dialogue that accompanies this process in the run up to completion of the document has sometimes The PRSP documentation and guidelines been intrusive.13 The difficulties donors have (including for JSAs) say relatively little about stepping back from micromanaging country government partnership with donors and do not policies was raised by both donor agencies and provide a "code" for government-donor rela- NGOs in the PRSP comprehensive review.14 tions (though a country may choose to empha- size this aspect--Uganda's PEAP, for example, Timeframes has a volume devoted to partnerships). The Despite the exhortation for long-term visions in emphasis on "country ownership" has implied CDF and PRSP documentation and the exis- donors stepping back while the PRSP is being tence of long-term sector plans in some coun- prepared. Nonetheless, the PRSP provides a tries (such as 10-year health and education concrete basis for partnership that has not strategies in Burkina Faso), many PRSPs have a existed before.16 It has given added impetus to three-year planning horizon, which broadly cor- the debate about donor coordination and har- responds to the Bank-Fund cycles.15 Some monization already promoted by the CDF, by countries have argued for longer timeframes to providing detailed objectives, targets, and budg- coincide with their national planning or parlia- ets to which donors are encouraged to align mentary cycles (say, five years). But few coun- their programs (World Bank 2002a, 2001b). C H A P T E R T W O Implementing the CDF Principles: 17 The Evidence The CDF proposes that a country's development porated a longer-term holistic perspective. We framework take a long-term view, that it be then turn to assess progress in adopting a results holistic, that it be country-owned, linked to meas- orientation. This is followed by a discussion of urable results, and that donor involvement should country ownership and country-led partnership. be a country-led process. In most cases, this implies a significant change from the status quo. The six country case studies and four thematic Long-term, Holistic papers examined for this evaluation revealed Development Framework that a variety of processes and instruments have been employed to implement these principles. This chapter looks at whether these processes Introducing a Long-term Perspective and instruments in fact led to the design and The CDF approach, as embodied in the PRSP implementation of development frameworks, process, emphasizes that a country's long-term partnerships and behaviors that are consistent goals should influence the design of its shorter- with the CDF characteristics described above. It term development program (3­5 years) and that also considers whether the characteristics them- these shorter programs should be tied to the selves appear to have strengthened the policy national budget and consistent with macroeco- reform process and development outcomes. nomic goals. The task of introducing long-term considerations into the rough-and-tumble The answers to these two questions can differ. process of short-term decisionmaking is a diffi- For example, it is possible that CDF practices cult one, but the case studies point to some ini- will be effective in promoting the proposed tial successes. changes, but that the changes have proved ineffective in strengthening policy reform All five PRSP case study countries, four of and improving development outcomes. It is also which were also CDF pilots, had prepared long- possible that the CDF practices are ineffective term strategies prior to the introduction of the in promoting the proposed changes, but that CDF in l999--preparation of a long-term vision where these changes have come about and development framework began in l995 in anyway--for reasons other than CDF prac- Ghana and Uganda, and in 1997 in Bolivia. tices--they have improved policy reform and Burkina Faso began preparing long-term sector development outcomes. plans in the late 1980s that became the build- ing blocks of its PRSP. Vietnam's poverty reduc- This chapter is divided into four sections; we tion strategy is embedded in a 10-year look first to evidence of whether development socioeconomic development strategy covering frameworks in case study countries have incor- the period 2000­2010. Becoming a CDF pilot 18 country gave additional legitimacy and Box 2.1. The CDF, PRSPs, and momentum to these efforts, but they were country-driven rather than donor initiatives. Political Change There was initial concern among observers that When the PRSPs were introduced, planners did PRSPs and CDF approaches could survive only not confine their planning horizon to the three- in states such as Uganda or Vietnam, which year frame of the PRSP. Nor has the horizon have not been subject to frequent changes of been artificially set at a uniform five years, as in leadership. Early evidence from the case studies traditional development planning. To the shows that this is not necessarily the case. extent that any horizon is proposed, it is pro- Bolivia and Ghana have recently had demo- vided by the MDGs, which set a range of objec- cratic changes of government. In these cases, tives for 2015, well beyond the timeframe for the new governments did not completely aban- practical political decisions, but appropriate for don the PRSP and CDF initiatives begun under previous regimes, though they made changes the discussion of structural transformation. in name or substance to differentiate these initiatives from programs trademarked by their This is important in several respects. Bringing predecessors. For example, in Ghana, the new in the long-term perspective can inform and government confirmed its commitment to the discipline short-term decisionmaking, speed PRSP and poverty reduction, but chose to reform, and provide a buffer for (unpopular) change priorities, revising the PRSP document short-term policies that need to be adopted. For to include more emphasis on accelerating example, Romania's long-term goal to gain growth. Such freedom to change priorities, admission to the European Union is concrete, essential for the democratic process to be explicit, and broadly internalized, which meaningful, indicates that PRSP/CDF processes enables the country's leaders to galvanize soci- have not created straitjackets for national ety more easily for the substantial changes that policy, but rather can be made to serve a it needs to make. And short-term costs of variety of political programs. change borne by some groups become more palatable if set in the context of broad-based improvement for many other groups--to a goal increase for the business community. Since no widely recognized as desirable. political party can make credible long-term commitments, the CDF advocates having a The repeated articulation of the long-term long-term vision that can help guide strategy vision is also important for sustainability, par- through successive short-term political cycles-- ticularly where country leadership changes reg- where rival programs for the short term exist, ularly. The day-to-day political process of policy but within a shared, broad, long-term frame- reform is seldom consensual. Indeed, the role of work. Under this scenario, new governments democratic political oppositions is to offer would change some policies without systemati- alternatives to what the government is doing. cally undoing the country's longer-term goals But if these oppositions reverse plans and and reforms. Box 2.1 illustrates how this process reforms significantly when they become gov- has unfolded in Bolivia and Ghana. However, it ernments, the confidence of the public in the is still early, and the Ghana case study argues consistency of government policy will be that in countries with a four-year electoral eroded. In consequence, the pace of develop- cycle, the feasibility of a long-term (that is, at ment could slow, and policy uncertainty least 10-year) vision statement necessitates cross-party agreement and, therefore, a consen- focus on macroeconomic and structural reforms 19 sus-building political culture and institutions. of the last decade. For countries not on track to join the EU, the A medium-term expenditure framework international community is attempting to pro- (MTEF) is a critical instrument for implement- vide a substitute long-term vision around the ing longer-term plans; it requires that priorities attainment of the 2015 MDGs. Although the be costed, linked to the annual national budget, MDGs have not yet been fully integrated into and embedded in a multi-year expenditure donors' and recipients' visions, they do have the framework, consistent with the maintenance of potential--once translated into country-specific macroeconomic stability. Burkina Faso, Ghana, milestones--to put the CDF principle of results and Uganda have systems that link develop- orientation into practice. The launch of NEPAD ment plans to a budget and MTEF. But only in is a somewhat analogous African initiative to Uganda is that link fully operational and rigor- bring peer pressure to bear in support of a long- ously enforced. This is the most important term vision. Ownership of a long-term vision by shortcoming related to implementation of this the Region is used to reinforce commitment at principle--national plans have little opera- the level of individual member nations. tional meaning if they are not linked to a hard budget constraint and expenditure framework. Introducing a Holistic Approach Setting priorities is particularly important where public resources and government capacities are It was often difficult limited: not everything can be done, and cer- for the PRSP, as a tainly not all plans can be realized at once. Effective priority setting requires informed government-led, analysis of alternatives. The CDF's emphasis on a holistic strategic approach is meant to encour- consensus-based age informed prioritization, more balanced process, to address development investments, and the recognition of interconnections between sectors. The evalu- politically divisive issues ation therefore considered the extent to which the case study countries' development plans and strategies were more holistic, sectorally bal- anced, and prioritized than previous versions. An examination of the PRSP process--as the CDF's main implementation vehicle in low- Evidence from the case studies and the thematic income countries--showed that it helped open study on long-term holistic planning prepared up the process and led to strategies that were for this evaluation (Ali and Disch 2002) show more sectorally balanced than in the past that with some exceptions, country develop- (Bolivia, Burkina Faso, Uganda's PEAP, ment strategies have become more balanced, in Vietnam). that consideration was given to social, eco- nomic, and institutional aspects of develop- It was often difficult for the PRSP, as a govern- ment. And the poverty emphasis of the PRSPs ment-led, consensus-based process, to address constitutes a departure from the more exclusive politically divisive issues, even when these were 20 direct causes or symptoms of poverty. The that previously overlooked issues are consid- PRSPs did not directly address key development ered. As stated by a senior government official issues such as coca, land tenure, social exclusion of Uganda: in Bolivia, water policy in Ghana, and the role of state-owned enterprises in Vietnam. In The CDF liberated us from the excessive focus Bolivia, opposition parties and civil society crit- of the IFIs on macroeconomics and structural icized the PRSP because it did not address "hard adjustment. ...What is new and comprehen- poverty" with a strategy to improve rural pro- sive about the CDF is that you put all your duction and growth. In Ghana, some stakehold- problems and constraints on the table. You dis- ers argued that the early drafts of the PRSP were cuss what can be done to overcome those prob- too dominated by the social sectors, while eco- lems and constraints. And then you see to nomic growth had been abandoned. When what extent the available resources can allow more growth-oriented content was eventually you to implement the proposed interven- inserted by the new administration, poverty tions...and you sequence the actions as appro- advocates complained that the pendulum had priate given the resource constraints. You say swung too far the other way and the social focus what government can do and what it won't do had all but evaporated. because others can do it better. And the things that can't be afforded should be forgotten. NGOs and civil society groups in several case study countries argued that the poverty policy Uganda is an example of where the process of debate generated through the PRSP was too prioritization is well established and regularly narrow. In Ghana, several NGOs complained involves nongovernmental stakeholders. But that the government was reluctant to engage even in Uganda, segments of the private sector with the more radical critics of the mainstream and civil society, particularly in outlying dis- political and economic model. In Burkina Faso, tricts, complain that they are not being invited Ghana, and Uganda private sector actors com- to participate in these processes.17 plained that participatory processes associated with the PRSP were unrepresentative because Promoting a Longer-term the development NGOs, often very numerous, and Broader View are overwhelmingly interested in a social agenda, and issues of interest to private sector CDF processes will need to evolve and mature to development were thus underrepresented. create space and capacity for debating divisive Other observers noted that because the contest issues, resolving conflicts, and illuminating cross- seemed to be between macroeconomic and sectoral linkages. In some countries (such as social investments, microeconomic policy issues Ghana, Uganda, and Vietnam), the CG process pertinent for enhanced growth (such as improv- has begun to serve this purpose. In the same ing the investment climate) were overlooked. countries, SWAps have proven effective vehicles for sector debates (health in Ghana, several Yet, in theory and design, the CDF/PRSP sector working groups in Uganda, and the processes should create the opposite effect: Forestry Sector Support Program in Vietnam). insisting that all competing needs and ideas be brought to the table and given equal consider- Implementation of results orientation--another ation (and that multiple parties be invited to CDF principle--can reinforce cross-sectoral sit at the table) widens the debate and ensures interconnections. How does this work? Line ministries are organized not by results but by Results Orientation 21 inputs. Many results, such as better health, require inputs from a wide range of ministries-- for example, water and education as well as Pre-CDF Efforts health care. So, the more that objectives are The case studies reveal that a number of coun- specified in terms of outcomes and final results, tries had begun to adopt a results approach prior rather than inputs, the more the development to the CDF initiative. This trend is closely framework will need to cut across sectors to linked to the pressure for greater accountability identify the variety of inputs needed to achieve of service providers to the public. One manifes- a given result. For example, in Uganda, hygiene tation of this trend is the use of Participatory education is an important component of the Poverty Assessments in Uganda. The assess- water and sanitation strategy, and the Ministries ments link the perspectives of poor people to of Education and Health are active participants, policy formulation, and since the expectations along with the Ministry of Lands and Water, in of the poor are framed as outcomes, this has the Water SWAp. helped shift monitoring and evaluation per- spectives. In Ghana, although the basic machinery to foster a results orientation is still The crucial next step-- weak, the health sector pioneered one of the first and most innovative SWAps in Africa. to link these plans to Performance data are used to review the sector's programs annually and set the workplan for the the discipline of the following year. annual budget and As early as 1997, Romania's Ministry of Health medium-term and Family decided to move to program-based expenditure framework budgets. Why? It was done to direct national resources into preventive care in addition to curative health programs and to give more weight to funding public health centers in addi- In sum, the case study countries have made tion to hospitals. Physical output indicators, by progress in adopting a long-term planning per- program, were added in 2000. Another bench- spective and have had success in generating mark involving the early adoption of CDF-like development plans from a more comprehensive principles is the Conditionality Reformulation consideration of options. However, except for Test Exercise, initiated in Burkina Faso in Uganda, these countries have yet to take the l997. The test was intended to build a consen- crucial next step--to link these plans to the dis- sus between donors and the government on a cipline of the annual budget and medium-term common set of performance indicators to be expenditure framework. Though the PRSP used as a basis for disbursement of financial process has helped to broaden the development assistance, preferably in the form of budget sup- debate in case study countries, it has not been port. The government provided all the finan- able to easily address or negotiate contentious cial information necessary for donors to (but legitimately contestable) issues that would understand implementation progress and apply ideally be resolved through the political trade- their disbursement criteria, while agreeing to offs involved in budget decisionmaking. support the government's economic manage- 22 ment program and to work on harmonizing PRSPs to be concrete progress on the part of their disbursement procedures. both donors and governments. Experience Since the CDF Constraints Despite these early initiatives, implementation The obstacles to institutionalizing the results- of the principle of results orientation has been oriented approach are significant, and this is the most elusive of the four CDF principles, and manifested in many different ways in case study arguably the most demanding. This is not sur- countries. A number of recipient countries prising as there are formidable obstacles on appear to have signed on to the results orienta- both the donor and recipient country sides, tion approach primarily to satisfy donors, and including insufficient investment in monitor- few have embedded the principle into the ing and evaluation, weak technical capacity, normal operations of government. There have inadequate incentives, poor statistical data, and been some attempts to operationalize the results institutional disincentives. Although the perspective government-wide, outside the donor results principle has been slower to catch on, domain,18 but most efforts are still limited pri- there has been notable recent progress: most marily to specific aid projects. donors now agree on the need for a focus on results and they are beginning to direct A common complaint from informants across resources to outcome-oriented efforts (box 2.2). case study countries was that donors often Many consider the emphasis on results in propose elaborate and complex monitoring Box 2.2. Results Orientation: Examples of Recent Achievements and Innovations Uganda's education expenditure tracking study was an extremely effective tool for building government accountability and it is now being replicated widely. The government also publicized results of its com- prehensive household survey that quantified improvements in poverty outcomes; public access to these data deepened support for the government's initially controversial pro-poor policies of the early 1990s. The Uganda Participatory Poverty Assessment Program sits within the Ministry of Finance, Planning and Economic Development and provides feedback on absolute and extreme poverty into the budgeting process on a timely basis. Results-oriented disbursements--based on targets--are increasingly applied at the central and local levels. However, these systems still need to be aligned and made fully operational. Vietnam created its own timetable and indicators for reaching MDGs. These fit better with Vietnam's development strategy, and in some areas are more ambitious than the generic MDGs. Civil society and the Catholic Church will take an active role in monitoring Bolivia's Poverty Reduction Strategy (which is replete with specific monitorable targets). Indicators will be developed and monitored at the municipal level. Bolivia's Institutional Reform Program requires ministries to sign results agree- ments (with specific outcomes such as staff reductions) and meet these goals before they can enter the larger reform program. Ghana's Health SWAp pioneered a results-based approach to performance measurement, and all donors participate with government in the intensive annual review of progress. approaches, with indicators that are unwieldy, being of the public. The PRSP process--with 23 hard to substantiate, and conform more to sufficient emphasis on MTEFs and monitoring donors' reporting requirements than to what is and evaluation--could build up systems to mon- needed to help the country manage national itor mainstream public programs. But efforts in service delivery. Most government systems are the case study countries did not extend far ill-adapted to take on more demanding moni- enough beyond PRSP sectors and programs to toring requirements in a short period--and this realize that potential. has led donors to establish enclave M&E sys- tems that exist separately from core govern- A recent internal review of a World ment processes, which do not build public Bank­financed capacity building project in sector capacity and are not sustainable. The Africa, for example, concluded that the project presence of so many donor-sponsored M&E was unable to deepen a results-oriented culture processes can cause confusion among govern- across the civil service primarily because the ment partners, especially since donors use dif- results-oriented management (ROM) methods ferent terms for the same concepts, promote and techniques were not adequately embedded contradictory techniques, or push indicators in routine government processes. It specifically that are internally incoherent. This situation is noted that ROM was not initially linked to exacerbated when donors add new targets or annual budget and medium-term planning monitoring requirements and do not factor in processes, systematic improvements in wages or the human and financial costs of collecting working conditions, and the strategic review additional data. process underpinning ministerial restructuring. Corruption was a serious obstacle in all of the The Ghana case study cautioned against setting country cases. Respondents believed that civil up special-purpose mechanisms solely to track service reform was required for the results prin- PRSP performance separately from existing sys- ciple to gain serious traction. tems: "Neither donors nor national actors should focus exclusively or excessively on the GPRS How Can Orientation to Results (Ghana's name for the PRSP). By doing so, they be Strengthened? risk sidelining or distorting other policy processes and broader institutional capacity development High-quality and timely data are critical to processes." Apart from being unsustainable, such effective results monitoring and management. mechanisms promote a "two-class system" of Because macro-level data tend to be unreliable monitoring--one for donor-funded programs and and outdated in developing countries, care one for the rest of the public sector. Further, should be taken when relying solely on these PRSPs do not address all of a country's problems, data to determine budget allocations.19 Most and critical sectors may go unmonitored when data collection efforts are now donor funded, scarce resources are devoted to the programs most raising concerns about how they will be sus- popular with donors. tained. A balance has to be struck between aiming for comprehensive systems and country Moving the results focus beyond the donor capacity. There is a need to settle on acceptable domain is important since the decisions made by structures that do not require excessive donor mainstream operating agencies (e.g., providing assistance nor overtax local capacity. In addi- health, education, water, or solid waste manage- tion, the right balance has to be struck between ment) determine the economic and social well- central and local monitoring and evaluation 24 (M&E) functions, because capacity is usually (Maxwell 2003). Finally, monitoring and report- much weaker at the sub-national level. ing is one of the areas where donors have made the least progress in harmonizing practices, and Much can be done to develop the technical the burdens on government are enormous. To capacity to carry out outcome monitoring by move forward, donors need to downplay indi- focusing resources on training and on deliber- vidual agency interests for more effective joint ately developing a skilled cadre of M&E staff. action. Results monitoring should be seen as a In current programs, there are many lost oppor- shared responsibility in which donors and gov- tunities for M&E capacity building. For exam- ernments each use the same information flows ple, in many countries data collection tends to for their individual purposes. In other words, be done by expert consultants and national governments and donors need to view country staff in the field, providing hands-on training. outcomes and the M&E process as a group prod- However, analysis of the data is usually done by uct, to which donors contribute with the coun- external consultants back in their home institu- try in the lead. tions, which deprives national counterparts of the opportunity to absorb more advanced ana- lytical skills, and thereby perpetuates the coun- Country Ownership try's reliance on external consultants for critical analytical functions. Moreover, coordination Change driven by domestic interest is usually among entities responsible for collecting, ana- more acceptable and more sustainable than lyzing, reporting, and disseminating results change induced by external sources. That is why information is weak, and few developing coun- the CDF emphasizes country ownership. That is tries have a coherent monitoring system to inte- not to say that donors should support whatever grate data collected from different sources. a government chooses to do. But when a gov- ernment initiates improvements in its policies and institutions, donors should be prepared to Donors need to change their policies and practices to permit and facilitate these improvements. Donors downplay individual would permit such change by leaving more space for domestic initiative than past practices of agency interests for more conditionality afforded. And they would facili- effective joint action tate it by encouraging and supporting processes of analysis and discussion that lead to more informed and balanced domestic decisionmak- ing, complemented by predictable financial sup- More thought should go to addressing the prob- port. The most effective change in practices has lems or "unintended consequences" associated been the shift from Policy Framework Papers, with the results-based approach. In this vein designed by the IFIs, to PRSPs, designed prima- Maxwell proposes to negotiate targets locally rily by governments, with inputs from other (apply subsidiarity), use qualitative as well as domestic stakeholders, a shift that has promoted quantitative indicators, reward genuine value country ownership. added, adopt the principle of shared responsibil- ity and mutual accountability, and in general Countries differ enormously in their degree of follow process rather than blueprint approaches governmental and societal ownership of CDF/ PRSP change processes. Since donors require a Planning and Economic Development (MFPED), 25 country to draw up a PRSP as a precondition for owing to its mandate to link the planning and debt relief, the process can be seen at one budgeting functions, has taken the lead in pro- extreme as a further imposition of donor power, moting the strategic investment plans of sector and thus be treated in a perfunctory way. At the ministries. The main mechanism for linking other extreme, countries can use the space cre- these functions has been SWAps that are not ated to genuinely take charge of their own only required to show clearly what each sector affairs. Ownership is best served when the will do to deliver the objectives of the PEAP, impetus and development of a PRSP initially but must also prioritize these expenditures to be comes from the recipient country (Ali and consistent with the MTEF. The budgeting Disch 2002, p. 28; Kanbur, Sandler, and process has increasingly become open and trans- Morrison 1999). Uganda's successful PRSP, parent, following a consultative process wherein issued in 2000, was essentially a rebranding of stakeholders, including civil society, donors, the Poverty Eradication Action Plan (PEAP), a NGOs, and line ministries are invited to partic- pre-existing nationally inspired strategy that ipate in the budget formulation, monitoring, was first published in 1997, following a two- and review process. Increased openness of year process of stakeholder consultations. By the budget process has encouraged some donors contrast, governments have in some instances to integrate their financing with the govern- launched national planning processes almost in ment's budget system and to commit their parallel with, but distinct from, PRSPs, suggest- planned financings early in the budget cycle. ing in these cases that the PRSP is seen as an This has allowed government to have greater exercise for a donor audience. For example, control over the budget and has increased the Vietnam originally had two seemingly inde- efficiency and effectiveness of expenditure allo- pendent poverty reduction strategies. The cation. Moreover, MFPED has also curbed the "national" Hunger Eradication and Poverty traditional habit of individual donors to exert Reduction (HEPR) program targeted disadvan- pressure and tempt line ministries into under- taged households and was being implemented taking projects centered on donor interests. It with government resources. The Interim PRSP has limited line ministries' authority to accept was an initiative of the World Bank/IMF and donor-funded projects that either fall outside of emphasized a comprehensive vision. These pro- sector investment plans or fail to directly grams coexisted until the government ulti- address priority areas identified by the plans. mately produced a Comprehensive Poverty (See "Country-led Partnership," below, for fur- Reduction and Growth Strategy (CPRGS), ther discussion.) which serves as Vietnam's PRSP and builds on both the I-PRSP and the HEPR strategy. The experience of the six case study countries suggests that in the absence of broad-based soci- The case studies confirm that country owner- etal ownership, country ownership is expressed ship of strategy and reforms is more likely to through strong government ownership. For be sustained when, consistent with the CDF example, in Vietnam the government has a tra- country ownership principle, there is regular, dition of strong political control, so the CDF broad-based dialogue. Country ownership in approach of letting the government take the lead this respect has varied in practice among the is widely acknowledged to be "the only way to do case study countries. Closest to good practice business in the country." Where a donor seri- is Uganda, where the Ministry of Finance, ously disagrees with the government on a 26 political processes at center stage, whether or not Box 2.3. Quality of the Domestic they are influenced by international partners." Dialogue The quality of dialogue among central govern- How Should the CDF Promote ment and local government, civil society, and Country Ownership? the private sector leaves room for improvement in all the case study countries. The Role of Participation Documentation arrives late or not at all, The mechanism the CDF uses to encourage or is not easily understood, in Uganda country ownership is to give voice to diverse and Vietnam. interest groups in a systematic manner. This The PRSP (CSLP) in Burkina Faso is not well alone tends to produce a lobby for increased known outside the central government and spending rather than a process where tradeoffs higher provincial authorities. are made within budget constraints. The Ghana While the earlier main text of the PRSP case study notes that although the Poverty (GPRS) in Ghana was well known by the Reduction Strategy was nationally owned and populace, the new priorities inserted by consensual, it may have had a debilitating effect the new government did not involve on the making of hard choices. The aspirations broad consultation. expressed by the expenditure lobbies (through Consultative processes in Romania are CDF/PRSP processes) have to be reconciled nascent and fragmented. with the realities of a consistent medium-term macroeconomic framework, which is ultimately the responsibility of government. While the policy--as with the World Bank's assessment of process of broad consultation can yield benefits Vietnamese health care policy--the donor steps for all participants, problems can also emerge. back. Because it is evident in such a situation For example, several studies note the danger that the government rather than the donors will that CDF participatory processes could generate determine the development framework, when unrealistic expectations, and the Romania case the CDF approach is adopted, donors and the study noted the potential for participation to IFIs may disengage when they disagree with degenerate into "gripe sessions." Even the best country policy in a sector. In these circum- representations of sectional interests will under- stances, country-level staff would not be state the priority of policies that produce dif- expected to bend normal procedures to accom- fused nationwide benefits such as trade modate local realities. Noting that direct consul- liberalization and exchange rate reforms. Such tation in Bolivia is "undoubtedly useful as a policies are "orphans" in the jungle of interest supplement to the weak representative system, group lobbying, and so must be espoused by the where most of the population does not have an government itself (see box 2.3). effective voice," the Bolivia case study neverthe- less concludes that "country ownership concerns Development experience shows that a participa- the ability of a country's government to continu- tory process can be captured by vested interests ously balance the interests of all citizens and that use the opportunity to delay widely benefi- cannot be reduced to ad hoc participatory cial reforms. In such cases, the government processes. A more nuanced understanding of the needs to inform and guide a participatory ownership principle would place all the country's process, not simply listen. The practices intended to broaden country ownership beyond the government to explain the case for them to 27 the government through a wider participation of parliament, whose members, when taken as a society in national priority setting have had whole, do not primarily represent special inter- mixed results. In societies where parliaments do ests, and who should therefore take a holistic not exist or function ineffectively, the attempt view. Even parliament may not have an to create ad hoc measures of consultation has adequate understanding of the poor and margin- had only limited impact.20 And as observed in alized. Government can draw on the findings several of the case studies (Burkina Faso and of such sources as Participatory Poverty Uganda), in countries where parliaments do Assessments (PPAs) in making the case to par- function, ad hoc procedures risk bypassing and liament for programs and policies benefiting the undermining this key representative institution. poor. The Ghana case study concludes that Where a government is regarded by its citizens "there are possibilities of enhancing national as legitimate (normally established by a trans- ownership through the further deepening of parent democratic process), the government has democratic values, institutions and accountabil- the authority for reform. In such situations, ad ity within the political culture." hoc, non­government-led participatory meas- ures, inevitably unrepresentative, can be coun- Government processes are sometimes seen as terproductive. They may appear to question the "remote from the people." The widespread trend right of the government to lead a process of toward decentralization was expected to correct change, and they risk undermining sustainable this by increasing local participation. But as democratic institutions, most notably parlia- ments. As such, they may run counter to the Box 2.4. Perils of Counterproductive CDF principle of country ownership (see box 2.4). The Bolivia case study concludes that the Participation various ad hoc consultative mechanisms lack Several country studies noted the dangers of strategic clarity and that there is a consequent creating processes parallel to existing demo- "risk that perceived deficiencies in the political cratic institutions. In Burkina Faso, the chairman system will continue to lead to parallel mecha- of the National Assembly complained that the nisms instead of more important changes which Assembly was marginalized from the consulta- would enable the democratic system to function tion process for the CSLP (the PRSP for the more smoothly." country), with discussion being limited and rushed. Similarly, in Uganda members of parlia- But even if ad hoc processes were made more ment were quite explicit in their criticism of representative of sectoral interests, there is what they regarded as the excessive role of the an intrinsic problem in processes that bring "development NGOs." Their concerns were that these organizations were insufficiently represen- together representatives of such interests. Much tative, had little capacity, and lacked accounta- of the necessary reform agenda involves "public bility. They felt that these agencies were good" policies that produce small, individual deflecting both the government and the donors benefits distributed over many people, perhaps from a more direct engagement with parlia- at the cost of losses concentrated on small but ment, an engagement they would welcome. well-organized lobbies. No interest group will These observations from the country case stud- devote its efforts to promote such reforms, and ies echo a finding from the tracking surveys of some stakeholders will try to block them. The the CDF Secretariat: parliaments have been proper process for promoting such reforms is for unduly neglected. 28 noted in the Ghana case study, and to some within the society was perhaps insufficiently degree in all the case studies, the scope for par- thought through. The case for privileging an ad ticipation through decentralization has probably hoc assortment of social actors differs consider- been exaggerated. Local government structures ably between countries. Such an undertaking is are often in limbo, still too centralized to con- necessarily biased toward interests that have the nect with genuinely grassroots organizations, cohesion and the skills to make use of the but lacking in capacity and oversight. So, with- process. A strong reform process is indeed more out effective leadership by central government, likely to be sustained if the decisions for reform participatory processes tend to produce a are seen as legitimate. But the normal process of "lowest common denominator" consensus, in gaining legitimacy is through parliaments and which complex public good issues, such as elections. In countries that have credible repre- exchange rate policy, are crowded out by obvi- sentative institutions, evidence from the case ous special interests. For this reason a high studies suggests it is desirable for donors and degree of societal involvement in the PRSP is governments to work through such institutions, unlikely to foster effective change in the as advocated by the CDF country ownership absence of government leadership. principle, rather than to undermine them with ad hoc approaches. Thus there are important caveats for how the CDF should promote broader country owner- ship of the reform process: emphasized here is The CDF triggered the strategic choice between strengthening existing representative institutions and building consultation practices new, ad hoc consultation practices. that were new to How Can Participation Build--Not society and have set Undermine--Support for Reform? an example of how Several country studies noted the dangers of policy formulation can creating processes that are parallel to existing democratic institutions. In most countries, be more inclusive however, the existing processes for consulta- tion were quite limited, and the processes required by the PRSP widened the awareness A further potential tension in the emphasis on of development policy issues in the society. country ownership noted in some of the coun- Even where there was no PRSP, as in try studies is that development programs owned Romania, the CDF triggered consultation by a developing country government and soci- practices that were new to society and have set ety may still be subject to endorsement or an example of how policy formulation can be approval by the Boards of the IFIs. This tension more inclusive. is to a certain degree inevitable. Although country ownership may be necessary for the While the shift from IFI ownership of the reform success of a program, it evidently is not suffi- process to country ownership has been both cient. Governments and societies may on occa- considerable and beneficial, the attempt to sion opt for populist policies that appear broaden participation beyond government alluring but are known from the experience of other countries to run a high risk of failure. Donor and Recipient Incentives 29 While countries have the option to adopt such What are the incentives--and disincentives-- policies, the donors have the duty to protect for aid partnership? Both donors and recipients scarce aid resources from being wasted, and must share a common objective and have incen- they cannot be expected to provide financial tives to pursue it, under the leadership of the support in circumstances where either the effi- recipient, for the country-led partnership princi- cacy of policies is dubious or fiduciary assur- ple to work well. Unlike a corporate partner- ances regarding the use of aid are inadequate. ship, where the partners might be driven by a However, there is a major difference between common profit objective, the development part- the ex ante design by IFIs of programs for coun- nership is not always driven by the aim to tries and the ex post assessment by IFIs of coun- "develop" the recipient country. try-designed programs. The intention of the CDF was to move from the first to the second. Almost every donor cites development or poverty reduction as the major reason for pro- viding aid. But there are other (sometimes dom- Country-led Partnership inant) reasons for donors to give aid. Some aid is given for the strategic commercial and politi- Country-led partnership represents a paradigm cal interests of donors. Sometimes new loans are shift from donor-led, conditionality-driven, and given "to enable the old loans to be paid back" fragmented aid delivery to a system that puts the (Easterly 2001, p. 117). For some aid agency recipient in the driver's seat. This approach is staff, power, prestige, and budget size depend on intended to ensure better coordination of exter- the volume of disbursements they make. Foreign nal assistance, harmonization of practices, aid also creates and sustains substantial consul- better alignment with the country's develop- tancy employment in developed countries. The ment strategy, and reductions in the inefficien- recipients know that aid has often been given cies, asymmetrical relationships, and tensions in regardless of their behavior. Therefore, it can be the donor-led approach. With the recipient difficult to get the actors to behave coopera- country providing leadership based on its coun- tively, especially if doing so would threaten their try-owned development strategy, the new part- own objectives. nership in aid coordination could focus more on development outcomes. The new country-led partnership principle is supposed to be based on both country selectiv- A coherent development program, as intended ity and program selectivity. Improved partner- by the PRSP, is important for effective country- ships are more likely to occur in countries led partnership and aid coordination in low- with better governance and institutional struc- income countries. Crucial features in changing tures. And institutionalizing partnerships can donor behavior and fostering country-led part- strengthen civil society, institutions, and nership include ownership of the design of the governance structures. Some donors favor country development plan (such as Uganda's proactive, country-led partnerships to create PEAP), strong government leadership and incentives for long-term institutional changes, capacity, and clear institutional and organiza- while others take a wait-and-see approach to tional setups and information systems for aid adopting budget support and program coordination. Where these are missing, donors approaches to aid relationships (Evans 2002). generally drive the coordination process. Under the proactive approach, support for 30 capacity building and support for implementa- aid monies in operating environments often tion, through budget aid and/or projects utiliz- characterized by weak public administrations ing existing structures, go together. and corruption. IMF-World Bank studies indi- cate that most HIPC countries cannot yet track The new country-led poverty expenditures adequately. Most controls focus on procurement of inputs, disbursement partnership principle and auditing of funds, and compliance with social and environmental safeguards--all entail- is supposed to be based ing substantial administrative costs. on both country Changing Partnership selectivity and Performance: Progress and Issues program selectivity Several instruments offer potential to foster more effective country-led partnerships. They are: country-led coordination mechanisms, The country-led partnership principle is process- alignment of donor support with country strat- intensive and targets long-term institutional egy and programs, more effective modes of aid development. This is at odds with the short-term delivery, and harmonization of donor practices pressures on donors to disburse funds, meet and procedures. The evaluation case studies reporting requirements and budget cycles, and examined the extent to which each of these is show the results of their activities. This provides effectively fostering country-led partnership.21 an incentive problem for the long-term institu- tional orientation of the new partnerships. Country-led coordination Finally, with the swing in development thinking Progress toward coordination of aid activities by favoring social sectors as previously neglected the recipient country is reported in four CDF but effective means of poverty reduction, donors case study countries--Bolivia, Ghana, Uganda, overcrowd these sectors. Health, education, and and Vietnam. Both donors and recipients report public administration often receive more atten- that the number and frequency of in-country tion than other sectors and more resources than donor-recipient meetings and coordination they can absorb (see Uganda case study). Donors activities have risen over the past five years, fight for turf, with potential duplication and although the effectiveness and efficiency of waste--and sometimes without due considera- increased activities have been mixed.22 Overall, tion for absorptive capacity constraints. governments are playing a more active role and local donor coordination has been intensified. Country-led partnership is also hampered by For example, the first survey undertaken for the several other factors affecting recipients Vietnam case study found that 97 percent of (Holmgren and Soludo 2002). These include: respondents felt that there had been an insufficient or uncertain political commitment improvement of relationships between develop- to poverty reduction and sustainable develop- ment partners because of the CDF (with 64 per- ment, limited implementation capacity, and low cent citing "a lot"' of improvement). standards of governance. Donors resort to proj- ect rather than program aid because it provides A significant break with past practices has been a credible fiduciary environment for channeling the change in venue of Consultative Group (CG) meetings from donor to recipient country capitals. strategies and priorities is the number one 31 This allows increased participation by various burden that the recipient countries identified branches of government and a wide range of in a recent survey (see box 2.5). As of July domestic stakeholders (including civil society 2001, only a third of the 46 countries tracked organizations and private sector associations), and by the CDF Secretariat reported any form of yields more interaction with external partners. improvement in donor selectivity.23 Among the CG and mini- or mid-CG meetings are held in case study countries, donor alignment with Bolivia, Ghana, Uganda, and Vietnam with the PRSPs is the closest in Uganda and Vietnam. A participation of all donors as well as domestic review of the PRSP approach undertaken nongovernmental stakeholder groups. Donors and jointly by the IMF and the World Bank in early government also hold macro, sector, and project- 2002 concluded that "nearly all donors have specific coordination activities in each country agreed in principle to align their programs with with varying degrees of frequency and participa- tion. The participation of domestic nongovern- mental stakeholder groups in these events tends Box 2.5. Main Burdens of Donor to be more sporadic. Government and other Practices in 11 Countries stakeholders interviewed for the country case studies also report that World Bank operations Recipient country officials surveyed for the OECD DAC Task Force on Donor Practices iden- staff in the field are much more committed than tified the major burdens listed below imposed they were five years ago to country leadership by donor practices. of aid coordination processes and to close consultation with other donor representatives. Type of Frequency Rank burden of mention Complaints of inadequate consultation by head- quarters teams with other donors persist, however. 1 Lack of fit with national 11 priorities and systems In Uganda, the MFPED firmly leads the aid coor- 2 Donor procedures in 10 dination process, acting as the main overseer of partner countries donor activities and sector working groups. All 3 Inconsistency among 7 donor-funded projects/programs must be cleared donors by the Aid Liaison Department and an inter- 4 Excessive demands on 6 ministerial Development Committee (housed in time (transaction costs) MFPED), which strictly limit line ministries' authority to accept donor-funded projects and 5 Disbursement delays 6 programs that either fall outside of sector invest- 6 Lack of information 4 ment plans, fail to directly address priority areas 7 Inconsistency with 3 identified by sector investment plans, or are not national systems consistent with government's MTEF. The result 8 Demands beyond 2 has been increased alignment of donor country national capacity assistance strategies with government develop- ment strategy. Source: Amis and Green. 2002 The survey covered officials from central government, line ministries, project implemen- tation units, and relevant civil society organizations in Alignment of donor support Bangladesh, Bolivia, Cambodia, Egypt, Mozambique, Romania, Senegal, the South Pacific, Tanzania, Uganda, The lack of alignment of donor country assis- and Vietnam. tance strategies with country development 32 PRSPs, but much remains to be done to achieve reduce information and accounting asymmetries this objective" (IMF/IDA 2002, p. 20). in donor-recipient relationships and enhances Admittedly, alignment cannot take place coordination efficiency. Putting the recipient overnight, since projects in a donor portfolio country in the driver's seat in program selection, typically last for three to five years. Although design, implementation, and accountability the process can take time, this is not an excuse could minimize the problems of bunching in a for not beginning. Donor alignment has made few fashionable sectors. It could result in greater most progress at the sectoral level through the sensitivity to absorptive capacity issues and use of joint aid instruments (such as budget sup- improve the collective learning involved in port through SWAps and PRSCs). joint rather than disparate activities. In addi- tion, it could significantly lower aid delivery Many of the observed changes in donor behav- transaction cost burdens as compared with those ior in the country cases and other analyses of multiple projects absorbing the same volume derive from these instruments that implement of aid as the common pool. country-led partnership (Holmgren and Soludo 2002). Surveys show that recipients generally Despite efforts at coordination, project rather prefer program and budget support over a proj- than program aid still dominates, and the harmo- ect approach. This "common pool" or "basket- nization of donor procedures and practices seems funding" has been promoted as a way of to be minimal. Even for Uganda, where country- fostering country-led partnership (Kanbur, led partnership seems to have moved furthest Sandler, and Morrison 1999). Common pool along, government data indicate that as recently initiatives have been under way in Bolivia, as 2001 the aid project portfolio was massive and Ghana, Uganda, and Vietnam. Proponents of fragmented, with 42 different donors providing the common pool approach argue that it helps assistance through 524 projects and 825 agree- ments. In Bolivia, despite efforts at coordination, donor assistance remained highly dispersed and Box 2.6. Disadvantages of Project fragmented, with 850 projects or programs. Implementation Units Modes of aid delivery PIU managements are poorly linked to In all the country studies, donors and govern- most other government agencies. ments voiced concern about public sector Capacity building within a PIU does not capacity constraints, especially a weak civil necessarily strengthen the ministry where service. And while major institutional reforms it is located. relating to public sector management, public Selection and composition of technical finance management, and civil service pay assistance through PIUs tend to reflect reform have been recognized as necessary and donor rather than government preferences. fundamental to bring about significant changes, the pace of reforms has been slow across coun- PIU pay scales are typically beyond govern- tries. Consequently, technical assistance (TA) ment scales, causing resentment and distorting incentives. remains a significant mode of aid delivery. However, the customary manner of delivery has Reporting of disbursements is inadequate. contributed to a vicious cycle where topping up of salaries and reliance on project implementa- Source: Government of Uganda 2001. tion units (PIUs) and highly paid consultants 33 Figure 2.1. Use of Project Implementation Units (Percentage of donor respondents) NO NO NO NO 15% 25% 45% 13% YES YES YES YES 85% 75% 55% 87% Bolivia Ghana Romania Vietnam have undermined mainline public sector capac- that the practice has either increased or ity and demoralized low-paid civil servants (box remained the same over the last year. The 2.6). Teams in Ghana, Romania, Uganda, and Romanian country case study finds that salary Vietnam heard complaints about supply-driven, incentives for staffs of PIUs, typically located in high-cost TA of marginal quality. the relevant line ministry, have had a demoral- izing impact on regular staff. Donors provide The use of PIUs and provision of salary incen- salary incentives for the same reasons they use tives for PIU staff is pervasive across all five sur- PIUs. Other reasons include: pressure for fast veyed countries. The share of donor respondents implementation from donor HQs; common supporting PIUs ranged from 87 percent in practice among other donors; reluctance by gov- Vietnam to 55 percent in Romania (see figure ernment to tackle the issue; and pressure by the 2.1). On average, half of the donors now using recipient government to have incentives. program/project implementation units plan on continuing their use (all donor respondents in A bright spot in aid delivery is the declining Vietnam plan to continue using PIUs). Only in reliance on international technical assistance. Uganda are the majority of donor respondents Almost all donor respondents in Bolivia, preparing for the eventual phasing out of PIUs, Ghana, Romania, and Uganda use less than 20 in accordance with government policy. percent of their annual disbursement on inter- nationally and locally recruited technical assis- Donors say they use PIUs to improve imple- tance, and a majority of these donors report an mentation effectiveness and efficiency, ease increase in the share of locally recruited techni- recruitment difficulties, and reduce the work- cal assistance over the past five years. This load on thinly spread, poorly qualified field staff. reflects their recognition that local consultants are more in tune with the country conditions The provision of salary incentives is particularly and realities, and, in most instances, more cost- widespread in Ghana and Vietnam, where effective than international consultants. on average half of the responding donors top up salaries and/or provide meeting allowances, per The obstacles posed by donor practices diems, consultancy contracts, and other This evaluation has confirmed the OECD DAC perquisites to nationals involved in implemen- survey findings (cited in box 2.5) that donor tation. In all five countries, most donors report practices and procedures represent a significant 34 Figure 2.2. Joint Donor Preparation/Appraisal Missions in the Preceding 12 Months 8% 8% 13% 4% 4% 8% 84% 92% 87% 92% Bolivia Ghana Uganda Vietnam Less than 5 5 to 10 More than 10 obstacle to country-led partnership. Although b) Joint projects and programs many donors point to changes to their adminis- Moves by donors toward jointly funded projects trative rules and procedures, the evidence from and programs (such as SWAps, basket fund- the case studies and the survey indicate that ing, and cofinancing arrangements) have improvements are rarely observed in the recipi- been limited across countries. In Bolivia, 6 ent countries. Three kinds of practices--the of 13 donors--the Netherlands, Sweden, number and type of missions, the design of proj- Switzerland, the United Nations Develop- ects, and the harmonization of donor proce- ment Program (UNDP), the United Nations dures--emerge as areas offering considerable Children's Fund (UNICEF), and the U.K.-- scope for improvement. responded that over 40 percent of their total annual disbursements over the past two fiscal a) Donor missions years was accounted for by jointly funded Large numbers of preparation/appraisal mis- operations. In contrast, in Vietnam, only one sions from donor headquarters continue to donor of 23 has over 40 percent of its total stretch recipient countries' human and time disbursements accounted for by jointly fund- resources, and contribute to high transactions ed operations. Donors in Bolivia also note costs in managing aid. In the surveyed coun- that cofinancing does not necessarily imply tries, donors have not reduced the number of sound cooperation and that there are quali- missions they undertake from headquarters over tative factors that determine government- the last five years. In the past twelve months, donor relations. 90 percent of donor respondents in Romania, 75 percent in Uganda, and 50 percent in Most donors report that the number of joint Vietnam had undertaken more than five mis- preparation/appraisal missions has either sions, with the Asian Development Bank increased or remained the same over the past undertaking as many as 30 missions in Vietnam, five years. The World Bank and the U.K. report the European Bank for Reconstruction and that at least half of their missions to the case Development (EBRD) between 10 to 20 mis- study countries were undertaken jointly with sions in Romania, and Ireland Aid 12 missions other donors. Nonetheless, very few joint mis- in Uganda. The World Bank undertook more sions are actually undertaken--most countries than 10 missions in the past 12 months in each average fewer than three missions per donor. of the three countries.24 Furthermore, the joint missions are concen- trated in the few sectors where there is either a c) Simplification and harmonization of procedures 35 functioning SWAp or basket funding, and/or Donors have put considerable effort into discus- cofinancing arrangements (figure 2.2). sions of harmonization of their procedures; box 2.7 reports some of these efforts. A modest positive trend emerges for joint mon- itoring, supervision, and evaluation missions. Despite these initiatives, progress in harmoniz- About half of donor respondents in Bolivia, ing procedures is moving very slowly. The Uganda, and Vietnam report an increase in such number of progress and financial reports missions over the past five years.25 In Ghana, required by donors from line ministries contin- the number of joint monitoring and evaluation ues to impose major burdens on governments. In missions has remained flat over the past five Bolivia, Ghana, and Romania, for instance, 60 years, with just over half of donor respondents percent of donor respondents require four or reporting no change, and two--the Food and more reports per project annually. In Bolivia, Agriculture Organization (FAO) and the half of the donors (Corporación Andina de Netherlands--reporting declines.26 Fomento, or CAF, Denmark, the Netherlands, Box 2.7. Recent Harmonization Efforts at the Global and Country Levels Global Level The Presidents of the Multilateral Development Banks (MDBs) have been meeting twice a year since the mid-1990s to discuss common issues, including responding to the 1996 MDB Task Force Report, Saving a Changing World, which called for greater harmonization of policies and procedures among the MDBs. This led to the creation of joint working groups covering procurement, environmental assess- ment, financial management, and evaluation. Accomplishments have included agreement on standard bidding documents and standards and good practices for financial management and evaluation. In late 2000, the OECD/DAC Task Force on Donor Practices was established to identify and document changes in donor practices that could reduce burdens on recipient governments and enhance recipient-led aid coordination. In April 2001 the Development Committee of the World Bank and IMF asked that an overall framework be developed, including time-bound action plans for progress in harmonizing opera- tional policies and procedures regarding financial management, procurement, and environmental assessment. The United Nations Development Program is harmonizing and simplifying operational poli- cies and procedures among UN funds and programs. Country Level Several donors are supporting country-led harmonization efforts in selected countries: The World Bank, Asian Development Bank, Japan Bank for International Cooperation, and eight "like-minded" bilateral donors in Vietnam and the Special Partnership for Africa (SPA)--with a focus on budget support--in Ethiopia (with the Bank), Rwanda, and Senegal. Other discussions are being held with Bangladesh, Bolivia, Cambodia, Honduras, Jamaica, Kenya, Kyrgyz Republic, Morocco, Nicaragua, Niger, the Pacific Islands, Papua New Guinea, Tanzania, and Zambia. It is hoped that these country pilots will provide a stimulus to the typically slow progress at the international level. The High-Level Forum on Harmonization in Rome, February 23-25, 2003, provided a challenge for donor and partner country leadership to make meaningful progress. 36 Spain, Sweden, and the UNDP) report that the Box 2.8. Three Donors, Three Sets number of reports required has actually of Procedures, No Building increased over the past five years, while most donors in Ghana and Romania report that the A building project in Bolivia shows the high trend has remained flat. In both Uganda and cost of current donor practices. Three major Vietnam, where the majority of donors require donors in the health sector agreed to cofi- nance construction of a building. The ministry fewer than four reports yearly, a few responding put up the land, but each donor had its own donors--43 percent in Uganda and 20 percent procedures, which made it difficult to find a in Vietnam--are able to report a decrease in the common approach to construction. The three number of reports they require from govern- donors could not pool their contribution in a ment. In both countries, and especially in common fund, because the rules of the agen- Uganda, this reflects the gradually increasing cies prohibited channeling money to another confidence of donors in the government's com- agency. One donor did not require competitive mitment and control systems and capacity for bidding, but the other two did. None of the financial and expenditure management. agencies could accept the procedures of any of the others, and two of the agencies were Although there are isolated instances where unwilling to adopt Bolivian rules. donors in Bolivia, Ghana, and Uganda have adopted joint reporting formats endorsed or pre- A "thematic" approach was considered. One donor would finance the design, another pared by government, this has largely occurred the construction works, and the third could in sectors where there is either a SWAp or mul- contribute the equipment. As an alternative, tidonor cofinancing arrangement. In Ghana, all to avoid one agency blaming another if seven donors that report adopting a format something went wrong, it was proposed that endorsed by the government--Danida, the each donor would finance particular floors, European Commission (EC), the Netherlands, procuring the materials and hiring builders the U.K., the UNDP, the United States, and the according to its own standards and proce- World Bank--participate in the health sector dures. This would have greatly prolonged SWAp or common basket funding arrangement, the construction period. while all four donors that have adopted the gov- ernment's format in Uganda participate in the After long debates, one of the donors with- education and health SWAps. As noted in the drew from the project, and the other two Uganda country case study, donors providing signed an agreement of their intentions of constructing the building. Thanks to revisions budget support on a sectoral or general basis are of the regulations and numerous coordination more prone to accept government procedures, meetings, the donor contributing the smaller and more so in sectors where joint sector review amount has accepted the rules of the processes are entrenched. Donors who have not other donor. adopted joint government-donor reporting for- mats tend to be those whose own headquarters' After two years, the foundation stone has yet policies and regulations restrict their ability to to be laid. move to budget support or participate in instru- ments such as SWAps.27 Source: Bolivia country case study. Harmonization has been slow, particularly in the area of procurement rules. The majority of donor respondents in Ghana, Romania, indicators of effective partnerships. The second 37 Uganda, and Vietnam report that they have not step would be to design institutional arrange- adopted common donor procurement rules. In ments or governance structures to mainstream Ghana, for instance, only two donors--the these indicators. Netherlands and the U.K.--have adopted the procurement rules of other donors: both use Independent evaluation at the country level, those of the World Bank. Similar situations are together with possible oversight of donor behav- noted in the other countries, where only one or ior by national legislatures, might be a powerful two donors have adopted the procurement rules means of ensuring donor-recipient transparency of other donors. In Vietnam, the Asian and accountability. Currently, Tanzania is Development Bank (ADB) and the World Bank experimenting with a form of independent eval- are harmonizing their procedures. For the most uation of aid relationships (box 2.9). These part, however, stringent donor headquarters and other worthy experiments deserve further policies and complex and intertwined political exploration. Respondents in several case study and business interests in donor countries have countries suggested the Consultative Group made it difficult for donors to harmonize among mechanism or SWAp meetings as possible themselves. As noted in the Bolivia case study, forums for reviewing donor performance (issu- it appears particularly costly and irrational for ing an annual "donor report card," for example). member countries of the European Union, A wide range of potential indicators of country- which have nine representations in Bolivia, led partnerships could be quantified and com- including the EC, not to have coordinated or bined in a composite index of effective harmonized their procedures among members development partnerships. The impact of the more extensively. Box 2.8 graphically illustrates index on outcome variables, such as MDGs, the costs that multiple donor procurement and would need further examination. related procedures can impose. Conclusions on Monitoring and Evaluating Country-led Partnership Country-led Partnership Is country-led partnership working in practice? Performance: Breaking with The answer is a qualified yes. There is evidence the Past that donor-recipient coordination activities To break with the past and make real advances have increased in number and frequency over on the country-led partnership principle the past five years. Progress has been highly requires serious monitoring. This includes rat- uneven across donors, across countries, across ings and rankings of both the recipients and the various elements or instruments of the coun- donors on major elements of the country-led try-led partnership principle, and across sectors partnership principle so that all partners have within countries. Donor headquarters policies, the information to continuously refine and regulations, and incentive structures set bound- improve their practices and policies. All parties aries on how much donor field offices can must be evaluated, in joint performance reviews change their behavior. True, there are more co-designed and co-evaluated by both sides of joint appraisal missions, more joint reporting the partnership. A first step would be to harmo- formats, and more joint monitoring. But these nize existing criteria among aid agencies and operations tend to be limited to a few (popular) agree on a set of "international best practice" sectors, particularly the social sectors. Har- 38 Box 2.9. Independent Donor Performance Monitoring: The Tanzanian Experiment In Tanzania, efforts have been made to improve the aid relationship since 1995, when an independent assessment, funded by the Danish government in agreement with the Tanzanian government, made a number of concrete recommendations for the government and the donors. Agreement was reached between the government and the Nordic countries on how the aid relationship could be improved, and this led to a broader discussion with the donor community on concrete steps that needed to be taken. At the meeting of the Consultative Group in 1999, it was agreed in principle that an independent process of monitoring of aid relationships should be instituted. This was followed in 2000 by the prepa- ration of the Tanzanian Assistance Strategy (TAS) to govern the ongoing aid relationship between the government and its development partners. At the meeting of the Consultative Group in 2000, it was agreed that implementation of the TAS would include independent monitoring and evaluation of donor performance as well as of Tanzanian performance. Since then the Economic and Social Research Foundation, an independent Tanzanian NGO, has been appointed to work as an honest broker coordinating the independent monitoring with donor funding coordinated by the UNDP. The Independent Monitoring Group consists of two Tanzanian experts, three experts from donor countries, and one non-Tanzanian African. All members of the group were selected based on their independence from the Tanzanian government and from donor administrations. The work of the group started in early 2002, and its report was presented at the Consultative Group meet- ing in December 2002. All parties are committed to supporting the work of the group up to the end of 2003, after which the situation is to be reviewed in light of the experience gained. Source: UNCTAD, 2002; Helleiner 2000. monizing rules and procedures, particularly for trust. These considerations surface most explic- procurement, remains a major problem. Donors itly in the Vietnamese case study, where inter- still work through separate implementation locutors said that trust and personal working units in or outside line ministries and provide relationships were key factors in building effec- supply-driven technical assistance. tive country-led partnership. For donors, this included staff competence, long-term use of In addition to conscious efforts by donors and "true experts," communication, and dissemina- recipients to address the plethora of bureaucratic tion of information (in Vietnamese). For the constraints, more fundamental reforms have to government, this included staff compensation, focus on building the infrastructure to institu- information sharing, long-term staff, language, tionalize and sustain aid coordination oriented and other skills. A related priority among toward country leadership. Three major ele- Vietnam stakeholders is transparency. One inter- ments of this infrastructure include the accelera- locutor suggested that the World Bank should tion of mechanisms that promote common pool add transparency as a fifth CDF principle, and funding, demand-driven capacity building, and argued that the most effective way to enhance effective monitoring and evaluation that puts ODA effectiveness and reduce corruption is peer pressure on nonconforming partners. Two to facilitate the involvement of independent additional interrelated elements that apply to local experts. The Bolivia case study concluded both sets of partners are transparency and mutual that trust matters not only in the relations between international partners and national program to retain capacity. With enhanced 39 stakeholders, but also for relations among inter- capacity, a recipient country is not only better national cooperation agencies themselves. equipped to develop its own PRSP or other Implementing the CDF therefore presupposes long-term development framework. It can also ongoing efforts to rebuild trust at every level and assume the role of a more equal partner, over- among all stakeholders. coming some of the asymmetries that character- ize donor-recipient relationships. Donors can Intensified partnerships inevitably place greater foster the right preconditions for country own- demands on the staff and institutional capacities ership if they permit the recipient country to of the recipient countries, requiring heavy determine where and from what source it desires investment in domestic capacity and a skillful augmented capacity. C H A P T E R T H R E E The CDF Principles as a Package: 41 Interaction and Impact The CDF is more than its four component prin- ment Authority (Sida) projects and programs in ciples. First and most important, it aims to bring three East African countries--Kenya, Tanzania, all these principles together in one package or and Uganda--concludes that transparency and overarching framework, giving them all promi- mutual trust must come first, followed by the nence. It has thus provided all aid donors and establishment of country-led partnership. Only recipients with a common structure to shape then will donors be prepared to permit the recip- their strategies, policies, and programs. The ient country to have meaningful ownership and potential of this package is most evident in management of the aid process (Anderson and Uganda, and is apparent at the sectoral level in others 2002, p. 4). health in Ghana. Second, the CDF, by focusing on poverty reduction, reminds development actors of many of the goals and processes that Complementarities and have long been part of development work--but Tensions among the CDF may have been neglected in recent decades with the crises of stabilization, adjustment, and Principles growth. The CDF is intended to rebalance the approach to development and correct the major There are important complementarities among flaws in aid management of the recent past. the elements of the CDF. The principles are The CDF has had this effect to at least some mutually reinforcing and were intended to be extent in each of the case study countries. implemented together. The country case studies Third, the World Bank's advocacy of CDF prin- showed that country ownership is necessary for ciples and processes has increased their visibil- effective country-led partnership, but neither ity and application. Indeed, other UN agencies, country ownership nor country-led partnership as well as bilaterals and multilateral develop- alone is sufficient. A long-term, holistic ment banks, have affirmed their support for the approach to development is essential for CDF and the PRSPs. embarking on a broad set of economic, social, and political changes, and a results orientation A sequence among the CDF principles can run is needed to mark progress for accountability in various directions. The Uganda case study, and improved performance. Case studies also based on that country's experience, suggests showed that progress on the results principle is that country ownership comes first: followed linked to the country's level of ownership of its by the long-term, holistic development frame- development agenda and the extent of its con- work; then, country-led partnership; and, trol over the national monitoring and evalua- finally, results orientation. A recent study of tion function. As shown in Uganda, ownership ownership of Swedish International Develop- is essential to the sustainability of the country's 42 long-term holistic framework, the Poverty study countries. But in no case has "country-led Eradication Action Plan (PEAP). partnership" eliminated the tension between ownership and partnership. The case studies are However, there also are tensions among these replete with examples of long-standing tensions elements that are frequently obscured. These between recipient countries and donors over tensions derive from the development process such issues as health (Vietnam), civil service itself, and represent problems the CDF is reform (Ghana), and coca eradication (Bolivia). intended to solve. The tensions must be Where these tensions have been successfully acknowledged and managed if they are not to managed, it is because the government has defeat the good intentions of the CDF and PRSP. taken the initiative to establish mechanisms and to invite donors to participate (for example, One tension is between country ownership and Sector Working Groups in Uganda). partnership. Country ownership implies a situa- tion in which the balance of decisionmaking is Another tension is between the long-term focus in the hands of governments and other domes- and the emphasis on results. Long term in devel- tic stakeholders receiving external funding. opment can mean decades or more. But political Partnership involves two or more parties work- pressures in aid-giving countries often demand ing together to achieve desired outcomes. indicators of results within a year or two. Therefore, greater partnership can lead to less Despite the MDGs' horizon of 2015, "results ownership without the CDF principles. By foster- management" has come to imply a much shorter ing country-led partnership, the CDF encour- time horizon. There can also be tensions in ages governments to only accept external aid in measuring meaningful results, especially in the line with country-owned policies. In addition, it short term. It is often true that the more quanti- encourages development assistance agencies to tative and shorter term the results indicators, align their assistance with country-owned the less they have to do with the impact of aid. strategies rather than impose conditions devel- Short-term indicators do not necessarily reflect oped at donor headquarters. The tension causality--a relationship that can be obscured between ownership and partnership stems in the eagerness to show positive outcomes. At largely from the asymmetry of power between the same time, to the extent that a results the recipient country and donors. Because aid emphasis is at outcome and goal levels, it sup- donors are accountable to their legislatures or ports the long-term, holistic approach. A focus boards, they must require that certain condi- on health outcomes, for example, requires look- tions be observed in the use of their funds. ing beyond health projects to initiatives in other Those conditions may not always coincide with sectors, such as education and water, and vice- the wishes of the governments receiving aid. versa. For example, water projects in Uganda Despite the rhetoric to the contrary, the recipi- have a significant health education component. ents are never completely "in the driver's seat." Thus, stakeholders are forced to see inter-sec- The key in any paradigm of aid management is toral relationships, to "see across sectoral silos." how much--and under what conditions--do donors cede decisionmaking to recipients on A third tension involves the possible diver- how the aid is used. The CDF attempts to cor- gence between the need for a framework for rect an imbalance in this relationship by shifting long-term, holistic development and the real- responsibility to the recipients. This has hap- ity of economic decisionmaking in a demo- pened in varying degrees in each of the case cratic, market economy. Although some suggest that the CDF implies a form of indica- A further tension is between ownership and par- 43 tive planning that is inappropriate for free ticipation. It can be very useful for governments market economies and democratic discourse, to consult widely with their citizens on develop- the experiences of the six case study countries ment strategies and programs. But this process is show that the planning style of the CDF and time-consuming and costly, and it may not pro- PRSP is quite consistent with free markets and duce clear-cut preferences. Indeed, it could well broad participation of the population, includ- produce conflict, as different interests demand ing in national elections. different approaches. The CDF recognizes that Table 3.1. Complementarities and Tensions among the CDF Principles Holistic, Country Country-led long term ownership partnership Holistic, long term Country ownership Complementarity Links strategy with political vision Builds consensus and sustainability Country-led partnership Complementarity Tension Holistic ­ division of Country-led labor among partners partnership implies shared responsibility Long term: wide & accountability support of MDGs among partners Results orientation Complementarity Tension Complementarity Results focus supports Leads to scrutiny & Results orientation long-term, holistic by accountability of leads to increased requiring tracking results government that may accountability, an in all related sectors be resisted objective of external partners Tension May be offset by broad Hard to measure long-term ownership that includes indicators in short term nongovernment stakeholders Holistic approach can lead to goals multiplicity 44 in the end, elected governments should be had one or more long-term strategies in place, a responsible for policymaking and must decide degree of country ownership of the strategy, and the extent of consultations, weighing the time some experience with participatory approaches. needed to consult widely and the risk that broad Bolivia, Burkina Faso, Ghana, and Uganda had consultations can raise expectations that such strategies at the national or sector levels. inevitably will disappoint some groups, as hap- Romania and Vietnam had slightly different tra- pened among some participants in the CDF ditions of long-term central planning and mech- Consultations in Romania in 1999. Here again, anisms for public consultation through official there is a question of balance--between exten- government machinery. The problem was not sive consultation and action. the lack of long-term visions and plans, but their confusing proliferation. These and other potential complementarities and tensions among CDF principles are summa- In most cases, these CDF-type processes were of rized in table 3.1. limited effectiveness. The long-term strategies were seldom comprehensive and holistic; they The Role of the PRSP in Implementing the Box 3.1. Participatory Budgeting and the PRSP CDF Principles A review of experiences of one key aspect of Implementing a set of development principles participatory PRSP processes, participatory budgeting (PB), has revealed both best prac- requires a plan that translates those principles tices and pitfalls. By reshaping both the sub- into action, and institutional processes that stantive content of public budgets and the enable the actions to be resourced and carried process of budget policymaking in developing out. The CDF principles alone could not pro- countries, PB can support poverty reduction, vide these mobilizing elements, but in low- social justice, citizen empowerment, and public income countries, the PRSP has emerged to play learning, as well as build confidence in nascent this role. It not only provides a framework for or precarious public institutions. But participa- the comprehensive holistic vision, but also an tory processes that emphasize civil society action plan linked to government delivery sys- participation while marginalizing the role of tems, the budget, and national monitoring legislatures can undermine an institutional processes. On the process side, the PRSP pro- counterpoint to executive power. Participatory vides specific arenas where the CDF principle of processes also risk being hijacked by interest country ownership can play out--in poverty groups or local elites, or becoming a venue for distributional conflict. According to the review, diagnosis, priority setting, and participatory realizing the benefits of PB requires that gov- budgeting and monitoring (see box 3.1). While ernments, civil society, and legislatures be will- the PRSP still has some way to go to realize all ing and able to play their part. External actors of this potential, it is a considerable advance on can best support the phasing in of PB pro- the earlier situation. grams by helping all domestic stakeholders to build their capacity to participate effectively. In all six case study countries, some elements of a CDF approach were already in place before the Source: Bonaglia, de Macedo, and Bussolo 2001 and Heimans 2002. official launch of the CDF and the PRSP. Most lacked results-based monitoring, the consulta- into an annual budget (over a three-year rolling 45 tion process was sporadic and narrowly based, period).29 Thus, a cascade of instruments was and country ownership was limited. Donor sup- put in place in response to adopting new princi- port was fragmented and dispersed. Plans and ples. When Uganda became one of the CDF strategies lacked implementation, and had little "pilot" countries in 1999, the authorities credibility with government, donors, or the declared, "The PEAP is the CDF." In addition, wider public. Elements of the CDF existed, but when it was necessary to prepare an interim and the CDF was not implemented completely. full PRSP as conditions for receipt of Enhanced CDF-type processes were more effective in the HIPC debt relief, a summary of the PEAP context of sectorwide approaches (SWAps), but became the PRSP.30 Donor alignment to the these were usually limited to particular sectors, PEAP was already evident in increased donor such as education and health, where capacity coordination and a rise in budget support. and data were more adequate.28 There are many reasons for the relative success of the Ugandan PEAP. These include the Romania provides an interesting example of a focus on implementation and accountability, trajectory of a CDF pilot in the absence of a strong political leadership, and good donor PRSP. The case study indicates that, despite coordination--that is, a CDF-type framework recognition of the benefits of more open, con- and a PRSP-type action plan. sultative policymaking, the initial enthusiasm for institutionalizing the 1999 "CDF consulta- tions" subsequently waned. There were many Visions, objectives, reasons for this, including a change of govern- ment, ministerial changes, and the preoccupa- and targets were tion of political leaders with preparations for accession to the European Union (EU) and essentially wish lists NATO. But a common complaint from the without priorities for 1999 participants was that the CDF consulta- tions had high transaction costs, with few con- actions and resources crete results. A vehicle was needed to translate principles into action. Visions, objectives, and targets were essentially wish lists without priori- ties for actions and resources: "Romania has Could other mechanisms have fulfilled the enough plans... we don't need any more strate- role of the PRSP in implementing the CDF prin- gies. What is needed now are implementation ciples? The Ghana case study evidence suggests and results" (Romania case study). that while Vision 2020, the National Strategy for Sustainable Development (NSSD), and other In Uganda, the CDF principles were in place poverty reduction strategies were important fore- well before the CDF and the PRSP were runners of the PRSP, they also lacked implemen- launched and were articulated through an tation details and a link to resources. The most action plan known as the Poverty Eradication successful alternatives were the sector-based Action Plan (PEAP). The PEAP spawned SWAps, constructed at a more gradual pace, in SWAps and the medium-term expenditure line with CDF principles, and arguably with framework (MTEF) operationalized the SWAps more success in changing donor behavior. 46 Did PRSP Conditionality Distort Box 3.2. PRSPs and HIPC the Application of CDF Principles? Conditionality: Evidence from The case studies report that the HIPC initia- the Case Studies tive was a major motivation in preparing a PRSP (box 3.2). PRSPs were rushed to meet "There were some initial concerns, voiced on HIPC schedules, and the participatory process both sides, that in the end, the CPRGS docu- and analytical work were less satisfactory as a ment might serve no other purpose than com- result (Bolivia, Ghana).31 Time pressures may pliance with a World Bank and IMF requirement. These fears were not borne out, also have undermined the institutionalization and the CPRGS is fully owned by government of the process, as in Ghana, where a small ad and recognized as the guiding framework for hoc group was set up to deliver the product. almost all ODA." (Vietnam) Even without HIPC, the incentive of the PRGF and IDA financing had the same effect. "Many still perceive the GPRS as little more PRSP activities were rushed to fit in with the than HIPC conditionality or an instrument to schedules of consultative groups or PRGF mon- convey a new generation of SAPs." (Ghana) itoring (Ghana). The Joint Staff Assessments (JSAs) reveal some concludes: "The JSA is a good example of an of the areas in which the PRSPs for the case unreformed donor practice continuing in the study countries needed strengthening. It is dif- face of the need for broader coordination and ficult to say how far the "weaknesses" identified rationalization of different donors' systems." in the JSAs resulted from the pressure of the HIPC timetable or other problems, such as Limitations of the PRSP capacity or political constraints (box 3.3). Even While the PRSP offers an unparalleled opportu- so, the achievement of the PRSPs in develop- nity for the implementation of CDF principles, ing a single overarching framework for poverty there are major constraints. The first relates to reduction should not be underestimated, nor weak capacity in government and civil society, should the ownership of the document by gov- and the second to the discontinuities created by ernment and domestic stakeholders through national politics. These two constraints are not the participatory process. The virtue of the specific to the PRSP; they reflect the reality of conditionality link is that the PRSPs were actu- development and would undermine the applica- ally completed. They resulted in increased tion of the CDF principles even in the absence resources for the countries that produced them, of a PRSP. and they are now on the path to implementa- tion and revision. Capacity issues Even for a developed country, it would be a tall The JSA itself, as an instrument of IFIs, raises order to prepare a PRSP that provides a fully issues of partnership and coordination. A repre- integrated, comprehensive strategy linking pri- sentative of a donor based in Bolivia observes ority public actions to a multidimensional that the JSA remains a somewhat exclusive poverty analysis and a list of monitorable tar- process, involving the government, the IMF, gets; includes bottom-up costings and relates and the World Bank, and leaving the bilateral them to medium-term expenditure ceilings; and donors to assess the PRSP on their own. He does all this on a participatory basis. The requi- site capacity involves technical and analytical good institutional delivery mechanisms, an 47 skills, appropriate information, and an ability to effective budgetary process, and good monitor- organize, mobilize, and facilitate. It also ing and information. In many countries, just as requires facilitating rules and incentives. Civil these capacities are beginning to be consoli- society groups require similar skills and institu- dated in central government, new processes of tional competencies. Such capacities are scarce decentralization will require that these strategies in developing countries and are acquired over be implemented by local authorities, where time with great difficulty in the face of donor capacity is in even shorter supply. These decen- poaching of skilled personnel, the brain drain, tralization processes are likely to pose a chal- and corruption. lenge even for Uganda. It is not just the capacity to deliver a document The capacity constraint has varied in the case that is involved here--it is the capacity to study countries, being more acute in Burkina implement it on a sustainable basis. This implies Faso, Ghana, and Uganda than in Bolivia, Romania, and Vietnam. Even in the latter countries, however, the different kinds of Box 3.3. Feedback on the Early PRSPs capacity are unevenly distributed. For the PRSP to succeed, it will need coherent and sustained The JSA of the Bolivia PRSP recommended an external support, as well as a lengthy process analysis of past policy successes and failures, of learning by doing. The ideal PRSP will the sources of growth, and the impact of future not be produced and implemented overnight, reforms on the poor. with or without conditionality. The process of implementing the CDF principles will The JSA of the Vietnam Interim PRSP contained strengthen capacity, but this will be a long-term many positive statements but called for further affair, involving widespread public sector analysis of policy impacts on the poor, tracking reform and institution building. These are long- public expenditures, and measures to address term processes that will not deliver results the nonmaterial dimensions of poverty, includ- within the life of an average PRSP cycle.32 ing the position of ethnic minorities. The JSA of Uganda's PRSP recommended that it Politics pay more attention to gender and develop a The case studies--particularly Bolivia--raise special program to alleviate poverty in the challenging questions about the feasibility of north. More analytical work was called for on reaching a national consensus around divisive the implementation of the Land Act, the provi- issues such as land tenure, ethnicity, and social sion of rural infrastructure, and access to agri- and political exclusion.33 The capacity of gov- cultural extension services and rural credit. ernments to implement government policy in an efficient, transparent way is challenged when The JSA of the first annual PRSP progress report they are surrounded by a culture of patronage for Burkina Faso recommended updating the and patrimonialism. A government's margin for macroeconomic framework, developing a maneuvering is limited when there is a high global vision on rural development, and level of public discontent, as there has been strengthening social statistics. The endorsement of future PRSP progress reports will be influ- from time-to-time in Romania, or when elec- enced in part by these recommendations. tions are imminent, as in Bolivia and Ghana. This is a clear lesson of the case studies. 48 The case studies also indicate that despite incomes--essential to poverty reduction--in extensive civic participation, even involving most countries. Barring such factors as conflict opposition parties and politicized groups such or trade and weather shocks, full involvement of as unions, there is a risk that PRSPs will the private sector in CDF processes should become identified with the political party in encourage a more favorable environment for power and be discarded--entirely or par- business activity. tially--when there is a change of government. Thus, even the most holistic, results-oriented Analysis of the CDF Secretariat tracking rat- PRSP may be unsustainable and its long-term ings of implementation of the CDF principles holistic vision unrealistic if only the govern- suggests that despite the relatively better ment in power participates in its preparation. progress in implementing the country owner- ship principle, one of the key implementation weaknesses has been the failure of governments There is a risk that to provide for adequate consultation with the private sector (see World Bank 2001b). To the PRSPs will become extent that failure to involve the private sector identified with the in the CDF leads to a less favorable regulatory environment or inadequate service delivery for political party in power private sector growth, the development impact of the CDF, as well as its sustainability, may be in jeopardy. Clearly there is great diversity in the impact of country political processes, and some countries Four indicators of the business environment in may manage to preserve continuity better than 2002 for the 46 countries tracked by the CDF others. In particular, if--consistent with CDF Secretariat and the index of CDF implementa- principles--a broad social consensus is built for tion for July 2001 are used to analyze the extent a long-term vision, the key elements of the of association between CDF implementation vision can transcend political change, as they and the business environment (see figures in have in Bolivia, Ghana, and Romania. Annex 5).34 The analysis makes clear that once However, the point here is to recognize that country-specific characteristics that are likely to the aspiration of the PRSP to promote more influence the indicators of "business environ- efficient and accountable government is a ment" (level of development; whether or not political--not merely a technocratic--ambi- the economy is or was socialist; or whether the tion (see box 3.4). country is affected by conflict) are controlled for, there is no relationship between CDF imple- mentation and the four business environment The CDF and the Business indicators. This result is consistent with the finding that government consultation with the Environment private sector has been less than adequate. It also suggests that there is no evidence that The private sector comprises a crucial stake- countries with a better record of CDF imple- holder category for the CDF. It is also the engine mentation have managed to create a better busi- for growth and expansion of employment and ness environment for private sector­led growth. Box 3.4. The Intersection of the PRSP and Domestic Politics 49 In Ghana: The feasibility of a long-term vision statement necessitates cross-party agreement and, therefore, a consensus-building political culture and institutions. These conditions do not readily apply in Ghana. Democratic traditions are still too new, the influence of clientelist systems of reward remains too strong, the base of educational achievement is still too limited, information flows too poorly, and local-level participation is still very incomplete. In Vietnam: Planning in Vietnam is part of a political process. Policy content has emerged from long and difficult political debates within the Party. The 10-year and the 5-year Plans are compromises between different views on reform with respect to the place of global integration and the role of the state in the economy. Certain basic differences cannot be bridged by dialogue and partnership initiatives. In Romania: The CDF was introduced in Romania in a context of negative economic growth, rising poverty and unemployment, and political paralysis. The 1999 CDF consultations gave voice to dis- senters both in and out of government. Should raised expectations go unmet, there is a real danger of backlash against the sponsors of such consultations. But the key elements of the long-term vision that emerged from the broadly based consultations survived the change in government. In Burkina Faso: One of the sequels of Burkina's post-independence history is the heavy presence of the government in all aspects of the country's socioeconomic life. Public debate is still very much sub- dued, except in a structured context such as the legislative elections. Private media are limited. A high number of rural dwellers confront land tenure insecurity, and their representation at the local level is often questioned. This is a very tough and potentially explosive issue for the government. In Bolivia: Building shared comprehensive visions--even at the sector or issue level--entails processes with a high political content and potential to arouse fierce dispute. The pressure of deadlines set by international partners naturally influences the nature of participatory processes. Unless harnessed to the pace of other endogenous political processes, there is an additional risk that the country ownership requirement--demanded by international partners themselves--may be an inappropriate intervention in domestic political processes. Source: Country case studies. Business Conditions in the Six median ratings, while Bolivia and Romania Case Study Countries scored above average ratings. In terms of Although each of the six countries in the case "labor regulations," only Ghana and Uganda study group performed better than average scored slightly better than the median ratings, (among a sample of 46 countries) for the four while Burkina Faso, Ghana, and Romania CDF principles, this is not the case in terms of scored better than the median performance in the business environment (table 3.2). For the area of "entry regulations." The private example, in the area of "contract enforce- sector in Bolivia, and especially in Ghana, ment," three of the six countries (Ghana, appears to face much more serious impedi- Uganda, and Vietnam) scored lower than the ments than the median country in the sample. 50 Table 3.2. CDF and Business Environment in the Six Case Study Countries Contract Credit Entry Labor Country CDF score enforcement market regulations regulations Bolivia 3.7 6.0 0 77 3.2 Burkina Faso 3.0 36.4 39 3.2 Ghana 3.9 2.6 126 2.6 Romania 2.7 4.4 3.7 46 3.1 Uganda 3.6 2.6 36 2.8 Vietnam 3.2 3.3 7.9 68 3.7 Sample average 2.4 3.8 13.2 70 3 of 46 countries Source: Doing Business Dataset, The World Bank. The Development Impact gauged by analyzing the experiences of countries that have adopted development strategies that of the CDF and CDF-like approximate the CDF principles. A pilot effort Experiences to use econometric techniques to investigate this question was undertaken as part of the eval- The limited available evidence on the imple- uation. The following section summarizes the mentation of CDF development strategies-- results (see Annex 5 and Elabadawi and Gelb based on the tracking indicators developed by 2002 for more details). To give empirical con- the World Bank's CDF Secretariat for 46 PRSP tent to this concept, indicators of "CDF-like" and former CDF pilot countries for 2001--sug- principles were developed, which are combined gests that progress on CDF implementation with other global development data to analyze tends to be positively associated with better the development impact of CDF-like develop- policy and institutional environments and with ment experiences. The key question to be successful project implementation (see Chapter addressed by the quantitative analysis is whether 2 and Annex 5 for further evidence on this from CDF-like development strategies have actually the country case studies). This finding of associ- contributed to better development outcomes, ation does not, however, demonstrate causa- including better institutions, higher growth, tion--that is, whether a CDF approach directly lower poverty, and enhanced human develop- promotes better policies and institutions-- ment, as well as greater aid effectiveness. much less whether it contributes to ultimate development goals. Until more experience is CDF-like Indices gained with the CDF approach and adequate For each of the CDF principles an index was historical data are developed, a quantitative constructed as follows: causal analysis of these questions is not possible. Long-term, Holistic Development Framework. Though the CDF is new, the principles upon An index of the dispersion in ratings of which it is based are not. This suggests that the policy and institutional quality across three development impact of the CDF might be broad sectors is viewed as a measure of a country's ability to maintain balanced devel- Country-led Partnership. An index composed 51 opment, an expected outcome of implemen- of two elements that seek to measure the tation of the long-term, holistic principle.35 quality of aid delivery: first, the concentra- A large dispersion among the three sectors tion of donors in the country as a whole suggests failure to adopt a long-term, holistic divided by the sectoral concentration of aid; approach to development. and, second, an index of "excessive" techni- cal assistance, which is measured by the ratio Results Orientation. An index of the quality of optimum to actual technical assistance of poverty monitoring.36 (see Annex 5 for further details).37 Country Ownership. A simple average of four The overall CDF-like index is a simple average components of a widely quoted index on of the four indicators of CDF-like principles, democracy (the "Polity IV index"), including which regards all principles as equally important governance indicators that account for par- for shaping a CDF development strategy. The ticipation and accountability. indicators are constructed for 88 countries Figure 3.1. Index of Overall CDF-like Development Strategies by Region 0.75 0.70 0.65 e scor 0.60 0.55 CDF-like 0.50 0.45 0.40 1985 1988 1991 1994 1997 2000 SSA MENA LCR EAP SA Note: Of the 88 countries, 36 are classified as low-income countries, 33 as low-middle-income, and 19 as higher income: Sub- Saharan Africa (SSA, 26 countries); Middle East and North Africa (MENA, 7 countries); Latin America and Caribbean (LCR, 23 countries); East Asia and Pacific (EAP, 7 countries); South Asia (SA, 5 countries). East European countries are not included in the figure because the indexes are only available for the 1990s.See Annex 5 for explicit algebraic expressions of the indicators of CDF-like principles and a complete list of countries. The CDF-like index and the four component indicators are normalized to fall between 0.0 and 1.0. 52 (including the 46 countries tracked by the CDF and the capacity of society to mediate conflict Secretariat) for 1980-2000. among different economic or social groups, in addition to exogenous factors. The evidence Figure 3.1 suggests that the CDF-like index strongly supports the hypothesis that: The choice roughly tracks with development experiences of CDF-like development strategies is endogenous pursued over the last 20 years or so. It suggests to such structural and social factors. Countries that the development experiences in East Asia with initial conditions of a higher level of have tended to approximate CDF strategies development (higher per capita income), high more closely than have the experiences of other social cohesion (lower ethnic fractionalization) regional comparators, and that Sub-Saharan (Rodrick 1999),38 and better institutions for Africa has substantially failed to approximate promoting cooperation among social groups the CDF vision of development, despite having (functioning democratic and civil society insti- improved quite sharply since 1994. tutions) tend to adopt CDF-like development strategies. Moreover, with functioning demo- The Determinants of CDF-like cratic institutions, even highly fractionalized Development Strategies societies could manage to achieve a CDF-like A country's propensity to adopt a CDF-like development approach. Figure 3.2 shows how development strategy, as well as good policy much these structural and social factors are and institutional environment, may be driven associated with the index of CDF-like strate- by fundamental factors such as initial level of gies. The same findings also apply to the over- development, the degree of social cohesion, all institutional and policy environment as Figure 3.2. Structural and Socioeconomic Factors and Choice of CDF-like Development Strategies .8 .6 CDF-like .4 Actual .2 .4 .6 .8 Predicted CDF-like explained by structural and socioeconomic factors Note: Based on the regression equation of column (2) of table 4, Elbadawi, Mavrotas, and Randa 2003. Independent variables used are initial per capita income, social fractionalization, democracy and the interaction between democracy and social fractionalization. measured by the Country Policy and Table 3.3. Marginal Contribution of 53 Institutional Assessment (CPIA) (see Elbadawi, Mavrotas, and Randa 2003, table CDF-like Development 4).39 The same analysis also finds that the fun- (Percent change in dependent variable associated with a 1% damental economic and social factors do not increase in the "Overall CDF-like Index") influence institutions and policy (the CPIA) Marginal contribution once the CDF-like index has been taken into Dependent of overall CDF-like index account.40 This suggests that these fundamen- variable (in elasticity form) tal factors shape policies and institutions through the development processes and strate- GDP growth 2.68 gies measured by the CDF-like indices. Institutions 0.57 (CPIA as a proxy indicator) Sustaining CDF-like development is more challenging in fractionalized societies. The litera- Adult illiteracy -0.89 ture suggests that societies with socially or economically divided societies and weak institu- Note: The table contains the derived long-run elasticities based on coefficients of the "predicted" overall CDF-like tions for mediating conflicts among group inter- index from the regressions of table 5 of Elbadawi, Mavrotas, ests are more likely to experience policy and Randa 2003 in the performance equation for growth, CPIA, and illiteracy. The mean of the predicted CDF = 0.58, reversals and thus be unable to sustain growth growth = 3.73, CPIA = 3.14, illiteracy rate = 32.24 [Top (see Rodrik 1999 and Elbadawi 2001). In CDF quartile of CDF = 0.66]. implementation, it has also been noted that sus- tainability becomes a problem when countries are facing economic crisis. In such countries, results from the non-randomness of the decision macroeconomic issues dominate social and to pursue a CDF-like development strategy. structural ones in terms of policy priorities (World Bank 2001a, p. 8). The analysis here CDF-like development strategies produce superior confirms this observation. The volatility of the development outcomes. The magnitudes of these overall CDF-like strategy indicator is positively marginal impacts can be translated into elastic- associated with terms of trade shocks, social ity estimates (table 3.3). These estimates suggest fractionalization, and autocracy (see table 4 of that moving up the ladder to a more compre- Elbadawi, Mavrotas, and Randa 2003). hensive CDF-like development strategy could have substantial development impact. While Impact on Institutions, Growth, and Literacy these results are only suggestive, they do imply A country's decision to adopt a CDF-like (or that the development impact of CDF-like devel- any other type of) development strategy is opment strategies is not negligible. Take the endogenous to country-specific socioeconomic case of Africa, which constitutes the most seri- and political characteristics. These same factors ous challenge for development. A recent study are also likely to be important determinants of finds that in a sample of 44 African countries, 8 country economic performance. The marginal had negative growth rates in 2000 and 14 had contribution of a CDF-like strategy to growth, growth rates of less than 3 percent (Elbadawi institutions, and literacy is estimated by employ- and Gelb 2002). Only 7 countries currently ing a model that accounts for the above compli- have growth rates high enough to prevent cations (see table 5 of Elbadawi, Mavrotas, and poverty from rising by 2015. For the remaining Randa 2003), and corrects for the bias that 37 countries, poverty is expected to increase rather than decrease under prevailing growth gies. For example, a 10 percentage point 54 rates. The results of this analysis suggest that if increase in the CDF-like index would accel- the 22 African countries with an average rate of erate growth by 9 percent for any given level growth of 1.5 percent were to improve their of aid. CDF-like development index by a third, to reach the median for South Asia (from 40 to Conclusions and Some Policy Implications 60), their growth rates would approach the 5 The following conclusions emerge from the percent threshold needed to prevent poverty econometric analysis. First, CDF-like develop- from rising. ment strategies, as well as institutions, are endogenous to "deep" country-specific charac- teristics, such as initial level of economic and These results...imply political development and extent of social frac- that the development tionalization. Second, these country characteris- tics appear to influence institutions through the impact of CDF-like processes associated with CDF-like develop- ment strategies. Third, sustaining a CDF-like development strategies development strategy is more challenging in is not negligible fractionalized societies with weak institutions for promoting inter­social group cooperation, especially when faced with external shocks. This finding has two important implications for Aid Effectiveness of CDF the design of future CDF implementation. More Development Strategies attention should be given to flexible and Having identified a likely positive development counter-cyclical assistance programs to help aid- impact of CDF-like development strategies, it is recipient and other middle-income countries natural to subsequently investigate whether smooth the impact of external shocks. At a these strategies make aid more effective--or, put deeper level, the PRSP and other lending differently, whether there are higher payoffs to instruments should provide more time and space be gained by providing aid to countries pursuing for a genuine national bargaining process to CDF-like development strategies. This analysis evolve and mature, especially in fractionalized is based on estimation of an extended growth societies. Fourth, CDF-like development strate- model, which in addition to accounting for gies appear to produce superior development policy environment also controls for the type of outcomes. Analysis of growth, poverty reduc- development strategy, as measured by the CDF- tion, and human development indicators seem like index (table 7, Elbadawi, Mavrotas, and to suggest that a CDF-like development strategy Randa 2003). has made positive marginal contributions to these pivotal development indicators. Finally, a CDF-like development strategies enhance aid CDF-like development strategy increases aid- effectiveness. Simulation analysis based on effectiveness, where simulations based on analy- the above models suggests that there could sis of growth suggest that there could be very high be high payoffs from providing aid to coun- payoffs to be gained by providing aid to countries tries pursuing CDF-like development strate- pursuing CDF-like development strategies. C H A P T E R F O U R Conclusions and Recommendations 57 The CDF principles are individually and collec- leadership, whether at the highest levels of gov- tively important. A major value added of the ernment, in line ministries, or among elected CDF initiative has been to bring all the individ- officials, remains a critical factor for making ual principles together as a package and promul- and sustaining progress on each of the princi- gate them on the global agenda, initially ples and for managing CDF processes so they through the personal advocacy of the World are complementary--e.g., country ownership Bank's President, and more recently through the and country-led partnership processes. In the PRSP mechanism. case study countries, government leadership appears fundamental to sustaining the long- While the CDF alone is not sufficient for term vision; fostering adequate processes for poverty reduction, it does comprise fundamental broad country ownership of that vision; manag- and necessary principles for strategic develop- ing development cooperation at the country ment planning and key development coopera- level; and enhancing the profile of monitoring tion processes, without which sustained poverty and evaluation findings by using them in deci- reduction would be unlikely to occur. The sionmaking. A lack of leadership in any of these process of change is young and the full benefits areas poses a key obstacle. The case studies also of the CDF approach are unlikely to be realized show that donors can play an important quickly, painlessly, or without cost. It is clear enabling role by supporting enhanced country that all parties will need to make changes if cur- leadership on various levels through capacity rent inefficient aid practices are to be rational- building, the use of more flexible financing ized and the ambitious development targets arrangements, and the simplification and har- set out in the MDGs and the international monization of processes. consensus that emerged from the Monterrey Conference are to be met. This chapter draws the evaluation's main con- clusions about the implementation of the CDF The evaluation also found that there can be principles, and makes recommendations for the tensions in how the CDF principles are applied, consideration of donors and recipient countries. specifically involving ownership and partner- Recommendations for donors apply to all ship, and the long-term focus and the emphasis donors, international financial institutions, and on results. These tensions have been longstand- bilateral and multilateral development agencies. ing in development practice and are neither The recommendations for donors include sev- novel nor unique to the CDF. There is also eral specifically directed to the World Bank as evidence at the country level that the CDF the initiator of the CDF and lead sponsor of the principles can be complementary. Country CDF evaluation. public sector reform and institution building 58 Long-term, Holistic that the PRSP requires for success are long-term Framework processes that will not deliver results within the life of a single PRSP cycle. Conclusions: Long-term development frame- works have operational meaning only if they are translated into affordable priorities through a Development strategies disciplined budget process. Development strate- gies without a hard budget constraint are merely without a hard budget "wish lists." Without a link to an annual budget process, there is no guarantee that resources will constraint are merely not be diverted away from priorities identified in "wish lists" the PRSP process to other ministry or donor interests. A development framework linked to the budget is thus necessary as (i) a rationale for Recommendation for recipient countries: the allocation of resources; (ii) a basis for coun- Strengthen the link between long-term frameworks try-led partnerships; and (iii) a framework for and budgets. Strengthening the link between assessing results. At the same time, the PRSP longer-term plans, such as the PRSP, and the and other similar medium-term instruments budget or MTEF should be a priority if countries need to be anchored in a longer-term framework are to maximize the operational usefulness of if they are to achieve long-range goals. the national development plan for making policy choices in the context of limited Five of the six case study countries now have resources. Donors should support such efforts. PRSPs that call for linking development frame- works to the budget process. While several Recommendation for all donors: Provide-long- countries, including Burkina Faso and Ghana, term assistance for capacity strengthening. This have adopted medium-term expenditure frame- should include sustained support for public works (MTEFs), few follow as rigorous a process sector reforms and institutional development. as Uganda in costing and setting priorities, and grounding the PRSP in the short-term budget Conclusions: If recipient countries are expected process through the MTEF. In Ghana, for exam- to adopt a long-term approach for solving devel- ple, the MTEF exists on paper, but has yet to opment problems, so too should donors. It is exert influence on everyday fiscal realities. encouraging that many donor countries are now increasing the share of budget and sector program The PRSP is a powerful tool for implementing support, signing more long-term indicative fund- the CDF principles, and from the perspective of ing agreements, and basing their assistance on advancing CDF-like principles, it merits contin- the recipient's own long-term development strat- ued support and development. For middle- egy and MTEF. Yet recipient governments and income countries and others not following the some donors express concern that program aid is PRSP track, different instruments and mecha- still vulnerable to sudden shifts in donor deci- nisms will be required, but have not yet clearly sionmaking. Recipient countries in particular emerged.41 For the PRSP to succeed, it will need argue that it is difficult to plan when aid flows are coherent and sustained external support, as well not predictable beyond annual commitments, as a lengthy process of learning by doing. The and disbursements are not timely or reliable. Recommendation for all donors: Provide pre- There has been some progress on this principle, 59 dictable and reliable financing. Donors need to pro- despite the many constraints. The discourse vide predictable, transparent, and multi-year has shifted from aid expenditures as a measure indicators of financing, based on clear, mutually of achievement to increasing acceptance that agreed country performance criteria. They should stakeholders should be held accountable for ensure that disbursement is not interrupted achieving development outcomes. The PRSPs unexpectedly. The PRSP provides an organizing and the high visibility of the MDGs have con- framework for multi-year commitments. tributed to this shift. SWAps have helped to institutionalize a results focus and MTEFs have Additional recommendation specifically for emerged as a viable vehicle for introducing a the World Bank: Improve cross-sectoral program- results orientation into the budgetary process. ming and implementation. The Bank's internal Still, many recipient countries appear to have structure, which organizes staff mainly by sec- adopted a results-oriented approach primarily toral specialties, can sometimes discourage to satisfy donors, at least initially. Application cross-sectoral dialogue and integration of inter- of the results approach is primarily limited ventions. This can exacerbate "silo" thinking or to specific aid-funded projects and has rarely inter-ministerial competition in client coun- been embedded in the normal operations of tries. Efforts that are cross-sectoral by nature, government. And donors continue to encour- such as gender or decentralization, also face age the development of measurement systems these challenges. Conscious effort is needed to that meet their own institutional needs first counteract the sectoral focus of organizational and foremost. arrangements and to support a holistic program that links vision, budgets, and results. Recommendations for recipient countries: Results orientation through greater accountability to the public. Worthwhile as specific monitoring Results Orientation mechanisms may be for learning and for accountability to donors, the key to a greater Conclusions: Progress in implementing the results orientation is strengthening the political results orientation principle has been the most accountability of the government to citizens. elusive of the four CDF principles. There are a But citizens' channels to demand results and number of reasons for this: weak technical participate in development policy debates have capacity, poor statistical data, inadequate incen- been very limited. The country case studies (and tives, inappropriate analytical tools, and lack of lessons from innovation in other countries) sug- demand for monitoring and evaluation results. gest several measures governments can under- Furthermore, efforts to ensure policy change and take to strengthen public accountability: performance have been driven primarily by needs of the donor community, and the desire 1) Train and expect public servants to open up for new forms of aid partnership have not yet led policy and program information channels, to significant change in the degree or type of aid and educate the public to participate. reporting required (see box 1.2). Finally, results orientation requires countries to have strong 2) Strengthen systems for financial manage- capacity not only in monitoring but also in ment, performance ("value for money"), and budgeting, execution of budgets, and financial regulatory audits, and internal and external and political accountability. accountability. 60 3) Provide development plans in languages and to educate their own constituencies (politicians, through concepts the broad public will audit offices, treasury, and the public) about the understand; use the media to support limited value of trying to attribute increasingly informed dialogue and demand for results. complex development outcomes to single- agency interventions. 4) Engage civil society organizations in budget and implementation monitoring and employ Donors should invest in further exploration and participatory poverty assessments (PPAs) exchange on the following: (a) how to establish and other monitoring approaches in order to country-owned M&E systems that bring obtain feedback from the poor and marginal- stakeholders together and that build from PPAs, ized on policies that impact them. budget monitoring processes, and other recent innovations; (b) how to include M&E of donor 5) Develop and use instruments such as "citizen performance; and (c) how to start a debate in service delivery report cards," used in Ghana donor countries about changing incentives; and proposed in Vietnam, among other pooling resources and results; and attitudes countries, or the recently passed Freedom of about aid, accountability, and donor attribution. Information Act in Romania. Additional recommendation specifically for Conclusions: Donors continue to overtax exist- the World Bank: Enhance the capacity of the ing monitoring structures by proposing complex, World Bank to track and analyze the implementa- special-purpose approaches, with indicators that tion of CDF principles and their impacts. The are unwieldy, difficult to substantiate, and con- tracking activity undertaken by the CDF form more to donors' reporting requirements Secretariat of the World Bank provides one than to the needs of service delivery manage- mechanism for periodic assessment of CDF ment. This is exacerbated by the proliferation processes. To permit rigorous causal analysis of of project implementation units (PIUs) for the impact of the CDF on development and project management purposes. Donor-estab- intermediate goals, the generation of quantifi- lished enclave monitoring systems are not sus- able tracking indicators on a regular and system- tainable, and can even weaken local capacity. atic basis should be a top priority. The CDF Secretariat should also work toward acquiring Recommendation for all donors: Strengthen and the necessary quantitative and analytical skills utilize country-led M&E systems. Donors should to undertake impact analysis on a regular basis. downplay individual interests in favor of effec- This analysis should take into account the tive joint action in strengthening and utilizing views of other stakeholders (donors as well as country monitoring and evaluation systems. recipient country governments and other key Real progress will be realized only when govern- groups) about the evolution, implementation, ments and donors view country development and impact of the CDF principles. outcomes as a joint product, to which donors contribute with the country in the lead. It will also require significant investment in capacity Country Ownership building. While there are great systemic gains to be had from making these changes, there are Conclusions: Governments and donors have currently few tangible incentives--individual or expanded their consultation activities for the agency--to motivate that change. Donors need purpose of strategy formulation with key stake- holder groups in civil society and the private poorest and women. Several consultative 61 sector. This process has helped to strengthen approaches will be required. ownership of reforms. But the ownership has been narrowly based when consultations have Recommendation for all donors: Work with been confined to the executive branch of the the government in devising an approach for con- government, supplemented by ad hoc consulta- sultations with elected officials and nongovern- tions with organizations that donors or govern- ment representatives. The views of diverse ment choose. Selected interest groups have interest groups, while valuable, do not sum to been given voice in these cases, while the wider the public good. Donors should work with public good--in the stewardship of elected rep- recipient authorities to establish the appropri- resentatives--has been overlooked. Parliaments ate role for elected officials at the national and and elements of civil society and the private local levels and ensure they have access to sector have complained of being marginalized in meetings. In consultations with interest the process (Burkina Faso, Uganda). groups, special efforts are needed to involve marginalized groups, typically the poorest-- particularly women--and private sector repre- Special efforts are sentatives, and to take account of their views. Materials and presentations should be easily needed to involve understandable to all local audiences, particu- marginalized groups... larly at the grassroots level. Donors should be prepared to provide assistance to strengthen and to take account the capacities of these groups to participate in strategy consultations. of their views Conclusion: Some elements of the PRSP process stand in the way of the principle of An important lesson from the experience of country ownership. For example, some country CDF-style participatory consensus building is governments believe that the PRSP requires that in countries with credible representative their development programs to focus too heavily institutions, it is desirable for the international on social expenditures. The fact that PRSPs development community to work through these must pass through the Boards of the Bank and institutions. But in order for parliaments and the International Monetary Fundis also thought governments to exercise their stewardship and to limit countries' range of options. Evidence accountability functions, they need better infor- from some of the country case studies suggests mation (such as PPAs) on the impacts of their that they resent what they see as Board decisions on different groups, particularly on the "approval" of what is supposed to be a country- poorest, who normally have little or no voice. owned document, the PRSP. Recommendation for recipient countries: To Additional recommendations specifically enhance country ownership, government and for the World Bank: parliament should consult among diverse interest 1) Clarify the Bank's openness to alternative groups. This embraces the full range of civil PRSP-consistent development strategies. Mis- society and the private sector, including perceptions of Bank policy could be miti- those who lack an organized voice, such as the gated by more openness by the Bank to 62 alternative and well-argued approaches to way (for example, the Health SWAp in Ghana), addressing poverty--for example, an approach sector strategies tend to be more coherent, and that emphasizes strengthening basic infra- information flow, monitoring, and resource structure. The resulting strategy should be planning are better than in other sectors. But it well buttressed by best practice retrospec- is too soon to tell if there will be sustained tive and prospective participatory and ana- improvement in higher-level development out- lytical instruments. comes, such as educational attainment and health status. 2) Differentiate more clearly the Board's roles in relation to the PRSP and the Country Effective country leadership of aid partnerships Assistance Strategy (CAS). The Bank should has been difficult to achieve because of the seek to communicate more clearly and con- asymmetries in the donor-recipient relation- sistently the distinction between the Board's ships. Some donors face domestic political resist- endorsement of the PRSP as a basis for con- ance to reducing the use of international cessional lending and its approval of strate- consultants and to harmonizing procedures. gies contained in the CAS. Careful wording Reform will require both donors and recipient in press releases following Board discussions countries to make significant changes in their can contribute to this, as well as clarifica- behaviors and processes, changes that may be in tions of the issue in the context of in-coun- conflict with the political environments of try PRSP consultations. donors. Although field initiatives, along with the quality of field staff and leadership, can influ- ence donor headquarters, the basic parameters Country-led Partnership for donor behavior are set at the headquarters level. In-country initiatives have only rarely Conclusions: The "new paradigm" of country- spearheaded changes in headquarters policies led partnership is intended to give the recipient and practice. One of the means for donors to country a leading voice with respect to the qual- facilitate country leadership is to provide finan- ity of aid it is receiving. Progress has been highly cial resources through mechanisms that apply uneven--across donors, across countries, across common pool principles, such as budget support sectors within countries, and across the dimen- through Poverty Reduction Support Credits sions of the country-led partnership principle. (PRSCs) and SWAps. At the same time, donors, Some donors in a number of countries have because they are accountable for public funds, risen to the challenge--or used it to underpin cannot accede to country leadership if recipient ongoing efforts.42 A few bilateral donors have countries are perceived to be suffering from cor- done a better job than multilateral agencies; ruption, economic mismanagement, and weak others have done worse. public expenditure management and accounta- bility. Causation can run in both directions: Some recipient countries also are beginning to good country performance can encourage flexi- meet the challenge. Where this has happened, ble financial support, i.e., budget support, and the quality of the aid program has improved. provision of flexible support facilitates country This is reflected in intermediate outcomes in leadership. Both influences can be observed in some sectors in at least one country (primary Uganda's experience, but a modicum of country school enrollment and access to safe water in performance was required before significant flex- Uganda).43 Where SWAps have been under ible support was forthcoming. Many countries will need capacity strengthen- recipient countries. The present unsatisfac- 63 ing in order to fully assume leadership of aid tory situation has continued for many years in management and coordination functions. But spite of years of multidonor harmonization capacity development is a long process and efforts (for example, to harmonize procure- cannot be effective as long as individual donors ment). A challenge, therefore, for follow-up propose different and sometimes incompatible to the February 2003 Rome Conference and forms of support, each following different other harmonization initiatives will be to timetables and goals. break from business as usual and introduce a new paradigm. This will involve giving the Recommendations for all donors: More impe- recipient country control it has not previ- tus is needed to accelerate the alignment of ously had. To strengthen the role of the coun- assistance strategies with country strategies, try and to bolster accountability for achieving information sharing between donors and part- progress, one proposal is to establish a system ner countries, focused local capacity building of regular country-level review panels com- with a long-term perspective, greater selectivity posed of independent reviewers representing of donor-funded activities, and harmonization of the recipient country and donors. The panels donor procedures and practices. Donor-only would review donor as well as recipient coun- coordination exercises, while they can consti- try performance against a mutually agreed tute a first step toward harmonization, and may code of conduct and targets. Results would be be the only option in immediate post-conflict accessible to the public and published regu- and some low-income countries under stress larly (e.g., for CG meetings). (LICUS), do not by themselves foster recipient country ownership. A second step would be 3) Decentralize staff and delegate more authority to recipient country leadership in moving to the field. Effective aid coordination at the greater strategic alignment, programmatic selec- country level requires that donor field offices tivity, and harmonization around country strat- have more flexibility and resources to foster egy, policies, standards, and procedures. recipient country leadership. To move in this direction, donors should provide their coun- 1) Step back from micro-managing the aid process try field offices with enough delegated at the country level. Country-led partnership authority and resources to participate fully in implies that donors are prepared to step back pertinent country-led aid coordination activ- from micro-managing the in-country aid ities. And they should select field staff who process and aid relationships to give coun- have proven partnership and relationship- tries space to innovate and take risks, and to building skills as well as the requisite subject make mistakes and learn from them, and at area expertise. For smaller donors, greater the same time, provide the capacity building selectivity may require them to specialize and and resources countries need to fully take upgrade their technical capacity. over aid management. This will require mutual trust and transparency on both sides. 4) Plan for phase out of PIUs. While a case can sometimes be made for temporary project 2) Give the recipient country voice and oversight implementation units (PIU) under conditions over aid quality. Changes in donor policies and of weak government capacity, every PIU practices require actions at the headquarters should be accompanied by a plan to phase out and international levels, as well as within over the life of the project or program. Salary 64 incentives typically associated with PIUs are a Recommendations for recipient countries: thorny but critical issue for donor harmoniza- 1) Put responsibility for aid coordination at a high tion. Training and capacity building would level of the government. If countries are to normally accompany the phase out process, grasp the leadership of development partner- and, if needed, which is often the case, public ships, they need to discourage fragmented, service reform, including pay reform. disparate relationships among individual donors and individual ministries and agen- Additional recommendations specifically for cies within the recipient government. The the World Bank: The country case studies high- aid management mechanism should be at the lighted some aspects of these recommendations cabinet (or other high) level, to ensure coop- that apply in particular to the Bank. eration across ministries--for example, at the level of the Minister of Finance and 1) Continue decentralization and delegation of Planning. One indicator of strong leadership authority to field offices. The Bank has made is when the government rejects any donor considerable progress in decentralization over offers that do not fall within the national the last five years. This has involved not only plan or MTEF. the placement of country directors in the field, but also the fielding of additional head- 2) Implement and enforce procurement and other quarters staff and decisionmaking authority. rules that will engender the confidence of donors. Donors require confidence in the integrity 2) Select, train, and reward staff--in part--on the and efficiency of the government's resource basis of their partnership performance. This management systems, especially procure- applies not only to field-based staff but also ment and accounting and auditing. This is to headquarters staff who service country especially true if donors are to accelerate programs (e.g., sector specialists). adoption of common pool funding mecha- nisms. One way to increase their confidence 3) Practice what the Bank preaches regarding har- is to promulgate procurement and other rules monization and simplification, program or that will meet donors' expectations, and to budget support, and selectivity and "stepping implement and enforce them consistently. back." Complexity of Bank procurement Another way is to encourage civil society to procedures was a common complaint heard help in reducing corruption by budget and by case study teams. Procedural simplifica- implementation monitoring, promoting tion as well as harmonization are needed. transparency, and educating the public. Transparent and timely provision of docu- ments should be the norm, given that diffi- culty in obtaining Bank strategy and The Road Ahead procurement documents was cited in several case studies. The Bank often leads in provid- and an Opportunity ing flexible instruments, such as PRSCs and sector budget support, but the case studies This evaluation concludes that progress in show that this is not uniform practice. There implementing the far-reaching changes posited is still a tendency for the Bank to involve by the CDF has been uneven, with the broadest itself in every sector. The CDF implies that it progress occurring in countries that have been should cede donor leadership when others applying one or more of the CDF principles for have comparative advantage. a number of years. This finding is not surprising given that it is still early and the CDF principles How to start a debate in donor countries 65 are about changing norms, behaviors, and insti- about agency incentives, public attitudes tutional practices--change that does not toward aid, pooling resources and results, and happen easily or overnight for either donors or the role of Audit Offices and Treasuries in recipient countries. Transparency and mutual compounding the problem trust are required of all parties. Therefore, con- tinuous political leadership and sustained will by How to expand learning between recipient all major development actors are needed if countries--e.g., disseminating Uganda's expe- today's dysfunctional aid practices are to be rience with the MTEF to other countries. changed. If not, internal politics, incentives, and capacity constraints will continue to frus- Promising opportunities have recently emerged trate the emergence of the new behaviors that for donors and recipients to move ahead. For the CDF encourages. instance, on the harmonization of donor proce- dures and practices--a heretofore largely intractable component of the country-led partnership--a momentum for change is build- Transparency and ing. The consensus that emerged out of the mutual trust are Rome High-Level Forum on Harmonization (as contained in the Rome Declaration on required of all parties Harmonization, February 2003) calls for donors and recipient countries to: (1) utilize and strengthen existing mechanisms to maintain peer pressure on implementing harmonization agree- One area that needs more attention is the rela- ments; and (2) conduct stocktaking meetings in tionship between the implementation of the CDF early 2005 in order to contribute to the review of principles and certain sectors and segments of soci- Monterrey Consensus implementation. There are ety. The analysis summarized in Chapter 3 sug- also a number of opportunities to bring the CDF gests that failure to involve adequately the principles into fuller implementation. These private sector in CDF processes gives them little include following up with the New Partnership or no impact on the business environment in for African Development, the Monterrey and developing countries. Country case study evi- Rome consensuses, the ongoing work on harmo- dence indicates that marginalized groups, such nization of the SPA, and the World Bank's initia- as the poorest and women, are not being ade- tive on monitoring and measuring for results, quately consulted in the formulation of develop- including the increasing adoption of MDGs in ment strategy. PRSPs and other country development frame- works. A related ongoing effort is the develop- The evaluation suggests several important areas ment of a Millennium Development Strategy. The that would benefit from expanded learning efforts, key elements of such a strategy are already to be including research and exchange of experience: found in the CDF and in the efforts of some countries to implement the CDF as a means to How to establish country-owned monitoring achieve the MDGs. The World Bank, in cooper- and evaluation (M&E) systems that bring ation with other development partners, should stakeholders together and build from innova- play a lead role in integrating the CDF principles tions in government and civil society into these and other global initiatives. A N N E X O N E The CDF Evaluation at a Glance 67 January 1999 proposes conducting research on the CDF. The World Bank President James D. Wolfensohn two units agree to participate in the CDF evalu- introduces the Comprehensive Development ation as a joint effort. Framework (CDF). October 2000 Late 1999 The CDF evaluation is officially launched at a The Operations Evaluation Department (OED) CDF Research Evaluation Workshop held on is asked by the Executive Board of the World October 19 & 20, 2000, that brought together Bank to conduct an evaluation of the imple- about 50 persons from CDF pilot countries, mentation of the CDF. The Bank's donors and other development agencies, NGOs, Development Research Group (DECRG) also and academic institutions. Governance of the CDF Evaluation CDF Evaluation Steering Committee Members: African Development Bank, Bolivia, Canada, Denmark, EU, Ghana, JBIC, the Netherlands, Norway, OECD, Oxfam, Pakistan, Romania, Sweden, Switzerland, Uganda, UK, UNDP, UNECA, USA, World Bank (CDF Secretariat) Management Group Evaluation Secretariat Members: Gregory K. Ingram (DGO-OED), Paul OED and DECRG staff Collier (Director, DECRG), Marianela Members: John Eriksson, Laura Kullenberg, Zeballos/Marcelo Barron (Bolivia), Niels Mirafe Marcos, Ibrahim Elbadawi, John Randa, Dabelstein (Denmark), Emmanuel Tumusiime- Julia Ooro, Tourya Tourougui Mutebile (Uganda) Funding of the CDF Evaluation Financial Contributors In-Kind Contributors Canada, Denmark, the Netherlands, Asian Development Bank, Japan, and UK Norway, Sweden, Switzerland, and UK 68 Products of the CDF Evaluation Six Country CDF Evaluation Four Thematic Case Studies Synthesis Report Studies Bolivia Country Authors: Long-Term Holistic Case Study J. Eriksson, Framework Authors: N. Boesen, I. Elbadawi, Authors: A. A. G. Ali, L. Kullenberg, C. Lancaster, A. Disch J. A. Peres, J. C. Requena A. Scott, P. Collier, L. Kullenberg, Ownership and Burkina Faso Country C. Soludo, Participation Case Study M. Marcos Authors: H. Handousa, Authors: D. Lallement, C. Lancaster D. McMillan,K. O'Sullivan, Advisors: P. Plane, K. Savadogo H. Kohama, D. Pressley, Country-led A. Scott, Partnership Ghana Country S. Wangwe, Authors: T. Holmgren, Case Study S. Maxwell C. Soludo Authors: N. Boesen, L. Kullenberg, T. Killick, A. Oduro, M. Marcos Results Orientation Authors: H. P. Hatry, Cross-cutting K. Yansane Romania Country Econometric Study Case Study "Development Impact Authors: J. Eriksson, of CDF-like Strategies" L. Salinger, D. Sandu by I. Elbadawi, G. Mavrotas, and J. Randa Uganda Country Case Study Authors: J. Eriksson, R. Rwampororo, D. Pedley, M. Marcos Vietnam Country Case Study Authors: A.J. Morten, L. Kullenberg, R. Mallon, K. Nishino, H.M. Tien A N N E X T W O Approach Paper for a Multi-Partner 69 Evaluation of the Comprehensive Development Framework Background evaluation of the implementation of the CDF. At about the same time the Bank's Development This paper describes the scope and basic compo- Research Group (DECRG) proposed conducting nents for a multi-partner, multi-country evalua- research on the CDF. This led to an agreement tion of the Comprehensive Development by the two units to undertake the CDF evalua- Framework (CDF). It is intended to serve as a tion as a joint effort. To launch and help guide basis for decisions by: the evaluation, about 50 persons from CDF pilot countries, donors and other development agen- CDF pilot and other countries to participate cies, nongovernmental organizations, and aca- in the evaluation demic institutions were invited to a workshop in Washington D.C. on October 19 and 20, 2000. Development agencies to contribute finan- A subsequent session on the CDF evaluation was cially and intellectually to the evaluation held at the November 2000 meeting of the OECD/DAC Evaluation Working Party. The Evaluation Steering Committee and the Board Committee on Development Effective- Based on the interest evinced at these meetings, ness (CODE) of the World Bank to empower 30 representatives of donors, developing coun- the Evaluation Management Group to pre- tries, and other development organizations were pare a detailed evaluation design. invited to a meeting in Paris, January 16, 2001.45 This group agreed to form a multi-part- The CDF comprises a set of principles that a ner CDF Evaluation Steering Committee. It number of developing countries and develop- confirmed that the design and governance of ment assistance agencies have been seeking the evaluation should reflect the CDF principles to put into practice over the last two years. of country ownership, country-led partnership, Although the CDF, as such, was articulated and participation, with full involvement of by the World Bank President in January 1999, development agencies46 and recipients, includ- the principles on which it is based are distilled ing civil society and the private sector. It also from development experience over the last highlighted the need for a timely assessment of five decades.44 how CDF principles are being implemented within countries' development processes, taking In late 1999, CODE asked the Operations full account of the Poverty Reduction Strategy Evaluation Department (OED) to conduct an Paper (PRSP) initiative.47 70 Objectives and Scope The primary orientation of the evaluation is on self-evaluation and learning. All main actors and The objectives of the evaluation are: (1) to their behavior, processes, and practices with assess how the CDF is being implemented on respect to CDF implementation will be subject the ground; (2) to identify the factors that have to evaluative review, whether donors, govern- facilitated implementation of CDF principles ments, civil society, or private sector. The eval- and those that have hindered it; and (3) to uation will seek to model sustainable ways of promote learning and capacity development implementing the CDF principles.48 in countries where CDF principles are being implemented. The evaluation will deal with the relevance, efficacy, and efficiency of the overall The Functionality of development assistance system in selected CDF Implementation: A countries, both on the ground and at the policy level, including linkages to the international "Balance Sheet" Approach development assistance architecture and the International Development Goals (IDGs). The CDF principles as outlined in box A2-1 have achieved broad acceptance. The evalua- A characterization of the four CDF principles tion will ascertain whether they have been and some of their properties is given in box A2- practiced and, if so, whether the ways in which 1. The elaboration of these principles continues they have been implemented have been func- to evolve. There is no authoritative set of defi- tional for pursuing overarching development nitions. This evaluation itself should help to objectives, such as the IDGs.49 This requires an refine their meaning. identification of the possible benefits and costs of CDF implementation. Twelve potential ben- The evaluation will consist of the following efits and ten potential costs are suggested in box main components: A2-2. This list is not exhaustive; it will be expanded and refined for the evaluation design. Case studies of five CDF pilot countries and The benefits and costs will be framed as one non-CDF pilot country hypotheses with associated "tests," or evalua- tion questions. Five thematic studies and a cross-cutting econometric study A Logical Framework for An overall synthesis study. the CDF Principles The primary target audiences for the evalua- tion are the key government, donor, civil soci- The employment of a "log frame" helps to clar- ety, and private sector stakeholders in the case ify how the CDF principles contribute to study countries, and senior managements and higher-order development objectives. Inputs governing authorities of development agen- are the actions taken and costs incurred by cies. Secondary target audiences are stake- partners to create a holistic, long-term devel- holders in other developing countries, and opment framework, and to improve the coun- tertiary audiences are the taxpaying citizens of try ownership, country-led partnership, and all countries. results orientation of poverty reduction Box. A2-1. The Principles of the Comprehensive Development Framework 71 I. Long-term, Holistic Development Framework Addressing a country's development priorities requires a strong anchor in an appropriate, consistent, long- term vision of its needs and ways to address them. The vision also needs to be holistic; that is, it needs to: Be all-encompassing, taking into account simultaneously the key economic and financial issues, the requirements of structural change, the social issues, and other factors impinging on the country's social and economic development Cover the inter-linkages between sectors, since success in one sector is often linked integrally to progress in others. Development interventions funded by the government and donors need to be carefully reviewed and appropriately sequenced in order to obtain the optimal mix of policies, programs, and instruments within an overall hard budget constraint. II. Country Ownership The country needs to be in the driver's seat, owning and directing the development agenda with the sup- port of all other actors. The executive branch of government, therefore, needs to build consensus internally within the government, including the legislative structures and all other levels of government, and to con- sult with all stakeholders outside government, including the private sector and other elements of civil soci- ety, as well as with the country's external partners. The purpose of such consultation is to draw out ideas, knowledge, and opinions and to promote the broadening of consensus on the strategy expressed in the long-term, holistic vision. Enabling such ownership will often require strengthening of capacity in govern- ment and elsewhere in society. III. Country-led Partnership Enabling the country to be in the driver's seat requires strong partnership among the executive branch of government, other levels of government, the legislature, local authorities, civil society, the private sector, donors, international agencies, and other development actors. Such partnership should: Bring together, within a single framework, under government leadership, analytical and diagnostic work. Align donor actions to the national strategy and promote selectivity to avoid duplication. Reduce wasteful competition. Encourage common procedures among all development partners. Support the government's lead in managing aid coordination. IV. Results Orientation A country's national vision needs to link its overall aim to concrete development results, in a way in which progress toward the aims of that vision can be assessed. These development results are those sought through a broad-based national dialogue process. Since a key aim of the CDF is more effective and sustain- able poverty reduction, the specific development results sought should be guided and informed by the Millennium Development Goals (MDGs), which have largely emerged from agreements in U.N. conferences during the 1990s. Source: Drawn in part from World Bank 2000a, pp. 80-81. 72 Box A2-2. A Balance Sheet Approach to Evaluating the CDF: An Illustration Potential Benefits of CDF Implementation Long-term, Holistic Development Framework The CDF focus on results, the long term, and a holistic vision promotes the quality of development policy and strategic planning and the link between planning and budgeting. This focus also improves the quality of public expenditures, and the capacity for public expenditure management and sustainable resource management in countries. Country Ownership The CDF fosters country ownership, based on "rules of the game" that are mutually agreed by development partners and are facilitated by changed behavior on the part of IFIs, bilateral donors, and other development agencies to create space for ownership. Such ownership, in turn, leads to better policies and processes than old-style conditionality (e.g., geared to greater equity and faster poverty reduction because of more inclusive decisionmaking). Holistic approaches have led to a better combination of quantitative and qualitative assessments of poverty in decisionmaking and policymaking. The CDF leads to broad-based ownership, anchored around the legislature in a democratic system, that recognizes other stakeholders (such as the civil service, NGOs, business and professional groups), as well as the need for greater political independence of some institutions (e.g., the central bank). Broad-based ownership leads to less policy reversal, better implementation, and better policy design. Vulnerable and marginalized groups, women in particular, have been better represented and more involved as a result of implementation of CDF principles. Country-led Partnership The CDF matrix--the categories in the rows (sectors) and columns (development actors and part- ners) and their degree of aggregation--can influence the incentives for development agencies in a way that fosters collective actions among them in a context of partnership. Implementation of the country-led partnership principle will result in greater coherence and efficiency (lower transaction costs) of aid programs. Results Orientation In turn, the ensuing new aid regime will enhance accountability and good governance. A results focus leads to a more flexible and immediate means for citizens to be involved in providing direct feedback on the quality of government-provided services. Potential Costs of CDF Implementation Country Ownership Enhanced ownership may lead to programs that sharply violate some broadly accepted develop- ment policies, and "old-style" conditionality may be the only alternative acceptable to donors. To the extent that there are only a few governments involved in the CDF, yardstick convergence generated by peer pressure may not be possible to realize under broadly based ownership. Box A2-2, continued... 73 Similarly, country capacity may be so weak under some conditions (for example, conflict and post- conflict countries) that reliance on country ownership will need to be circumscribed. The implementation of CDF principles may be perceived by civil society organizations as being imposed by donors. Broader participation could result in loss of prioritization; diluted role of professional expertise; and/or undue influence of narrow but well-organized lobbies. The CDF may tend to undermine sustainability of growth, because more inclusive decisionmaking might make it harder to sustain growth-enhancing policies. Broad-based participation could be very time- and transactions-intensive. Country-led Partnership Greater emphasis on partnership could result in higher transaction costs of aid delivery if develop- ment agency partners do not agree to harmonize procedures and conditionality. Closer alignment of donor and governmental priorities could come at the expense of civil society­ government relations. With intensified coordination, major donors may still not adequately consult relevant stakeholders. Results Orientation Could lead to excessive focus on indicators and bureaucratic proliferation of "score cards." programs. Outputs are the changed behaviors of fully articulate four log frames, one for each of partners, reflecting the CDF principles. The the CDF principles, and show how they feed purposes are the country-enabling conditions, into higher-level purposes and goals, thus com- as reflected in country-level policy, institu- prising a composite log frame. tional, and aid quality outcome indicators. Goals are measured by impact indicators defined case by case in light of the International Other Methodological Development Goals (IDGs). The assumptions on the basis of which the results chain is con- Considerations structed will be specified in each case. The log frames will help to connect the balance sheet Country-specific and thematic studies will be approach with the aid quality considerations, guided by the same conceptual model. This e.g., the role of country ownership in improv- will enhance cross-fertilization and compara- ing development results (a balance sheet type bility, though there will be room for useful of analysis) will depend in part on whether aid differentiation in terms of methodology of flows are influenced by the policy environment analysis, and tailor-made features will be intro- (an aid quality consideration). Costs and ben- duced to respond to individual country needs. efits and core evaluation questions will emerge Initial results of the thematic studies, which from the log frames. The Design Paper will will be desk studies, will be shared with the 74 country case study teams for potential in- The country studies will not constitute "country depth exploration. evaluations" as normally undertaken by the World Bank and other development agencies, Additional features of the design and methodol- although they will draw on relevant methodolo- ogy for the evaluation will be covered in the gies to assess the performance of development detailed Design Paper to be developed as a next actors. They will be planned and designed in full step in the process, taking full account of com- consultation with country authorities in order to ments made by Steering Committee members. reflect their special interests and their individ- For example, an aspect that will be addressed in ual circumstances. To the full extent feasible, the Design Paper is the potential interrelation- they will be implemented with participation ships among CDF principles and the possibility from the various partners active in the country-- that certain sequences may be associated with government, civil society, the private sector, more lasting impact than others (e.g., country donors, and other development agencies. These ownership before other principles). The evalua- stakeholders will constitute a country steering tion itself is expected to illuminate such rela- committee for each case study. The aim will be tionships. The methodology will be subject to to initiate a process that informs decisionmakers further refinement during the early stages of and leads to institutional change in support of implementation of the country and thematic poverty reduction. studies, as indicated below. The selection of case study countries will be Country Case Studies mainly determined by the interest of the coun- Country studies will review the experience of try concerned and its development partners to five CDF pilot countries and one non-CDF participate actively in the evaluation. Other pilot country where one or more CDF princi- selection criteria will include maintaining bal- ples have been applied.50 Given the short ance in terms of Regional coverage and avoiding period of time since the CDF has been con- overlap with other related evaluative efforts, sciously implemented, it will not be possible for such as the ongoing SPA-sponsored country the country studies to assess the impacts of CDF case studies of PRSP implementation in eight implementation on the overarching develop- African countries and the UNECA "PRSP ment goals represented by the IDGs. However, Learning Group" initiative. At least one non- the country reviews will throw light on the PRSP CDF pilot country will be included. evaluability of individual CDF programs in terms of the linkages intended by stakeholders The first field visit to a country will consist of a between CDF processes and priority develop- scoping mission of one or two persons, including ment goals. A main focus of the country studies at least one member of the evaluation secre- will, as explained above, be on changes tariat team. They will work with the country observed in outcomes, outputs, and inputs in steering committee and a local institution to:51 connection with the extent and quality of implementation of the CDF principles. Some of Agree on the detailed terms-of-reference these changes should favor positive develop- and implementation modalities for the coun- ment impacts, while others could be inimical to try study. development given the tensions (along with the synergies) that have been shown to exist Assemble and analyze pertinent literature, among the CDF principles. documents, and data sources. Design questionnaire surveys of key stake- the creation of an ongoing mechanism for peri- 75 holders to be subsequently administered by odic assessment of CDF processes and feedback the local institution. to decisionmakers. Design key informant and focus group inter- In those countries where PRSPs have been or are view protocols for the second visit.52 being prepared, the relationships between the CDF and the PRSP will be explored, taking into Plan and make arrangements for a more account the early stage of PRSPs. A thorough extensive second visit, to take place four to review will be made of ongoing PRSP monitor- six weeks after the first visit (after the results ing efforts, such as those conducted under the of the questionnaire surveys are available). auspices of the Strategic Partnership with Africa (SPA) and the U.N. Economic Commission for The second field visit will be composed of two-to- Africa (UNECA). The objective will be to avoid three highly qualified professionals, drawn from, duplication and to explore possibilities for com- or representing, the key development agencies plementarity and cooperation.53 The CDF eval- active in the country. They will join with a local uation should help define the methods and team composed of a like number of individuals processes for an in-depth PRSP evaluation at a drawn from the local steering committee and/or later stage of PRSP implementation. local partner institution. The combined team should be skilled at facilitating interviews and Thematic Studies workshops, as well as having good analytical Six thematic studies are proposed: one on each skills. They will conduct key informant and focus of the four CDF principles; one on CDF-PRSP group interviews and analyze the results of the linkages; and a paper that will attempt a cross- various data sources, including surveys. Meetings country econometric analysis. These studies will or workshops will be held with key stakeholders, give particular attention to the impact on aid including decisionmakers, to discuss and validate effectiveness of donor and development agency the findings of the analyses of surveys, interviews, behavior as reflected in their policies, proce- and other data sources, drawing out their policy dures, and practices.54 and programmatic implications. The roles of all partners in the development of the country, Although the CDF as such was introduced only including donors and other development agen- recently, its principles were previously practiced cies, will be assessed through these data-gathering at various times in different countries. and workshop activities. Senior officials of the Therefore, the cross-country analysis will meas- main donors and development agencies will be ure whether countries and development agen- invited to join decisionmaker-level workshops cies that practiced something close to the CDF during the last day or two of this second mis- principles performed better (in terms of sion--or perhaps slightly after the mission, when processes and outcomes) than countries and a draft report is ready. agencies that did not. The method for testing this hypothesis will utilize a modified-control- An important value expected to be achieved is group model. The analysis will require indica- the learning that will take place through the tors of "closeness" to CDF principles, processes, process, including the nurturing of domestic intermediate outcomes, and development monitoring and evaluation capacity develop- impacts. The log frames, to be developed more ment. Another objective will be to encourage fully in the Design Paper, will help to guide these analyses. Examples of types of indicators papers. At the same time, a dedicated thematic 76 to be used and/or developed for each of the four paper on CDF-PRSP linkages would make a dis- principles follow. tinctive contribution. While it will be impor- tant to analyze the linkages between the CDF Long-term, holistic development framework. and PRSPs in each of the four papers, these by Indicators of country priorities, as well as themselves would be rather limited and partial sector balance, will be derived from various analyses because they would be focusing on the sources, including the Bank's Social and marginal effects of individual CDF principles. A Structural Reviews (SSRs). This will permit thematic paper focusing on CDF-PRSP linkages testing whether countries where these prior- could provide a valuable input to the World ities were part of a long-term, holistic vision Bank and other multilateral and bilateral devel- performed better. opment institutions as they strive to ensure that the PRSP initiative is appropriately situated in a Country ownership. Indicators of country context of a holistic development vision. ownership have been developed by OED. These will be adapted to assess the condi- The sixth paper will attempt to link the range of tions under which greater ownership leads to intermediate outcomes and processes identified better results. Equally, the extent of partici- in the first five papers to key development pation can be approximated by governance impacts. Several indicators of development indicators being developed. impacts will be used, including aggregate meas- ures, such as GDP per capita; distributional indi- Country-led partnership. Various proxies have cators, such as the income of the bottom been proposed for the quality of partnership quintile; and, where available, other indicators and of aid coordination. For example, indica- of well being, such as infant mortality, life tors of extent of tied aid, project vs. program expectancy, and educational attainment. To aid, and harmonization of development facilitate comparison, among the indicators to agency processes will be used to examine the be employed will be those also used to measure linkage of these factors to outcomes. progress toward the International Development Goals. The focus of this paper will therefore be Results orientation. Indicators of focus on to assess the functionality of the four CDF prin- poverty reduction and other related dimen- ciples as a means for testing their potential sions of development outcomes and impacts, influence on development impacts. The main availability of long-term development frame- elements of the methodology are: works, and learning from results will be developed and tested for both countries i. Analysis of association/correlation among and agencies. functional indicators of CDF and development outcomes and impacts: The first four papers will focus on developing Hypothesis on possible association/correla- detailed indicators of "functionality" for these tion (though not necessarily causation) principles; while the fifth paper will address Corresponding preliminary methods linkages to the PRSPs. The overarching devel- of analysis opment objective of achieving a meaningful and lasting impact on poverty would be a cross-cut- The potential value of the exercise for ting issue in all four of the CDF principles the country-specific case studies. ii. Estimating the marginal contribution good policy environment for aid effectiveness. 77 of the CDF: However, effectiveness is also constrained by the quality of aid, including its instruments, delivery Framework for analyzing the marginal contri- mechanisms for knowledge and resources, and bution of a CDF-like environment to devel- donor coordination. Recipients and develop- opment, controlling for the counterfactual of ment agencies have a mutual responsibility for "what would the treatment country have development outcomes and the distinctive done without the CDF-like experience" accountabilities and reciprocal obligations of all Potential implications of the results for the parties. This is all the more important given that country-specific case studies. the developing countries and regions where poverty is the highest and institutions are the The Design Paper will elucidate additional tech- weakest, are also the ones that are likely to be nical details of the methodology to be employed the most aid-dependent. Although it is clearly in the analysis. not the only consideration, improvements in the quality of aid in these countries can make a crit- The foregoing suggests the following implica- ical contribution to achieving the levels of tions for analyzing the impact of CDF imple- growth required for poverty reduction on a sus- mentation on development outcomes: tained basis. Unfortunately, these countries com- monly experience what might be characterized as an "aid-bombardment" syndrome.57 This syn- Processes and intermediate outcomes of CDF drome (an unintended consequence of poor aid implementation are inseparable from the task of evaluating its effect on development impacts.55 coordination) is apparent in countries in which the sheer volume of resources and numbers of development agencies, activities, and complex Development partners are often faced with and inconsistent procedural requirements over- real tradeoffs when, for example, country whelm the government's capacity to plan, ownership produces programs that donors budget, manage, monitor, and evaluate. and development agencies find difficulty agreeing with.56 Meaningful improvement in aid quality will require greater coherence, selectivity, and effi- Adoption/implementation of CDF princi- ciency in development agency support of country ples is also influenced by the initial condi- development strategies. It will also require more tions relating to development perform- effective support for capacity strengthening so ance, intermediate outcomes, and processes, that recipient countries with a sound policy among others. framework can assume greater leadership and responsibility for the management and coordina- The analysis should be placed in a broader tion of aid resources. Success in tackling these and more dynamic context that accounts for issues cannot be achieved solely at the country the process of learning and capacity building level; actions are also required at the headquar- as the CDF proceeds. ters levels of development agencies. Therefore, both the country and thematic studies of the Assessing the Quality of Aid CDF evaluation will examine the quality of Recent research and policy work on develop- country-led partnerships and their relationship to ment aid has focused on the importance of a mandated aid procedures and to development 78 Box A2-3. Global and Country-Level Partnership Issues Expanding country-led partnerships and participation. Partnerships, coordination, and dialogue are needed to build consensus on coherent programs. How do donors translate the concepts of part- nership and participation into decisions on aid allocations--including to countries with views on devel- opment that differ from theirs? How do donors interact with civil society and with representative institutions--particularly parliaments--in ways that ensure they are well informed and properly involved in aid programs and processes and yet respect local institutions, policies, and processes? How might a "code of conduct" help? Reducing aid delivery transactions costs. Harmonizing aid delivery policies and procedures, so as to reduce the heavy burden they often impose on poor countries, may well require development agencies to seek changes from their authorizing environments. To what extent are they prepared to do this? Strengthening capacity and fostering country ownership. Are development agencies prepared to support programs for skills retention rather than for expatriate technical assistance? Are they willing to support comprehensive multi-donor programs and not fragmented programs that tend to undermine capacity (e.g., with their own administrative procedures and implementation units, ad hoc budgets, etc.)? Decentralizing development. Many countries are devolving greater responsibilities and authority to local governments and communities so as to be more responsive to the needs of the poor. How can aid agencies support this process and at the same time strengthen the capacity of local entities to manage development programs? Moving beyond boundaries. Aid mechanisms mostly focus on countries, but the twenty-first century will see a growing need for a range of international and regional public goods (e.g., in the areas of agricultural research, vaccine development, knowledge creation and acquisition, and conflict manage- ment). How should mechanisms for regional aid delivery be enhanced to support such efforts? Enhancing debt relief. It is well established that high indebtedness reduces the quality of aid and per- petuates aid dependency. Excessive debt creates expectations of future taxes and policy reversals, which reduce the incentives for current investment. High fixed debt service obligations increase countries' lever- age and raise uncertainty, especially if donor funding is decided on a short-term basis. Has incremental debt relief through the "HIPC" initiative proceeded at a sufficient pace to make a meaningful impact? What basis for selectivity? Reallocating aid toward countries that are poor but maintain good poli- cies would increase the development effectiveness of aid. In the past donors have undermined incen- tives by providing aid even when conditions are unfavorable. Is assistance being allocated on a more selective basis? Moving away from aid dependence. Though aid cannot be phased out rapidly, plans should be made to free countries from aid dependence. Such plans need to be endorsed by recipients and their development partners and anchored in mutually agreed strategies of making recipient countries eco- nomically competitive and reducing poverty. What role can donors and development agencies, includ- ing the World Bank, play in supporting the agenda of poor countries in the WTO to ensure that WTO rule making mechanisms are made compatible with the development requirements of poor countries? Source: Adapted from OED 1999; World Bank 2000d. effectiveness on the ground. In this connection, studies and presented in an overall synthesis 79 ongoing work of the OECD Development report. Based on these conclusions, the report Assistance Committee (DAC), such as its Peer will frame recommendations for consideration Reviews and the new Bilateral Task Force on by members of the Steering Committee and the Donor Practices, is particularly relevant. entities they represent. A range of partnership and aid quality issues at the country and global levels are presented in Dissemination Strategy box A2-3. While such issues as those dealing with international public goods, debt relief, and The workshops will be a major dissemination trade cannot be covered in depth in this evalu- vehicle for the country studies. A dissemina- ation, they do relate to aid effectiveness. These tion strategy for the final products, such as linkages will be examined in the country and the overall Synthesis Report and selected thematic studies where relevant. thematic and country studies, is an important issue that will be addressed in the Design Synthesis Report Paper. The proposed budget provides for a workshop in late calendar 2002 to discuss Cross-cutting conclusions and lessons will be the main findings and conclusions of the drawn from the country studies and thematic Synthesis Report. A N N E X T H R E E CDF Principles: Working Definitions 80 The following "working definitions" are drawn 2) Regular, broad-based stakeholder partici- from President Wolfensohn's January 19, 1999, pation, under government leadership, speech launching the CDF, and documents sub- including civil society, the private sector, sequently issued by the CDF Secretariat (see local governments, and parliaments, with sus- World Bank 2001c). tained public support from top political lead- ership and intellectual conviction by key Long-term, Holistic Development Framework policymakers, and strong links to institutions. 1) Design of 15-to-20 year vision statement con- taining monitorable development goals that: Country-led Partnership 1) Government leadership in management and a. Take account of broad aspirations of the coordination of development partners and population aid resources, including: b. Include sustainable poverty reduction as the overarching goal and related sub- a. Consultative groups, donor roundtables, goals aligned with the Millennium and other coordination mechanisms Development Goals. b. Analytical and diagnostic work 2) Formulation of a coherent medium-term (3- c. Alignment of external support--including to-5 year) strategy for making progress lending, grants, analytical and diagnostic toward vision goals, specifically addressing work, and capacity building--with coun- need for: try's development strategy and donor com- parative advantage a. Balance among macroeconomic and financial issues and structural and social d. Harmonization of development agency concerns procedures and practices b. Priorities in the face of capacity and hard e. Alignment of internal partners' (civil budget constraints; and time-bound, con- society, the private sector, local govern- crete actions, with attention to phasing ments) activities with the country's devel- and sequencing. opment strategy. Country Ownership 2) Relations among government, development 1) Identification of development goals and for- agencies, other stakeholders, marked by: mulation of strategy by the country, not by the donors. a. Mutual trust, consultation, and transparency b. Assumption of mutual accountabilities and work goals, and developing intermediate 81 review of partners' performance indicators toward these goals. c. Demand-led support for strengthening 2) Monitoring and regularly sharing progress, government management and coordina- with accountability for outcomes and goals, tion capacity. not just inputs. Results Orientation 3) Creating and enabling capacities to generate, 1) Designing programs with evaluable objec- monitor, and use results information to tives that contribute to development frame- improve performance in achieving goals and accountability. A N N E X F O U R The CDF and the PRSP: Key Facts 82 Chronology of Events mic change. Includes Population Conference in Cairo (1994); Social Development, Copen- Leading to the CDF hagen (1995); Women, Beijing (1995). and PRSP 1996 DAC members accept U.N. targets for socio- 1980s economic change as IDGs (see Shaping the 21st Century, OECD/DAC 1996). Era of structural adjustment. Aid conditioned on recipient government agreeing to change policies and/or institutions. 1998 New paradigm for development processes emerging at World Bank. Board agrees to ideas 1990s and approaches proposed in Partnerships Paper Widespread concerns about effectiveness and produced by newly formed Partnerships Group. outcomes of aid. Mounting criticism of struc- tural adjustment and lack of development 1998 Annual Meetings progress in Sub-Saharan Africa. World Bank President invites recipient coun- tries to pilot "new development framework" Mid-1990s (subsequently changed to Comprehensive Donors launch programs to increase aid effec- Development Framework, as below). tiveness, including: First HIPC initiative (1996), IDGs (precur- January 1999 sor to MDGs), OECD-DAC Partnership World Bank President proposes Comprehensive Forums, U.N. Development Assistance Development Framework (CDF) to address Framework. shortcomings in aid management to meet the challenge of equitable, sustainable development "Results Management" spreads from corpo- and poverty alleviation. CDF stresses process as rate world to development community well as content, including the idea that the way (IDGs contribute to prominence of results aid is delivered as well as the content of aid funded focus in the CDF). activities determines its effectiveness. 1994-95 March 1999 U.N. sponsors series of international confer- Thirteen countries agree to become CDF pilot ences that produced targets for socioecono- countries, with progress to be monitored and reviewed in September 2000. World Bank idents of multilateral development banks issue 83 forms CDF Secretariat to track progress and dis- joint statement endorsing CDF. seminate good practice experience. In April 1999, the Development Committee supports the CDF approach.58 PRSP: Principles, Elements, and Documentation Sept. 1999 IMF/Bank launch Enhanced HIPC and PRSP. Definition PRSP introduced as a medium-term strategy and Country-produced paper outlining the govern- expenditure program to be developed as a ment's strategy for reducing poverty. Updated requirement for HIPC funds and to be pre- every 2-5 years. The basis for HIPC initiative pared eventually by all IDA countries. The assistance and other concessional assistance from Development Committee decides that PRSPs the World Bank, the IMF, and other donors. should be based on CDF principles. Basic principles 2000 Country-owned process, with broad-based partic- U.N. accepts IDGs, with additions, as Millen- ipation, comprehensive focus, long-term perspec- nium Development Goals (MDGs). tive, and results oriented. Partnerships among government, domestic stakeholders, and donors. MDGs subsequently endorsed by most donors, including World Bank. Key elements Country ownership built through a national par- April 2000 ticipatory process; comprehensive poverty diag- IMF Managing Director and World Bank nosis; priority public actions; targets, indicators, President issue joint statement signaling that and monitoring. PRSPs reflect CDF principles and provide the framework for IMF/Bank collaboration. PRSP documentation (a) Interim-PRSP 2001/2002 Document that presents a road map of progress Recipient countries develop home-grown toward PRSP preparation. Includes a statement visions for improving aid effectiveness. of government commitment to poverty reduc- tion; the main elements of the planned PRS and Newly created New Partnership for African participatory process; a 3-year macroeconomic Development (NEPAD) calls for joint responsi- framework and policy matrix. If eligible, bility, collective action, and peer pressure among endorsement of the I-PRSP triggers HIPC deci- African countries to enhance development sion point and interim HIPC debt relief. effectiveness. By October 2002, 20 countries have produced full PRSPs and a further 29 have (b) PRSP preparation status report produced interim PRSPs (I-PRSPs). Expected one year after endorsement of the I- PRSP, to explain why more than one year is Several bilateral donor policies reflect CDF required to complete the PRSP and to identify principles (DFID, CIDA, and the like) and pres- any necessary technical assistance. 84 (c) PRSP (e) Joint Staff Assessments The full country-produced PRSP covering the Assessments by Fund and Bank staff as to whether four key elements outlined above. Required for the PRSP documentation presented to their HIPC debt relief, the PRGF, CAS, and PRSC. Executive Boards constitutes (or promises to be, or continues to be) a sound basis for concessional (d) Annual PRSP Progress Report assistance. Varies slightly according to what type of Short document outlining progress in imple- documentation is presented, i.e., (a)­(d) above. menting the PRSP, including any achievements, shortfalls, or revisions to the strategy. (f) JSA Guidelines Guidance to Bank and Fund staffs on the kinds In HIPC-eligible cases, the first report triggers of criteria to be used in assessing PRSP docu- HIPC completion point. ments. No requirements as to content. CDF-PRSP-CAS Link Comprehensive Development Framework (CDF) Long-term, Holistic Vision Country Ownership Partnership Development Outcomes Local Knowledge and Analyses Millennium Development Goals U.N. Common Country Assessment World Bank Group Advice and Analysis Other Partners' Advice and Analysis Country Strategy Paper/Poverty Reduction Strategy Paper (PRSP) Business Plan ACTIVITY Structural/ Social/ Physical/ Macroeconomic/ MATRIX Institutional Human Rural/Urban Financial Expenditure Government Framework World Bank Group CAS Private Sector Investments NGOs Plans Regional Development Business Plan Banks United Nations UNDAF IMF PRGF Bilaterals Business Plan EU Country Strategy Mid-term Strategy Implementation/Evaluation/Learning PRSP: Poverty Reduction Strategy Paper CAS: Country Assistance Strategy UNDAF: UN Development Assistance Framework PRGF: Poverty Reduction and Growth Facility Source: CDF Secretariat, World Bank. A N N E X F I V E Evaluation Methodology 85 The Synthesis Report draws from evaluative work The budget for the case studies was predeter- and analysis of CDF implementation under- mined, so the inclusion of more than six cases, taken in the context of three main tasks: while desirable, would have been at the expense I. Case studies of five CDF pilot countries of depth. Other configurations were considered; (Bolivia, Ghana, Romania, Uganda, for example, three CDF pilot and three non- Vietnam) and one non-CDF pilot country CDF pilot, or three PRSP and three non-PRSP (Burkina Faso) countries. However, the first alternative would have reduced the weight given to learning from II. Four thematic studies the CDF pilot experiences, and the second III. A cross-country econometric study. would have shifted the balance toward countries where aid plays a smaller role relative to other Selection of Countries sources of development finance. In the end, for Case Studies given that the primary purpose of the evaluation was to look at implementation issues, the The following six countries were selected for in- Management Group and Steering Committee depth case studies: Bolivia, Ghana, Romania, decided that selecting the longest running and Uganda, Vietnam (all CDF pilot countries) and best performing CDF pilots, together with one Burkina Faso (non-CDF pilot). Selection was determined in the first instance by a country's non-CDF pilot, was the optimal approach for desire to participate in the evaluation and by generating the type and quantity of data the willingness of the World Bank Resident required. (A discussion of sampling strategy Mission to facilitate. Priority was given to choices is found in the GAO guidelines for former CDF pilots with the most implementa- Case Study Evaluations (GAO 1990). The basic tion experience and highest performance rank- argument is that cases should correspond closely ings (according to the CDF Secretariat) on to the focus of the evaluation and the evaluative grounds that these cases would offer the greatest question being asked. In other words, if you are potential for learning. One non-CDF pilot interested in learning from implementation country was chosen as a control. experience, you need to go countries with the longest track record of implementation. Because the PRSP is defined as an instrument for implementing the CDF principles, all countries How Country Studies Were Conducted selected (except Romania) were also PRSP coun- Country studies were carried out in two phases, tries. Consideration was also given to Regional a preparatory mission followed by intensive balance and avoiding overlap with similar evalu- fieldwork. During the preparatory mission the ative efforts (such as the SPA study of PRSP evaluation was planned and designed in consul- implementation in eight African countries). tation with country authorities and donor repre- 86 sentatives in order to reflect individual country Literature reviews. Teams assembled and interests and circumstances. Small focus group analyzed pertinent literature, documents, and meetings were convened to propose and test key data sources, and key country reports were evaluation questions, taking as a point of depar- posted on the CDF Evaluation Web site. ture the questions in the Design Paper (see World Bank 2001c). In each country, priority Questionnaire surveys of key stakeholders. themes as well as sectors were identified (e.g., When possible these were designed in-coun- public sector reform, health, decentralization). try and administered by local institutions. In Interviews were conducted in country capitals Vietnam, a local consulting firm distributed and selected districts, municipalities, and proj- surveys to 290 Vietnamese and expatriate ect sites. The evaluation teams typically development practitioners about how they included a representative from OED, and a com- perceived changes in ODA management bination of national and international consult- with reference to the CDF principles. One- ants. In some cases, such as Vietnam, additional hundred and seven people responded, of consultants and agency staff were provided by whom about 75 percent were Vietnamese interested donors (for example, Japan from government and non-state agencies, MOFA/JBIC, ADB). DFID provided an agency and 25 percent were expatriates from donor staff member for the Uganda team. agencies and international NGOs. In Burkina Faso, a survey was administered to a In sum, the evaluation was carried out in a sample of local government officials, civil "CDF fashion" and involved a wide a range of society representatives, and the private stakeholders engaged in development work sector in four districts of varying socioeco- (from the policy level down to implementation nomic levels, and got a good response rate of of projects). These included representatives roughly 75 percent. In Romania the response from national and local governments, donor rate relative to the sample design exceeded agencies, mass organizations, the private sector, 100 percent for a survey administered to 722 national NGOs, legislatures, academia, civil experts and decisionmakers throughout the society organizations, and international NGOs. country who deal with development issues. In each case study, a small group of advisors was In Bolivia detailed questions were sent to selected from the various stakeholder groups international agencies before the start of the and asked to serve as a country reference group, evaluation. In Uganda, a survey was admin- whose job was to guide the evaluation process istered in a group meeting of stakeholders and provide feedback on design choices and from government, civil society, and the pri- early findings. The evaluation teams were to vate sector. An analysis of the 33 completed have met periodically with the country refer- questionnaires formed the basis for subse- ence groups to share preliminary observations quent workshops with parliamentarians and with donors and national stakeholders in a clos- questionnaire respondents. ing workshop. Structured interviews were conducted with Types of Activities representatives from national and local gov- A variety of activities and evaluation tools were ernments, donor agencies, mass organizations, employed during the course of the evaluation. private sector leaders, national NGOs, legisla- These were: tures, academia, civil society organizations, and international NGOs. Between 70 and Thematic and Cross- 87 84 interviews (group and individual) were conducted per country, resulting in an aver- Country Studies age of roughly 145 people interviewed in There are four thematic studies, one on each of each country. the four CDF principles. The fifth study under- Focus groups were organized based on sec- takes a cross-country econometric analysis on tors, themes (e.g., health, institutional CDF implementation and potential develop- reform), and professional affiliation (e.g., ment impact. A set of log frames and "core eval- ministry staff, private sector, church). uation questions" provided an overall conceptual framework for guiding the analysis of Field trips to selected districts, municipali- the thematic studies. Though the thematic stud- ties, and project sites were made in all ies drew on a broader range of literature and countries and included interviews with worldwide data on CDF principles, they were local government officials and politicians, also developed with an explicit objective of project managers, operational NGOs, ensuring complementarity with the country case donor agencies, and municipal or district studies. The thematic and cross-country studies council members. were undertaken by multidisciplinary teams composed of researchers and evaluators from Closing workshops were held with key developed and developing countries. stakeholders and decisionmakers to discuss the team's preliminary findings. These work- How Thematic Studies Were Conducted shops also served to draw out the policy and The four thematic studies were developed in programmatic implications of the findings. two phases. A survey on changes and trends in govern- In the first phase, the thematic papers were ment-donor relations was carried out in developed as desk studies, with most of the Bolivia, Ghana, Romania, Uganda, and analysis focused on donors and other devel- Vietnam as a specific parallel exercise to track opment agencies, especially in terms of progress in aid delivery and management prac- assessing the extent to which their behav- tices. The survey results identify, and to the ior--policies, procedures, and practices-- extent possible quantify, the impact on govern- has changed sufficiently to enable the ments of donor procedural requirements, and implementation of CDF principles to make the "transaction costs" they imply. A total of 17 an impact at the country level. This ensured bilateral and 9 multilateral donors across 5 CDF that the analysis addressed the issues related case study countries responded to the survey to quality of development aid--not just in that was administered between May and terms of development partnership, but also September 2002. Local consultants adminis- with regard to the potential influences of tered the surveys in Bolivia, Romania, Uganda, the quality of aid on the aid-receiving coun- and Vietnam. In Ghana and Romania, the sur- tries' ability to implement the other three veys were followed up by interviews with CDF principles. donors and key government representatives. Table A5-1 lists the bilateral and multilateral Donor survey questionnaires were con- donors that participated in the survey. structed by three of the four thematic study 88 Table A5-1. Bilateral and Multilateral Donor Respondents in Five CDF Case Study Countries CDF case study countries Bilateral donors Bolivia Ghana Romania Uganda Vietnam Australia - AusAID Belgium Canada - CIDA Denmark - DANIDA Finland France ­ AFD Germany - GTZ * Ireland Japan ­ JBIC & JICA * Netherlands New Zealand Ministry of Foreign Affairs, Norway Spain Sweden - SIDA Switzerland - SDC United Kingdom - DfID United States - USAID (Anonymous) * Multilateral donors Asian Development Bank (ADB) European Bank for Reconstruction and Development (EBRD) European Commission (EC) Inter-American Development Bank (IADB) International Monetary Fund (IMF) The World Bank (WB) United Nations Children Fund (UNICEF) United Nations Development Programme (UNDP) United Nations Food and Agriculture Organization (FAO) Note: * = response of two agencies from one country (for example, from both JBIC & JICA). teams and provided to donor members of the issues related to the construction of proxy indi- 89 evaluation Steering Committee, who cators for the CDF-like principles. requested their respective operational staffs to respond. The results were incorporated in The indicators of CDF-like principles attempt the report of each study. The draft reports for to approximate the range of development the first phase were posted on the CDF processes envisaged under each CDF principle, Evaluation Web site and shared with country using available global databases.59 The key fea- case study teams. tures of these indicators are informally described in Chapter 3. A more formal exposition follows. In the second phase, the focus of the analysis was expanded to examine issues related to country-specific aspects of the CDF, where the CDF-like Long-Term country case study experiences were used to supplement and enrich the first-phase reports. Holistic (LTH) The CDF-like long-term holistic indicator is Methodology of the given by the degree of dispersion in CPIA rat- ings across three broad sectors: Economic Cross-Country Management, Structural Policies, and Policies for Social Inclusion/Equity; and is given by the Econometric Study coefficient of variation (CV): The foci of the cross-country study were twofold: n 2 xi (a) assess the functionality of the ways in which (x i ) LTH = n 1 , i = 1,2,3 the four CDF principles are being implemented; (n ­ 1) xi and (b) assess their potential influence on devel- n opment impacts. In this context, two sets of ana- where Xi is the rating for the CPIA component lytical approaches were pursued. i. The CVk for country K is transformed into a CDF-like long-term holistic index (LTHk), Short-term impact of the CDF. The CDF using the following formula: Secretariat survey data on implementation for 46 CDF and PRSP countries (collected in March LTHk = Max (CV) ­ CVk x 100% and July 2001) were used to assess the short-run Max(CV) ­ Min(CV) impact of the CDF on processes and development outcomes. Unfortunately, the very short span of where max(CV) and min (CV) are taken for time between the two surveys precluded dynamic the entire sample across countries. Note that analysis of the CDF implementation. However, this index falls between 0 and 100. A greater because of the relatively diverse initial conditions degree of dispersion would suggest less coher- of the 46 countries, even the analysis of the short- ence in the country's long-term, holistic devel- term data suggests important policy lessons. opment framework. Indicators of the CDF-like Principles CDF-like country ownership (OWP): The following section provides further informa- According to the Polity IV codebook, the indi- tion on the conceptual, data, and methodology cators are defined in the following way: 60 90 Regulation of participation (RP): Partici- Regulation refers to the extent to which a pation is regulated to the extent that there polity has institutionalized procedures for are binding rules on when, whether, and how transferring executive power (Gurr 1974). A political preferences are expressed. One- four-category measure is used, ranging from party states and Western democracies both 1(closed) to 4(open). regulate participation, but they do so in dif- ferent ways, the former by channeling partic- Constraints on chief executive (CCE): ipation through a single party structure, with This variable refers to the extent of institu- sharp limits on diversity of opinion; the tionalized constraints on the decisionmak- latter by allowing relatively stable and endur- ing powers of chief executives, whether ing groups to compete nonviolently for polit- individuals or collectivities. Such limita- ical influence. The polar opposite is tions may be imposed by any "accountabil- unregulated participation, in which there are ity groups." In Western democracies these no enduring national political organizations are usually legislatures. Other kinds of and no effective regime controls on political accountability groups are the ruling party in activity. In such situations, political competi- a one-party state; councils of nobles or pow- tion is fluid and often characterized by recur- erful advisors in monarchies; the military in ring coercion among shifting coalitions of coup-prone polities; and in many states, a partisan groups (Marshall and Jaggers 2000, strong, independent judiciary. The concern pp. 22-23). A five-category scale is used to is therefore with the checks and balances code this variable, which ranges from 1 among these groups (Gurr 1974). A seven- (unregulated) to 5 (regulated). category scale is used, ranging from 1 (unlimited authority) to 7 (executive parity Competitiveness of executive recruitment or subordination). (CER): Competitiveness refers to the extent that prevailing modes of advancement give The CDF-like country ownership index (OWP) subordinates equal opportunities to become is a simple average: super-ordinates (Gurr 1974, p. 1483). For example, selection of chief executives OWP= 1/4(RP+CER+OER+CCE), where it is through popular elections matching two or scaled to be between 0 and 100, using the fol- more viable parties or candidates is regarded lowing transformation: as competitive (Marshall and Jaggers 2000). OWP ­ min(OWP) A three-category measure is used: 1 (selec- x 100% Max(OWP) ­ Min(OWP) tion) to 3 (election). Openness of executive recruitment CDF-like Country-led Partnership (PA): (OER): Recruitment of the chief executive This is represented by the average of "exces- is "open" to the extent that all the politi- sive" technical assistance and the ratio of con- cally active population has an opportunity, centration of donors in the recipient country in principle, to attain the position through to how aid is distributed across different sectors a regularized process. In considering re- in the economy of the country. A high ratio cruitment, it must first be determined suggests either high donor concentration, and whether there are any established modes at hence less difficulty in achieving coordination all by which chief executives are selected. among donors; low sectoral concentration of aid, which means that aid is more evenly (c) "Excessive" technical assistance: is calcu- 91 distributed across sectors, and hence the aid lated by taking the deviation of the share of regime is likely to promote holistic develop- technical assistance in total aid a country ment; or both. receives (T) from the "optimal" technical assis- tance for that country (T*). That is, excessive (a) Donor Fragmentation (HHID): measured by a technical assistance is given by: Herfindahl-Hirschman Index, which is a simple, yet sophisticated, way of measuring donor frag- T , where T* for country i is derived from mentation/concentration. Squaring the share of T* the following expression: each donor's share as a percentage of bilateral 2 aid and then summing those squares obtains the Iit = a0 + a1Tit-1 + a2Tt-1 x Iit-2 ­ a3Tit-1 x Iit-2 + 0x0 , Herfindahl-Hirschman Index. The Herfindahl- Hirschman Index helps differentiate between where a's are the coefficients,61 T is actual one country in which four donors contribute technical assistance, and I is the a measure of equal amounts of bilateral aid, and another institutional capacity in country i and is given where one donor contributes a 70 percent share, by the ICRG variable taken from the and three others, 10 percent each. The former, International Country Risk data base.62 X is which is more fragmented, would have a lower the initial conditions. The underlying assump- Herfindahl-Hirschman Index. tion in the above specification is that technical assistance improves institutional capacity (the (b) Sectoral Concentration of Aid (HHIs): is the positive coefficient of T), though its effect is Herfendal-Hirchman Index on sectoral allocation subject to diminishing returns (the negative of foreign assistance, which measures the concen- coefficient of T2). Moreover, the specification tration of aid (ODA) across five major sectors: suggests that countries with poor initial insti- social infrastructure and services; economic infra- tutions stand to gain more by receiving techni- structure; production sectors; multi-sector and cal assistance (the negative coefficient of the commodity aid; and general program assistance. Tx I term). Table A5-1. Descriptive Statistics of the CDF-like Variable and Its Components, 1980-2000 CDF-like Mean Median Std deviation Min Max Long-term holistic 63 67 17 16 100 Country ownership 46 43 29 00 100 Country-led partnership 55 55 11 21(16) 86 Results orientation 54 55 16 14 88 Overall CDF-like 55 54 12 24 82 Note: See notes to figure 3.1. The individual countries included in the table are--Sub-Saharan Africa: Angola, Benin, Burkina Faso, Burundi, Cameroon, Cape Verde, Central African Republic, Chad, Comoros, Ethiopia, Gabon, Gambia, Ghana, Guinea, Kenya, Mauritania, Mauritius, Mozambique, Senegal, Seychelles, Swaziland, Tanzania, Togo, Uganda, Zambia, Zimbabwe; Middle East and North Africa: Algeria, Egypt, Jordan, Morocco, Tunisia, Turkey, Yemen; Latin America and Caribbean: Belize, Bolivia, Brazil, Chile, Colombia, Dominican Republic, Ecuador, El Salvador, Grenada, Guatemala, Guyana, Haiti, Honduras, Jamaica, Panama, Paraguay, Peru, St. Lucia, St. Vincent, Trinidad and Tobago, Uruguay, Venezuela; East Asia: China, Fiji, Indonesia, Korea, Rep., Philippines, Thailand, Vietnam; South Asia: Bangladesh, India, Nepal, Pakistan, Sri Lanka. 92 Table A5-2. Distribution of CDF-like Principles Long-term Country Country-led Results Overall Score holistic ownership partnership orientation CDF-like 0-20 2 27 0 3 0 20-40 8 26 12 16 15 40-60 29 21 68 46 56 60-80 56 21 31 45 40 80-100 17 17 1 2 1 Note: The data are averages for the period 1980-2000. The optimum level of technical assistance (T*), an assessment of poverty monitoring and analy- which maximizes institutional capacity (I), is, sis. As previously, the RO is normalized. therefore, given by a1 + a2Iit-2 This index means that one of the four CDF-like Tit = * . 2a3Iit-2 principles that make up the overall CDF-like index is a component of the CPIA. This, how- The optimal level of technical assistance varies ever, should not pose a serious problem for sub- with a country's level of institutions. The regres- sequent regressions where both CDF-like and sion estimate of the above equation provides the CPIA indexes are included, since this sub-index values of the parameters that define the above is only one of 20 components of CPIA; it expression (see Elbadawi, Mavrotas, and Randa accounts only for 9 of the weight (see below the 2003 for further details). expression for the overall CDF-like index). Finally, all regression results remain unchanged Finally, the quality of aid index is given as a when using an overall CDF-like index without simple average of the concentration ratio and including the CPIA component. an "inverse" measure of "excessive" technical assistance: Overall CDF-like PA = (1/2) [ (T*) + ( HHID )] , The overall CDF-like score is a simple average of T HHIs the four principles, subscribing to the central concept of CDF that regards all principles as where it is scaled to be between 0 and 100, using equally important for a CDF approach to devel- the following transformation: opment. Therefore, PA ­ min(PA) x 100% . Max(PA) ­ Min(PA) CDF-like=1/4[LTH+ OWP+PA+RO]. CDF-like Results Orientation (RO) Some Summary Statistics The indicator for results orientation is the rating Summary statistics for the five CDF-like indexes given to question 15 of the Country Policy and for 88 countries spanning the 1980-2000 period Institutional Assessment (CPIA) that asks for are described in table A5-1. Table A5-2 presents the distribution of the CDF- The proxy indicators of CDF-like principles 93 like principles for the 88 countries over the 1980- were derived from 88 countries for the period 2000 period. It is notable that the index of overall 1980-2000. Similar to the numerical CDF CDF-like principles (which will be the only indexes, the indexes of CDF-like principles index subsequently used in the empirical analy- were normalized to fall between 0 and 100, sis) has an approximate bell-shaped distribution. and the overall CDF-like index was obtained by using the statistical technique of "principal Analysis based on CDF-like Principles. components." A detailed discussion of the Although the CDF, as an explicit initiative, was cross-country econometric approach and find- introduced in 1999, its principles were previously ings is found in Elbadawi, Mavrotas, and practiced at various times in different countries. Randa 2002, "Development Impact of CDF- Therefore, an index of CDF-like principles was Like Strategies," a thematic background paper developed and combined with global cross-coun- to this evaluation. The paper is available try data to estimate the marginal contribution of upon request. the implementation of CDF-like development strategies to development outcomes. The index of CDF-like principles was constructed to capture: Business Environment CDF-like long-term holistic principle: meas- Indicators and CDF-like ured by optimum investment gap Strategy Index CDF-like country ownership principle: meas- ured by participation and contestability Patterns of Association components of Polity IV Figures A5-1­A5-4 show CDF-like indices for countries plotted against four indicators of the CDF-like country-led partnership principle: business environment: contract enforcement, measured by quality of aid credit markets, private sector entry regulations, and labor regulations. The discussion of busi- CDF-like results orientation principle: meas- ness environment and the CDF in Chapter 3 ured by the CPIA sub-indicator of poverty- draws on these figures. tracking capacity. 94 Figure A5-1. CDF and Contract Enforcement 2 Bolivia Mozambique 1 Honduras Senegal Tanzania Morocco Romania 0 Georgia Malawi Vietnam Côte d'Ivoire Kenya Dominican Republic Uganda Ghana -1 Zambia -2 1 2 3 4 Overall CDF (CDF Secretariat data) Note: Contract enforcement is measured by an index of the degree of formalism in the procedures to resolve disputes. This index measures substantive and procedural statutory intervention in judicial cases at lower-level civil trial courts. It includes, among other measures: whether the resolution of the case provided would rely mostly on the intervention of professional judges and attorneys, as opposed to the intervention of other types of adjudicators and lay people; the level of legal justification required in the process of dispute resolution, and the formalities required to engage someone in the procedure or to hold him/her accountable for the judgment. The index ranges from 0 (weak contract enforcement) to 7 (superior contract enforcement). 95 Figure A5-2. CDF and Credit Market for the Private Sector Niger 20 Mali Burkina Faso Madagascar Morocco Benin Senegal Côte d'Ivoire 0 Romania Georgia Dominican Republic Bolivia Cameroon Honduras Yemen Vietnam Nicaragua -20 Mozambique 1 2 3 4 Overall CDF (CDF Secretariat data) Note: Credit market: This index is measured by the minimum size of a loan in order for information on the loan to be included in the Public Credit Registries, divided by GNI per capita. The data are objective, so there is no range given. However, higher values indicate that credit histories of large borrowers are the ones that are available, and small borrowers do not have a chance of building their credit history. That is ­ low values (environment good for both small and big business) ­ high values (environment good only for big business). 96 Figure A5-3. CDF and Private Sector Entry Regulations 100 Mozambique Honduras Ghana 50 Azerbaijan Yemen Armenia Côte d'Ivoire Dominican Rep. Albania Bolivia 0 Georgia Madagascar Vietnam Benin Nicaragua Mali Romania Morocco Malawi Senegal CameroonMoldova Zambia Kyrgyz Rep.Burkina Faso Tanzania Uganda Ethiopia Niger -50 1 2 3 4 Overall CDF (CDF Secretariat data) Note:Entry Regulations: This is measured by a time variable that captures the average duration that incorporation lawyers esti- mate is necessary to complete a procedure. It is assumed that the entrepreneur does not waste time; that is to say, he commits to the completion of each remaining procedure from the previous day, unless the law stipulates the contrary. When estimating the time needed for complying with entry regulations, the time that the entrepreneur spends in information gathering is ignored. The entrepreneur is aware of all entry regulations and their sequence from the very beginning. This is objective data-- higher values discourage the entry of new firms into an industry. Lower values indicate easy entry of new firms. 97 Figure A5-4. CDF and Labor Regulation 1 Mozambique Vietnam .5 Nicaragua Honduras Bolivia Senegal Madagascar Georgia Burkina Faso Benin Rep. Azerbaijan Armenia MaliKyrgyz 0 Romania Morocco Ethiopia Dominican Rep. Uganda Albania Malawi Ghana 1.5 Tanzania Kenya Zambia -1 1 2 3 4 Overall CDF (CDF Secretariat data) Note: Labor Regulations: This is measured by an overall index of the extent of labor regulations by taking a simple sum of the employment laws index and the industrial relations laws index. Each index may take values between 0 and 3, with higher values consistent with more regulation. Thus the overall index takes values between 0 (no regulation) and 6 (heavy regulation). A N N E X S I X Evaluation Survey Instruments 98 This annex reproduces the two survey instruments used for the six country case studies and four the- matic studies. A) Government-Donor Relations: Changes and Trends As a specific parallel exercise to the five CDF pilot case study countries (Bolivia, Ghana, Romania, Uganda, Vietnam), a questionnaire survey instrument was developed to capture facts and figures on the current status and trends in relations between government and development partners. Specifically, the survey attempts to identify the impact on governments of donor procedural requirements, and the transaction costs they imply. A. Composition of portfolio Please tell us about the activities that your agency is responsible for in Ghana. a1: Approximately how many projects, including "non-project" or programmatic assistance opera- tions, does your agency currently have in your approved portfolio for Ghana? Please check the applicable range. < 5 5-10 10-20 20-40 >40 If available, what is the exact number? ______ a2: Has the average size of such operations decreased or increased over the last 5 years? Please check the applicable range. Increased Remained the same Decreased a3: How many, if any, of your agency's project and programme operations in Ghana are part of wider funding schemes where other donors are also contributing (in joint sector-wide pro- grammes [SWAps], as cofinancing, or in other modalities)? Please check the applicable range. < 3 3-5 5-10 10-20 >20 If available, what is the exact number?_______ a4: What approximate share of your agency's total annual disbursements over your last two fiscal years is accounted for by jointly funded project and programme operations in Ghana? Please check applicable range. <20% 20-40% 40-60% 60-80% >80% a5. Has this share increased over the last 5 years? Yes No 99 a6. Please add possible other comments regarding trends of change in the portfolio of your agency in Ghana. B. Coordination with other donors Please give us your (and your colleagues') assessments of the efficiency and effectiveness of donor coordination activities. b1: In your perception, has there been a change in the frequency of donor coordination activities in which your agency has been involved during the last five years? Please check the applicable range and elaborate. Increased Remained the same Decreased b2: In which coordination activities of a recurring nature that deal with donor assistance as a whole does your agency participate? Consultative Group (CG) Meetings Mini-CG Meetings Other (please specify nature of meeting, frequency, and whether Government is invited) b3: In which coordination activities dealing with specific sector or project issues and involving other donors does your agency participate? In each category below, please specify sector, fre- quency of meetings, and whether Government is invited. (1) Sector Issues (2) Project Issues (3) Other b4: In your perception (and those of your colleagues), has (1) the efficiency of donor coordination improved, deteriorated or remained the same over the last five years? Deteriorated Remained the same Improved (2) Is donor coordination efficiency currently satisfactory in your view? Yes No (3) Have the results (effectiveness) of donor coordination improved, deteriorated or remained the same over the last five years? Deteriorated Remained the same Improved (4) Are they currently satisfactory in your view? Yes No (5) What specific means could improve efficiency and effectiveness? Please specify. b5. (1) In your view, are there instances where country-led aid coordination is or was particularly 100 efficient and/or effective? Please specify. (2) In your view, are there instances where country-led aid coordination is or was particularly inefficient and/or ineffective? Please specify. (3) In your view, have there been instances where donor-led aid coordination is or was more effi- cient and/or effective than country-led aid coordination? Please specify. b6: What in your view have been the key achievements of coordination activities, if any, over the last 5 years? Please check all that apply and provide examples wherever possible. Increased selectivity among donors so as to avoid overlap Better alignment of approaches and with Ghanaian strategy/GPRS More efficient policy dialogue Adoption of common procedures, such as joint missions, reports, procurement, or budget and disbursement cycles? Others (please specify) b7. Please add possible other comments regarding trends in aid coordination in Ghana. C. Donor administrative and procedural requirements Please tell us about your agency's administrative and procedural requirements. c1. (1) How many preparation/appraisal missions (or visits) have staff and/or consultants from your agency's headquarters undertaken to Ghana in the last 12 months? < 3 3-5 5-10 >10 If available, what is the exact number?_______ (2) How many of these have been undertaken jointly with other donors? < 3 3-5 5-10 >10 If available, what is the exact number?_______ (3) Over the last 5 years, has the number of joint preparation/appraisal missions Increased Remained the same Decreased c2. (1) How many monitoring and evaluation missions have staff and/or consultants from your agency's headquarters undertaken to Ghana in the last 12 months? < 3 3-5 5-10 >10 If available, what is the exact number?_______ (2) How many of these have been undertaken jointly with other donors? < 3 3-5 5-10 >10 If available, what is the exact number?_______ (3) Over the last 5 years, has the number of joint monitoring and evaluation missions Increased Remained the same Decreased c3. How many reports (progress and financial) does your agency normally require from 101 Government per year for: (1) projects? 1 2 4 Other:____ (2) non-project assistance or budget support? 1 2 4 Other:____ (3) Over the last five years, has the number of reports required: Increased Remained the same Decreased c4. (1) Has your agency ever adopted a joint reporting format endorsed or prepared by the government? Yes No If yes, in how many cases? < 3 3-5 5-10 >10 (2) What has been the trend, if any, over the last 5 years, in the use of joint formats? Increased Remained the same Decreased c5. For Ghana projects and programmes (non-project operations), has your agency adopted: (1) the procurement rules of other agencies? Yes No If yes, in how many cases? < 3 3-5 5-10 >10 (2) government procurement procedures? Yes No If yes, in how many cases? < 3 3-5 5-10 >10 (3) What have been the trends, if any, in each category over the last 5 years? Procurement rules of other agencies: Increased Remained the same Decreased Government procurement rules: Increased Remained the same Decreased c6. In your view, how has the move from project aid to various types of programme aid affected: (1) the amount of negotiation and meeting time before and during preparation? Increased Remained the same Decreased (2) the amount of paperwork? Increased Remained the same Decreased (3) Please give examples if possible: c7: Please add possible other comments regarding trends in and impact of donor procedural require- ments in Ghana. 102 D. Implementation practices The questions in the next two sections deal with donor practices. d1. Does your agency offer salary incentives (topping up, sitting allowances, consultancy contracts to civil servants, per diem etc.) to national staff (regular staff and local consultants) involved in implementation of project or programmes your agency funds in Ghana? Yes No If yes, (1) In how many of the projects and programmes funded by your agency are such incen- tives offered? <20% 20-40% 40-60% 60-80% >80% (2) Why does your agency find it necessary to provide these additional incentives? Please specify: (3) In the last 5 years, have the amount of such incentives Increased Remained the same Decreased d.2 Does your agency use project or programme implementation or management units in Ghana? Yes No If yes, (1) How many are currently active? < 3 3-5 5-10 >10 (2) Please explain the main reasons for using such units. (3) Does you agency plan to continue using such units as they exist today in the foreseeable future? Yes No (4) What are your plans for their future use? d3. To what extent does "disbursement pressure"--the explicit or implicit requirement in your agency's policy that assigned funds must be spent in the budget year or over a given period­ influence your agency's choice of the above or other implementation practices? Please give spe- cific examples. d4. Does your agency use special measures, such as fiduciary, environmental and social safeguards, on top of the government's own management & control system? Yes No If yes, (1) Please specify which measures you use. (2) Why does your agency find such measures necessary? d5. (1) How much of your agency's assistance to Ghana is tied to procurement of goods and serv- 103 ices, including TA, in your own country? <20% 20-40% 40-60% 60-80% >80% (2) In the last 5 years, has this share Increased Remained the same Decreased (3) Does your agency in Ghana have any discretion over source of procurement, or is this deter- mined by your headquarters? d6. Which approximate share of your agency's annual disbursement in Ghana is used on: (1) internationally recruited TA? <20% 20-40% 40-60% 60-80% >80% (2) locally recruited TA? <20% 20-40% 40-60% 60-80% >80% (3) What have been the trends over the last 5 years? Total amount of TA: Increased Remained the same Decreased Internationally recruited TA: Increased Remained the same Decreased Locally recruited TA: Increased Remained the same Decreased d7. (1) How often does your agency report on planned and actual expenditures to the Government of Ghana? Annually Bi-annually Quarterly Other:____________ (2) Has the frequency of reporting over the last 5 years Increased Remained the same Decreased d8. Do you (and your colleagues) consider the information that your agency provides to the Ghanaian government timely and sufficient, and in principle enabling the government to include your agency's assistance in the national budget and accounts? Yes No Please explain. d9. Please add possible other comments regarding implementation practices. 104 B) Government-Donor Relations: Changes and Trends As a specific parallel exercise to the four thematic studies (Long-term Holistic Framework, Country Ownership, Country-led Partnership, and Results Orientation), a structured survey questionnaire instrument was developed to assess the extent of implementation of the four CDF principles by development partners. The survey was administered to bilateral and multilateral donors both at headquarters and in country offices. Long-term Holistic Frameworks 1) Our organization has endorsed the principle of Long-term Holistic Framework as essential to development cooperation: -2 -1 0 +1 +2 ? 2) Our government has decided that bilateral aid shall preferably be based on longer-term financial frameworks. -2 -1 0 +1 +2 ? 3) Parliamentary debates/decisions on development co-operation support linking resources to longer- term framework agreements with partner countries. -2 -1 0 +1 +2 ? 4) Our direct bilateral agreements with partner countries are increasingly based on longer-term framework arrangements (strategies, memoranda of understanding). -2 -1 0 +1 +2 ? 5) The average time horizon for our country's bilateral agreements (project, programs, country strategies) is longer now than they were three years ago. -2 -1 0 +1 +2 ? 6) Our organisation/government is now allocating a higher share of bilateral aid to longer-term framework agreements (macro/budget or sector support, SWAPs). -2 -1 0 +1 +2 ? If the answer was a "+1" or "+2," please explain/specify: 7) Our partner countries see us as being a more predictable donor in terms of structure and com- mitment of financial flows. -2 -1 0 +1 +2 ? 8) We are systematically building our capacity for our own long-term holistic planning. -2 -1 0 +1 +2 ? 9) We have developed new instruments that strengthen our ability to carry out long-term holistic planning. -2 -1 0 +1 +2 ? If the answer was a "+1" or "+2," please explain/specify: 10)Our staff training is now focusing more on program approaches and less on project manage- 105 ment. -2 -1 0 +1 +2 ? 11) We are increasing our capacity to discuss, analyse and contribute to partners' longer-term plan- ning instruments and processes (PERs, SWAPs, etc.) (putting more staff in the field, hiring more strategy/macro and fewer project staff, etc) -2 -1 0 +1 +2 ? If the answer was a "+1" or "+2," please explain/specify: 12)Our organisation considers LTHF as important a concept in our dialogue with our partners as ownership and country-led partnership. -2 -1 0 +1 +2 ? 13)LTHF is discussed as a key concept in international donor co-ordination meetings (DAC meet- ings, SPA meetings). -2 -1 0 +1 +2 ? 14)LTHF is discussed as an important concept in in-country donor meetings. -2 -1 0 +1 +2 ? 15)LTHF is increasingly discussed as an important concept with partner country governments. -2 -1 0 +1 +2 ? B. Donors' Views on LTHF in Partner Countries 16)Partner countries' understanding of LTHF is clear. -2 -1 0 +1 +2 ? 17)Partner countries' embrace of LTHF is more rhetoric than reality. -2 -1 0 +1 +2 ? 18)We see that there is an increasing interest and political will in the partner countries to move toward LTHF approaches -2 -1 0 +1 +2 ? 19)If LTHF has not been adopted in some countries, it is due to lack of political will -2 -1 0 +1 +2 ? 20)If LTHF has not been adopted in some countries, it is due to lack of internal planning capacity -2 -1 0 +1 +2 ? 21)If donors had wanted partner countries to adopt LTHF faster/more consistently, they could have provided more political and capacity building support -2 -1 0 +1 +2 ? 22) The donor community has been consistent and constructive in its dialogue with partner coun- tries on the need to adopt and implement LTHF approaches -2 -1 0 +1 +2 ? 106 23) Some partner countries see LTHF as another donor-imposed concept that they are highly scep- tical to. -2 -1 0 +1 +2 ? 24) Some partner countries see the LTHF as too sophisticated and demanding for their own polit- ical and administrative structures to be able to cope with -2 -1 0 +1 +2 ? 25) In partner countries that have in principle adopted an LTHF approach, this is reflected in the following changes over the last three years: They are now producing better quality long-term vision documents. -2 -1 0 +1 +2 ? There are better economic/financial links and consistency between long-term vision documents, medium term frames (MTEFs, PRSPs) and annual budgets and plans -2 -1 0 +1 +2 ? There is better balance between sectors -2 -1 0 +1 +2 ? There is better balance between economic/financial and human/structural/social dimensions -2 -1 0 +1 +2 ? Programming/planning within sectors is now better (SWAPs, sector programs) -2 -1 0 +1 +2 ? 26) We see that countries that are CDF pilots have been better at moving toward LTHF than non- CDF partner countries. -2 -1 0 +1 +2 ? Comments: C. Donors' Support for LTHF in Partner Countries 27) We support partner countries that ask assistance for preparing larger frameworks for our assis- tance (sector programs, macro planning). -2 -1 0 +1 +2 ? 28) We now provide more assistance to activities that contribute to longer-term and more holis- tic planning (planning entities/activities in ministries of planning, line ministries where we provide assistance). -2 -1 0 +1 +2 ? 29) We use own staff time to support the development of LTHF-like processes in the field (SWAPs, PRSPs). -2 -1 0 +1 +2 ? 30) Our organisation now demands that partner countries provide larger frameworks as part of the request for assistance. -2 -1 0 +1 +2 ? Comments: D. LTHF Instruments 107 31) The basic LTHF instrumentarium in a country should be the country's own budget and plan- ning instruments. -2 -1 0 +1 +2 ? 32) A matrix was proposed as a basis for a holistic framework in Mr. Wolfensohn's January 1999 launching of the CDF. We see this matrix as a useful tool for holistic planning. -2 -1 0 +1 +2 ? We see this matrix as useful for analysing where donors should allocate their resources -2 -1 0 +1 +2 ? We see the matrix more as a heuristic/pedagogical way of talking about holistic planning rather than as a concrete planning tool/instrument. -2 -1 0 +1 +2 ? We use this matrix approach to verify if a partner's planning can be considered holistic. -2 -1 0 +1 +2 ? We have never seen the matrix used by any partner country's authorities. -2 -1 0 +1 +2 ? 33) Later in 1999, the Poverty Reduction Strategy Paper (PRSP) was launched: We see PRSPs as the best current instrument for LTHF. -2 -1 0 +1 +2 ? We see current PRSPs as a step in the right direction toward LTHF. -2 -1 0 +1 +2 ? We see the PRSPs as being too limited in terms of sectors/issues (not holistic enough). -2 -1 0 +1 +2 ? We see the PRSPs as being too short-term (3 year delivery horizon - not long-term enough). -2 -1 0 +1 +2 ? Comments: 34) The Millennium Development Goals (MDGs) are now the basic set of goals that development assistance/ efforts are to work toward. We see the MDGs as sufficiently long-term but not holistic enough. -2 -1 0 +1 +2 ? We see the MDGs as encompassing the key areas and thus for most countries being sufficiently holistic. -2 -1 0 +1 +2 ? 108 35) Please rank the following instruments for LTHF in order of importance, "1" most important, "5" least: ______ The CDF Matrix ______ PRSPs ______ MDGs ______ Sector plans (SWAPs etc) ______ National planning/budget documents that require Parliamentary approval Comments: E. Factors that Facilitated the Move toward LTHF 36) The trend seems to be toward LTHF in partner countries. Please rank the following factors in order of importance, "1" most important, "5" least: ______ Internal developments in partner countries ______ The launching of OECD's "Shaping the 21st Century" ______ The launching of the CDF ______ The launching of the PRSP ______The general international trend as reflected in the International Conferences with focus on overarching/long-term holistic goals 37) The CDF is seen as contributing toward LTHF. Please rank the following factors in order of importance, "1" most important, "5" least: ______ The CDF presented an original articulation of LTHF ______ The CDF was more focused, operational on the need for LTHF ______ The CDF was launched by the Bank, Mr. Wolfensohn ______ The Bank has put considerable resources behind the CDF ______ The CDF reinforced/strengthened widely accepted but not well implemented principles F. LTHF and Impact on Results 38) We see that partner countries that use LTHF are more efficient users of own limited planning and co-ordination resources. -2 -1 0 +1 +2 ? 39) We see that governments that use LTHF are more active/more efficient at donor co-ordina- tion. -2 -1 0 +1 +2 ? 40) One of the gains from using LTHF in the donor-partner dialogue is that it is supposed to reduce various transaction costs. We see that using LTHF reduces internal planning/co-ordination costs in our partner countries -2 -1 0 +1 +2 ? We see that using LTHF reduces aid co-ordination costs between donors and host authorities -2 -1 0 +1 +2 ? We see that using LTHF generates gains primarily to the host government. 109 -2 -1 0 +1 +2 ? We see that using LTHF generates gains primarily to the donors. -2 -1 0 +1 +2 ? Comments: 41) We see that countries that are moving toward LTHF are better able to align their own objectives and resources toward attaining the MDGs. -2 -1 0 +1 +2 ? If the answer was a "+1" or "+2," please explain/specify: 42) Indications are that those countries that are using more long-term frameworks are more suc- cessful at attaining the MDGs.63 -2 -1 0 +1 +2 ? Comments: G. Bottom Line/Summing Up 43) We consider the LTHF as a key concept for more efficient and effective use of development resources. -2 -1 0 +1 +2 ? 44) It is our impression that the donor community consider LTHF as a key concept for more effi- cient and effective use of development resources. -2 -1 0 +1 +2 ? 45) It is our impression that our partner countries consider LTHF as a key concept for more efficient and effective use of development resources. -2 -1 0 +1 +2 ? 46) It is our impression that use of LTHF approaches is increasing in importance and quality, and that this trend is likely to continue in the years to come. -2 -1 0 +1 +2 ? Thank you very much for your time!! 110 Country Ownership Please check the appropriate response Part 0 -- General 0.1. Do you feel that the principle of ownership within the CDF has had a significant impact in terms of influencing the design and implementation of your assistance programs? Yes No 0.2. Has your organization aligned its operational policies, procedures and practices to support country ownership? Yes No 0.3. If yes, please give the date: 0.4. How have these policies evolved since then? (Please elaborate). 0.5. Has the implementation of the CDF principle of greater ownership meant an increased con- cern for any of the following? Participation Women Humanrights Capacitybuilding Other(pleasespecify) 0.6. Has your organization, in support of the ownership principle, increased its support in these areas? Yes No 0.7 Please elaborate Part I -- Allocation and Prioritization of Aid Flows Until the mid 1990's,it would seem that most aid recipients did not have a long-term vision to help aid donors align aid allocation with country priorities or such a vision was not home grown. I.1 Before 1995, did most recipient governments you worked with have a long-term national vision? Yes No I.2 When did they start adopting such a vision? Date: I.3 Has your organization contributed to the formulation of any country's development strategy? Yes No I.4 If yes, in which sectors have you made a contribution: Poverty Reduction Civil Service Reform Governance Education Health Overall design of vision Other (please specify): I.5 In the formulation of the country's long and medium term development program, do you accept local priorities in a systematic way? Yes No I.6 Do governments publicly disseminate their long-term development goals? Yes No 111 Donors may feel there is a strong need to support institution building while governments prefer assistance in stone and mortar sectors: I.7 Rank the following sectors according to your priorities for donor assistance: ______ Sectoral (agriculture, finance, manufacturing....). ______ Infrastructure (electricity, water, telecommunications...) ______ Human Development (health, education, poverty alleviation, SMEs, human rights, good governance...) ______ Judiciary system (training of judges, reforming of dispute settlement mechanism...) ______ Legislative reform (competition law, labor law...) ______ Civil Service reform (transparency, accountability) I.8 Do you usually impose certain conditions on the borrowing/ recipient country in order to allo- cate your funds? Yes No I.9 If yes, what kind of thematic concerns of the project do you insist upon? Gender equality Capacity Building Social dimensions Governance Aid Performance Monitoring I.10 Has greater ownership of the country's development program resulted in a changed approach to performance monitoring and evaluation of the aid being disbursed by your organization? Yes No I.11 If yes, in what way? I.12 Is the information being reported, of the kind and form, that is also useful to the recipient country (e.g. budget preparation and planning) or does it serve your organization's purposes only? Yes No Part II -- Technical Assistance (TA) for Capacity and Institutional Building TA can be a double-edged sword, since increased allocation toward TA will not necessarily improve local participation and ownership and may be dissipated back to donors. II.1 Do you feel that the contracting mode of receiving TA affects the ownership outcome of the project? Yes No II.2 Do you as a donor require the involvement of TA suppliers from designated donor sources? Yes No II.3 Are foreign experts required to provide training for local consultants? Yes No 112 II.4 Do you agree that over the last 5 years, the role share of local advisors and policymakers has increased relatively to the role of external advisors/assistance in formulating both the long and medium term plans/strategies? I agree I do not agree II.5 Is training by foreign consultants of the local counterparts used as a measure of success of aid projects? Yes No II.6 Is the participation of local intermediaries in the management/execution of aid projects used as a measure of success? Yes No II.7 What other measures/indicators are used to determine the success of aid projects/programs? (List the most important five indicators) II.8 As a proportion of total cost of TA projects, what is roughly spent on salaries and fees for for- eign consultants? 25% or less 40% 50% 60% or more Other (specify) II.9 Has this percentage decreased over the last five years? Yes No II.10 How high is the level of salaries and fees of foreign consultants as a multiple of local fees? Twiceashigh Threetimes Fourtimes Morethanfourtimes Other(specify) II.11 How high are salaries for local persons working in donor-assisted projects as a multiple of public-sector salaries? 20%higher 40%higher 60%higher Morethantwiceashigh Other(specify) II.12 Does your experience confirm that the civil service is likely to witness any employment loss of best brains in favor of donor-assisted programs? Yes No II.13 In your view, do you think that technical assistance has strengthened capacity building over the last five years? Yes No Country-led Partnership 0. General 0.1 Has your organization accepted the CDF principle of partnership in aid relationships which is led by the recipient country? Yes No 0.2 Aside from the broad acceptance of the `principle' to `listen more to the recipients,' has your organization taken definitive steps in terms of changing the legal framework governing aid relationships? Yes No 0.3 When was the last change/ amendment to your institution's legal basis for or rules governing 113 aid relationships? State Year and month of change 0.4 Please cite the relevant clause(s) that make the critical difference between the `old' and the `new' relationships: 0.5 Beside any changes in the laws governing aid, are there fundamental changes in the adminis- trative rules and procedures governing development assistance in your organization that require recipient-driven ownership and partnership? Yes No Comments: 0.6 Has there been any internal restructuring of your organization (downsizing or creation of new departments or institutional strengthening) to respond to the `new' aid relationships? Yes No Comments: 0.7 Are there examples of projects or programs which are currently being funded in a manner that reflects country ownership and recipient-driven partnership? Yes No Comments: 1. Aid Coordination 1.1 Has aid coordination improved in the CDF countries after the introduction of the CDF/CDF- like principles? Yes No Comments: 1.2 Has the division of labor increased among donors in the CDF countries? Yes No Comments: 1.3 Are there effective government-led mechanisms for aid coordination in the CDF countries? Yes No Comments: 1.4 Has your agency/organization over the last two years actively supported partner countries in strengthening their capacity to lead aid coordination? Yes No Comments: 114 1.5 What are the reasons for lacking government-led aid coordination? (Please rank the reasons below 1-5, where 5 gives the highest weight) Constraints Rank Lack of local institutional capacity 1 2 3 4 5 Government unwillingness to lead 1 2 3 4 5 Constraints on donors by their respective 1 2 3 4 5 laws and procurement procedures Donor preferences differ 1 2 3 4 5 Poor mechanism for donor 1 2 3 4 5 coordination in the country Comments: 1.6 Could closer alignment of donor and governmental priorities come at the expense of civil soci- ety-government relations in recipient countries? Yes No Comments: 2. Donor Harmonization 2.1 Are recipient countries playing any significant role in the harmonization efforts? Yes No Comments: 2.2 Is there any difference in harmonizing procedures with CDF countries compared to non-CDF countries? Yes No Comments: 2.3 Does donor headquarters policy hinder the development of country specific harmonization? Yes No Comments: 2.4 Do you undertake any joint monitoring and evaluation (with recipient governments) of devel- opment programs since the CDF was launched? Yes No Comments: 3. Nature of Aid - Aid Modalities 3.1 Has there been any movement in your aid program from project support to program and budget support over the last two years? Yes No Comments: 3.2 Do you use program aid more frequently in CDF countries compared to non-CDF countries? 115 Yes No Comments: 3.3 Do you foresee an increased delegation of authority and decentralization of staff to in-country offices in countries where you provide program aid ? Yes No Comments: 3.4 Technical assistance often appears to be supply-driven, expensive and excessive, hampering true ownership and the use of local capacities. Have you changed the direction of your tech- nical assistance program over the last two years? Yes No Comments: 3.5 Do you support Project Implementation Units in the conduct of aid programs in the CDF countries? Yes No Comments: 3.6 Have you replaced any own reporting requirements with regular government reports in any CDF country over the last two years? Yes No Comments: 4. Policy Shifts among Donors/IFIs/Multilateral Organizations 4.1 Have you aligned your country assistance programs with the CDF countries' development strategies (PRSP)? Yes No Comments: 4.2 Do you consult with the national Parliament in the recipient country when preparing your country assistance strategy? Yes No Comments: 4.3 Has any steps been taken in your country to improve coherence among different policies affecting low-income countries, based on the poverty reduction objective? Yes No Comments: 4.4 Will the implementation of Country-led Partnership principle result in lower transaction costs of aid programs? Yes No Comments: 116 5. Partnership 5.1 What are the main obstacles to implement a country-led Partnership? Comments: 5.2 How can donors best promote country-led Partnership? Comments: 5.3 Given a willingness to listen more to the recipient country (being in the driver's seat) is part- nership development contradictory to genuine ownership? Yes No Comments: 5.4 Has the cooperation among different actors in your country (government, civil society organ- izations, private sector) increased with regard to your development assistance program as a result of the introduction of the CDF principles? Yes No Comments: 5.5 Has your cooperation with different non-governmental actors (civil society, private sector) in the recipient country increased since the introduction of the CDF principles Yes No Comments: 5.6 Is the implementation of the Partnership principle viewed by stakeholders in the recipient country as imposed by donors? Yes No Comments: 5.7 Has the quality of partnership been negatively influenced by the extent of indebtedness and by aid dependency in the recipient countries? Yes No Comments: Results Orientation 1. Has your organization done any of the following things to encourage aid recipients to adopt a "results orientation"? Please check all that apply. Required aid recipients to identify and track performance indicators; Called for plans that include performance indicators; Provided technical assistance to help a country and/or its local governments to develop a performance indicators process; Provided funds or other resources to the aid recipients for training in performance measurement and/or performance management; Other. (Please describe.) Have not attempted to encourage a results orientation. 2. What "results orientation" results do you believe have occurred? Please be as specific as possi- 117 ble. Please describe both results within specific projects and those in country development more generally. 3. Do you believe any problems have arisen because of different messages provided to aid recip- ients by donors as to what should be done to achieve a "results orientation"? Yes No If yes, please describe those problems. 4. Do you believe that any of the following problems have occurred in your organization's attempt to deliver "results orientation" messages to aid recipients? Please check all that apply. Lack of clear definitions of the key "results orientation" terms. Allocating too few resources to advance "results orientation." Not providing consistent aid for a sufficient amount of time. Not giving assisted countries enough time to implement the principles adequately; expecting too much too quickly. Conflicting objectives of donors Lack of coordination among donors Lack of interest/support by the assisted countries. Lack of assisted-country expertise to adequately implement the needed activities. Units within your organization not providing consistent messages. Lack of understanding within your own organization as to what "results orientation" is and how it should be promoted. Not adequately evaluating the progress your organization is achieving in a "results orientation" by countries you are assisting. Other. (Please describe.) 5. To what extent do you believe that the Millennium Development Goals/International Development Goals (MDG/IDG) will help assisted countries to improve development effectiveness: To a considerable extent To some extent Little, if any, effect Will have a negative effect Am not familiar with the MDG/IDG A N N E X S E V E N Multi-Stakeholder CDF Evaluation 118 Process: Analysis of Survey Questionnaire Results Introduction influence on the design and implementation of the evaluation. Fifty percent of respondents (5 The following analysis synthesizes findings on out of 10) felt they would have liked more the benefits and costs of the multi-stakeholder opportunity to influence the design of the eval- evaluation process to the members of the CDF uation, while 60 percent (6 out of 10) felt they Evaluation Steering Committee based on survey would have liked more opportunity to influence results from 10 representatives of 9 agencies. the implementation of the evaluation. One The questionnaire survey instrument is attached respondent expressed the view that although at the end of this analysis. many of the Steering Committee members' comments, presented at various meetings, were The response constitutes about 35 percent of the taken into account in the Design Paper, some of total number of entities (26) represented on the the methodological problems identified at ear- Steering Committee (SC). Established in January lier stages still remained. Furthermore, the same 2001, the SC included about 30 individual repre- respondent felt that once the evaluation started, sentatives from recipient countries, bilateral and few possibilities were given to Steering multilateral donors, a nongovernmental organi- Committee members to affect the implementa- zation (NGO), and two private sector firms. tion of the evaluation. The Steering Committee meeting held in Kampala, Uganda, in January SC member participation in the CDF evalua- 2002 was the only meeting organized during the tion has taken several forms: implementation process, during which very little time was devoted to discussing the Uganda Membership on the Steering Committee report, precluding the possibility of affecting the Financial support implementation of subsequent country studies. Commenting on drafts by e-mail Only two respondents felt they had considerable Formationofand/orparticipationincasestudy opportunity to influence the design of the eval- country "reference" groups and workshops, or uation. But respondents felt they had relatively assisting case study implementation in other more opportunity at the implementation stage ways (e.g., arranging meetings and interviews). compared with the design stage; three respon- dents asserted many opportunities for influenc- ing the evaluation's implementation. Design and Implementation of the CDF Evaluation Involvement in the CDF Evaluation Survey respondents had mixed views on the Since the launch of the CDF evaluation, there extent to which their organizations had an have been three official Steering Committee meetings, held in Copenhagen (Denmark), diversity and quality of the evidence for the eval- 119 Kampala (Uganda), and Santa Cruz (Bolivia), uation. Three respondents, however, had mixed in January 2001, 2002, and 2003, respectively. views on the same dimension, with two respon- The majority of respondents, 60 percent (6 out dents finding the approach to be largely positive, of 10), found their participation in Steering while one found it to be largely negative. Committee meetings to have been very useful, while the remaining respondents, 40 percent (4 Respondents were almost equally divided on the out of 10), found it somewhat useful. impact of the multi-stakeholder approach on the quality of the analysis, with 40 percent (4 CDF evaluation documents and drafts (country out of 10) finding the approach to have had a case studies, thematic studies, Synthesis Report) positive impact on the quality of the analysis, have been regularly posted on the evaluation while another 50 percent (5 out of 10) felt it was Web site, and 30 percent of respondents (3 out mixed but largely positive. Similarly, 40 percent of 10) found their involvement in reviewing (4 out of 10) of the respondents felt the these materials to be very useful, while 40 per- approach had a positive impact on the credibil- cent (4 out of 10) found it to be somewhat ity of the findings, while another 40 percent useful. Two respondents of the Steering held mixed views, with three respondents find- Committee felt that it was more trouble than it ing the approach to have had a largely positive was worth. One respondent found the Web site impact, while one found it to have a largely neg- to be a poor medium for triggering broader and ative impact. better debate among Steering Committee mem- bers. Another felt that draft reports were not The majority of respondents, 80 percent (8 out always available in time to give comments, lim- of 10), had mixed views on both the timeliness iting the Web site's usefulness as a channel for and the administrative costs of the evaluation. active involvement in the evaluation. Fifty percent (5 out of 10) of respondents felt that the approach had a mixed but largely posi- Only one respondent participated in a CDF tive impact on the timeliness of the evaluation, evaluation mission and found it to be very while 20 percent (2 out of 10) felt it had a neg- useful. Six respondents participated in country ative impact. Only one respondent was of the workshops, with two of the respondents finding opinion that the approach had a positive impact the workshops very useful, while three found on the timeliness of the evaluation. Similarly, them to be only somewhat useful. One respon- 50 percent (5 out of 10) of respondents felt that dent found the workshops not at all useful. the approach had a mixed but largely positive impact on the administrative costs of the evalu- Four of the respondents had provided financial ation, while 20 percent (2 out of 10) felt it had support to the CDF evaluation, and one found that a largely negative impact. Only one respondent mode of involvement very useful, while the other was of the opinion that the approach had a pos- three found it to be only somewhat useful. itive impact on the administrative costs. Impact of a Multi-Stakeholder Major Advantages of a Multi- Approach on the CDF Evaluation Stakeholder Approach Half of the respondents felt that the multi-stake- Most respondents who answered this question holder approach had a positive impact on the (no. 4 in the attached questionnaire) see the 120 diversity of the stakeholders that make up the Complexity of management process Steering Committee as a major advantage, Poor representation of domestic stakeholders which assured that the Approach Paper for the (i.e., government, civil society, business evaluation would reflect a multi-perspective sector) in the evaluation governance structure approach rather than a single-perspective approach. The diversity is also given credit for Inadequate triangulation and active search creating opportunities for consultations and for alternative views. comments and for stimulating debate, thereby increasing the quality of the analysis. Other Ways of Achieving the Advantages stated advantages include: at Lower Cost Diversity and credibility of evidence One respondent felt that the thematic studies should have been given less prominence, won- Relevance of issues to numerous develop- dering to what extent they really added value to ment agencies the fieldwork and to the combined knowledge of Impact of evaluation findings if acted upon the teams that undertook the evaluation. by development agencies Additional suggestions for lowering the cost of the evaluation include: Broadening of consensus Access to sources of information that no Fewer formal meetings and more virtual meetings and/or video conferencing single agency could have Mix of quantitative and qualitative assess- More participation by local evaluators ments More delegation to consultants. Transparency of the process Appropriateness of Role in the placement of a participatory Methodological Tools monitoring system The few respondents who expressed their views Help with the planning of new initiatives in on this issue found the methodological tools the future. employed, particularly for evaluating country case studies, to be adequate. However, one Major Disadvantages of a respondent expressed that there was a major dis- Multi-Stakeholder Approach connect between the log frame methodology outlined in the Design Paper and the methodol- Most respondents who answered this question ogy utilized in the country case studies. (no. 5) felt that as the evaluation got under way, the participation of the different stakeholders Respondents, however, found the methodology was not mainstreamed, thereby garnering lim- used for the cross-country econometric analysis ited participation from members. Other stated to be inappropriate, specifically the proxy indi- disadvantages include: cators chosen for the analysis. High transaction costs One respondent was of the opinion that the case Time consuming studies--some more than others--were thick Lack of focus and simplicity with the opinion of the authors. Questionnaire for Steering B. The implementation of the evaluation: 121 Committee Members (check appropriate answer): Considerably, amply on the Multi-Stakeholder Adequately CDF Evaluation Process Somewhat but would have liked more Given that the CDF evaluation has been a major multi-donor effort and that such evalua- Not nearly enough tions are likely to increase in frequency, the Comments: stakeholders in this evaluation should have an opportunity to say what works and what doesn't. Please tell us about any specific kind of input that you or your organization would have liked Stakeholders involvement in the CDF evalua- to provide, but could not or were not able to? tion has taken several forms: Why? Membership on the steering committee 2. For you and your organization, how useful was Financial support each of these forms of involvement? Commenting on drafts via e-mail Participation in steering committee meetings Participation (e.g., of a staff member) on Very useful country case study mission teams Somewhat Useful Not useful Formationofand/orparticipationincasestudy More trouble than it was worth country "reference" groups and workshops. Didn't use this method We would like to get your views about the ben- Review of documents/drafts on the Web site efits and costs of the multi-stakeholder approach Very useful and the forms that your participation took. Somewhat Useful Not useful There are seven questions in all; three that ask More trouble than it was worth for ratings and four that are open-ended. Your Didn't use this method answers will be recorded anonymously. Please submit only one set of answers for each partici- Participation (e.g. by staff) in missions pating government or organization. Very useful Somewhat Useful 1. To what extent did you (or your organiza- Not useful tion) have an opportunity to influence: More trouble than it was worth A. The design of the evaluation: (check Didn't use this method appropriate answer) Considerably, amply Participation in country workshops Adequately Very useful Somewhat Useful Somewhat but would have liked more Not useful Not nearly enough More trouble than it was worth Comments: Didn't use this method 122 Financial support Administrative costs Very useful Positive impact/improved Somewhat Useful Mixed but largely positive Not useful Mixed but largely negative More trouble than it was worth Negative impact Didn't use this method No impact either way Comments: Other (please state) Positive impact/improved 3. In your opinion, what impact did the multi- Mixed but largely positive stakeholder approach have on the evaluation, Mixed but largely negative along the four listed dimensions? Negative impact No impact either way Diversity and quality of evidence Positive impact/improved Comments: Mixed but largely positive Mixed but largely negative 4. What do you think were the major advan- Negative impact tages, if any, of a multi-stakeholder approach? No impact either way 5. What do you think were the disadvantages, if Quality of the analysis any, of the multi-stakeholder approach? Positive impact/improved Mixed but largely positive 6. What might have been done differently to Mixed but largely negative achieve the advantages at lower cost? Negative impact No impact either way 7. Did the CDF evaluation use the right methodological tools to answer the evaluation Credibility of the findings questions we set out to answer? If not, what Positive impact/improved should have been used? Mixed but largely positive Mixed but largely negative Thank you for your inputs. Negative impact The Evaluation Secretariat No impact either way Timeliness/elapsed time of the evaluation Positive impact/improved Mixed but largely positive Mixed but largely negative Negative impact No impact either way Endnotes 123 Chapter One 1. For a definition of development that includes much more than income levels, see Sen 1999. The annual UNDP Human Development Reports have also emphasized the broader dimensions of development. 2. Within a year after agreeing to have CDF implementation monitored for a pilot period, Jordan was dropped at its request. 3. The CDF Secretariat consists of nine World Bank staff members and is located in the Operations Policy and Country Services Vice Presidency. 4. The terms "framework" and "vision" require some clarification. As used in this evaluation, a vision becomes a framework when priorities are established for the elements of the vision. Therefore, the first CDF principle is called the Long-term, Holistic Development Framework. An alternative approach, adopted by the World Bank CDF Secretariat, is to refer to the principle as the Long-term, Holistic Vision. 5. Moreover, several donor countries have multiple agencies, each with their own procedures. Both bilateral and mul- tilateral agencies are likely to be undercounted in table 1.2, since non-DAC donors are not included, and some multilat- eral agencies are undoubtedly missing. 6. Poverty Reduction Strategy Papers--Operational Issues, December 10 (1999); The Poverty Reduction and Growth Facility (PRGF)--Operational Issues, December 13 1999 (World Bank/IMF 1999a, b). 7. A chart, reproduced in Annex 4[B], shows the CDF as the overarching framework; the PRSP as the mechanism for articulating macroeconomic, social, and structural policies with the actions of all relevant stakeholders; and the various donor country strategies, such as the CAS, PRGF, and UNDAF, as the business plans that support it. This chart provides the clearest depiction of the PRSP as an action plan of the CDF. 8. In April 2000, the president of the Bank and the managing director of the IMF issued a joint statement on the rela- tionship between the CDF and the PRSP. The statement synthesizes the basic principles of the CDF and shows the PRSP to be an operational vehicle based on it. It states that the CDF and the PRSP should be mutually reinforcing. See The Comprehensive Development Framework (CDF) and Poverty Reduction Strategy Papers (PRSP), included as Annex 1. 9. In 2001 the Bank introduced the PRSC to support policy and structural reforms set out in the PRSP, complement- ing the PRGF's medium-term support of macroeconomic policies. The PRSC provides for the possibility of cofinancing with other donors, thus enhancing multidonor support for the PRSP. 10. The prospect of the PRSP serving as the sole basis for conditionality, however, is still remote. Even the more streamlined IMF PRGF still has separate performance criteria and benchmarks (although fewer than before), and condi- tions for HIPC completion include a number of other "triggers." 11. Under the Enhanced HIPC initiative, debt relief is delivered in two stages. At decision point, eligibility for assis- tance and for receipt of interim relief is established and requires an I-PRSP. The country receives the bulk of assistance at completion point, which requires one year's implementation of a full PRSP. 12. Twenty-two countries were brought to decision point in one year, compared with seven between 1998 and 1999. 124 13. This issue was raised by government representatives at the International Conference on national PRSPs in Washington, in January 2002. It has been acknowledged in "lessons for staff" in the recent Bank-Fund comprehensive review of the PRSP: Review of the Poverty Reduction Strategy Paper Approach--Main Findings, and Review of the Poverty Reduction Strategy Paper Approach--Early Experiences with Interim and Full PRSPs (World Bank 2002a). 14. See External Comments and Contributions on the Joint Bank/Fund Staff Review of the PRSP Approach, Vol I: Bilateral Agencies and Multilateral Institutions; Vol II: Civil Society Organizations and Individual Contributions. IMF/World Bank, January and February 2002 (World Bank 2002d, 2002e). 15. The PRSP was originally to be updated every three years. This corresponded with the three-year cycle of the PRGF, CAS, and PRSC. However, following the PRSP Review, there is now more flexibility about PRSP updating. 16. This appears to have improved Bank-Fund coordination on PRSP-related issues. Chapter Two 17. Also allegedly never invited or underrepresented in PRSP processes are trade unions and women's organizations, as well as local governments in outlying areas. See Lister and Nyamugasira 2001. 18. The government of Uganda has internalized results orientation through its own poverty plan (the PEAP) and attendant M&E strategy. Chile has also developed a viable homegrown evaluation system. Brazil's emphasis on programs, clear goals/objectives, and performance indicators is very similar to the U.S. Government Performance and Results Act (GPRA). 19. For example, the macro data used in PRSPs and other strategic plans are often two or more years old. 20. Management notes that this is inconsistent with the CDF principles, which advocate regular and sustained stake- holder participation with strong links to institutions. 21. Evidence is largely drawn from two sets of survey questionnaires administered for the CDF evaluation (attached as Annex 6): (i) a survey on changes and trends in government-donor partnerships in five CDF case study countries-- Bolivia, Ghana, Romania, Uganda, and Vietnam--that sought information on composition of donor portfolio, donor coordination, donor administrative and procedural requirements, and donor implementation practices (a total of 26 donors--17 bilateral and 9 multilateral--across the five countries that responded to the survey); and (ii) a structured survey questionnaire on the implementation of country-led partnership in the context of the CDF prepared as part of a thematic study, "Country-led Partnership." A total of 13 responses were received from the headquarters as well as coun- try offices of six bilateral donors and one multilateral institution. For recipients' views, the findings of the six country case studies in the CDF evaluation have been drawn upon--Bolivia, Burkina Faso, Ghana, Romania, Uganda, and Vietnam. A reply was also received from the local World Bank team in Mozambique. For both donors and recipients, the sample is relatively small, but the responses are illuminating (see Holmgren and Soludo 2002, pp. 29-42). 22. Inference about progress depends largely on the country selection and the period covered by the study. For exam- ple, there are important differences between some of the survey results, the results summarized from the individual case studies for this CDF evaluation, and the summary of findings reported by an OECD study using a different set of 11 poor countries. 23. Forty-two of these countries are PRSP countries. World Bank. 2001a, pp. 13, 30-31. 24. It is likely that a number of these figures are underestimates. The Country Office of the World Bank in Romania reports that it received over 10 missions per month in the last quarter of calendar year 2002. 25. One factor that may account for this growth is increased emphasis by the OECD DAC Evaluation Working Party on joint evaluations, with the Evaluation Working Party serving as a point of coordination and follow-up for some efforts, such as a recent Joint Evaluation of the Education Sector in Bolivia. These data combine monitoring, supervision, and evaluation missions. There could be greater differences in each of the three components. 26. There remains a strong preference for separate missions in Ghana, partly due to a rise in donor competition and 125 need/pressure for "attribution" and "planting the flag," as Ghana is once again emerging as an African success story. 27. Germany and Japan, for example. The United States Agency for International Development (USAID) is restricted by legislative provisions, particularly regarding sectoral priorities. Within these parameters, however, its local mission directors have considerable latitude to engage in harmonization efforts. Chapter Three 28. These sectors have also been emphasized in HIPC programs. SWAps typically are constructed through participa- tory mechanisms, involve agreements between stakeholders (including donors) over objectives, activities and targets, pooled funding, and joint monitoring. 29. The MTEF provides aggregate and sectoral budget ceilings consistent with resource availabilities, typically over a three-year period on an annually rolling basis. 30. Uganda repackaged the PEAP summary as a PRSP to comply with the formalities. It had already received debt relief under the original HIPC initiative in 1998, and reached its decision and completion points under the Enhanced HIPC initiative in only two months. 31. Particular weaknesses were the selection of indicators and costings, which both tended to be left until the last minute. 32. In addition to country program efforts, particularly those supported through Poverty Reduction Support Credits (PRSCs), the World Bank and partner donors are supporting several PRSP-related capacity building programs, including the World Bank Institute "Attacking Poverty Program" and the PRSP Trust Fund. 33. The Bolivian PRSP consultation process had to be suspended in April 2000 when a state of siege was declared to suppress social protests. 34. The four indicators are measures of contract enforcement, credit markets, entry regulations, and labor regulations. 35. The ratings of policy and institutional quality are taken from the results of the Country Policy and Institutional Assessment (CPIA). The CPIA is an internal World Bank index of 20 equally weighted components. World Bank coun- try specialists rate each component on a scale of 1-6, using standardized criteria. The components are grouped into four categories: (a) Macroeconomic management and sustainability of reforms; (b) Structural policies for sustainable and equi- table growth, (c) Policies for reducing inequalities, and (d) Public sector management. Although the CPIA is an inter- nal World Bank index, and is, therefore, subject to concerns about subjectivity and transparency, it is comprehensive and available for all recipient countries. A recent report by the World Bank's CDF Secretariat (2001a, p. 25) observes, "the CPIA process has recently being strengthened by improving its clarity and making its criteria more explicit by requiring a written explanation of each country's rating on each question, and implementing regular annual discussions with IDA recipient countries on the results for their country, thereby allowing the CPIA to be an input in the upstream dialogue with IDA countries." Further work has begun on better linking CPIA with CDF/PRSP processes, the Bank's country assis- tance strategy, and other economic and sector work. 36. This is a component index of the CPIA (see previous footnote). 37. It is argued that country-led partnership is likely to be higher, or at least the conditions for country-led partner- ship would be higher, for countries where most aid is provided by a few donors and is evenly distributed across many sectors, and technical assistance is not too high, relative to what is required for enhancing institutional capacity of these countries. 38. Limited data precluded using income inequality, though it is likely to generate similar results, as in the case of the growth literature (e.g., Rodrick 1999). 39. Similar results also obtain in analysis of cross-country growth regressions, where the negative marginal effect of 126 ethnic fractionalization, estimated in this model, either disappears (e.g., Collier 2001), or is substantially reduced (e.g., Easterly 2000). 40. When the CPIA was regressed against these factors after it had been regressed against the CDF-like index (i.e., once the CDF-like index had been taken into account), it was found that the factors had no further influence (explana- tory power) on the CPIA. Chapter Four 41. Accession to the EU can serve this function to a certain extent, as it does in Romania. But there are limitations. As noted in the Romania case study, the EU does not attempt to harmonize policy in some sectors among its members (for example, health, education, and poverty), and therefore does not stress them in its requirements for accession. Romania seeks to link its new anti-poverty policy to the EU policy on social inclusion, but this is not a requirement for accession. 42. Among these donors are Denmark, the EC, Ireland, the Netherlands, Sweden, the U.K., and the World Bank. Performance of most donors varies from country to country. 43. See Eriksson and others 2002, Annex D-1, 2, and 4. Annex Two 44. See the OED 1999 for an examination of the lessons of development experience through the lens of the CDF principles. 45. Documentation can be found on the CDF Evaluation Web site. 46. The term "development agencies" is used to mean official bilateral donors and multilateral development assistance organizations, as well as international NGOs. 47. As discussed at greater length below, the evaluation will seek complementarity and to avoid duplication with ongoing PRSP monitoring and evaluation efforts, such as those being undertaken under the auspices of the Strategic Partnership with Africa (SPA) and the U.N. Economic Commission for Africa (UNECA). 48. This does not preclude the possibility that, for example, country studies or the synthesis study would recommend that an independent panel to review donor performance be considered, but this should not be confused with the country studies themselves. 49. The IDGs call for the achievement of quantitative global goals in poverty reduction, infant and maternal mortal- ity, primary and secondary education, access to reproductive health services, eliminating gender disparities, and environ- mental sustainability over the next 15 years. 50. The thematic studies will also include the experience of non-CDF pilot countries in their analyses but will not involve field work. 51. Members from the case study countries on the main steering committee will be asked to help identify members for each country steering committee and in the selection of the local institution. 52. The questions posed in the survey and interview protocols will include "core questions" developed in the evalua- tion Design Paper, but will also include country-specific questions proposed by the local steering committee and partner institution(s). 53. In-depth country case studies on PRSP implementation are being conducted in eight African countries under the auspices of the SPA. The countries are: Benin, Ghana, Kenya, Malawi, Mali, Mozambique, Rwanda, and Tanzania. The studies are being conducted by the Overseas Development Institute (ODI) in London, under the supervision of DFID. Preliminary reports are to be prepared by mid-2001. ODI, with DFID support, is also embarking on a PRSP monitoring effort. UNECA is establishing a "PRSP Learning Group" to provide a forum for exchange of views and experiences with the PRSP process among African countries. The CDF evaluation secretariat has initiated a dialogue with the managers of these efforts. 54. These considerations are discussed in two recent documents: Devarajan, Dollar, and Holmgren 2001; Collier and 127 Dollar 2001. 55. Ultimate development impacts (e.g., life expectancy) are jointly determined by the adoption and implementation of the CDF principles as well as the intermediate outcomes and processes (e.g., expenditure programs, central bank inde- pendence, or extent of engagement of civil society). Intermediate outcomes/ processes, in turn, are also determined by the application of CDF principles. 56. In a paper prepared for the October 19-20, 2000 Workshop on the CDF Evaluation, John Williamson suggests that in these cases, "striking the right balance will involve recognizing (a) that there is room for professional disagreement even among those who share a common intellectual framework, and (b) that no one is omnipotent and there is often scope for extra ideas to be injected into the policy dialogue to mutual advantage" (Williamson 2000, p. 3). 57. This term and the set of issues it entails are discussed in Eriksson 2001, pp. 15-17. Annex Four 58. Within a year after agreeing to have CDF implementation monitored for a pilot period, Jordan was dropped at its request. Annex Five 59. These processes are outlined in the Design Paper for this evaluation (World Bank 2001c, pp. 10-11). 60. Polity IV code is a dataset users manual produced by the POLITY IV project. This project is part of the Integrated Network for Societal Conflict Research (INSCR) Program in the Center for International Development and Conflict Management (CIDCM) at the University of Maryland, College Park, MD. 61. The estimates for these a's are Iit = ­9.7991 + 0.1946Tt-1 + 0.0993Tt-1 x It-2 ­ 0.0125Tt-1 x It-2 + 1.5564LnYt-1 + 0.0129Sch 2 (2.01) (2.34) (-3.48) (5.00) (3.27) (t-values in brackets). 62. Our ICRG measure is a simple average of two ICRG measures of law and order, bureaucratic quality. Higher values of the measures denote better institutions. Annex Six 63. The previous question asked about resource allocations/policy decisions. This one asks about results. Bibliography 129 Ali, A., and A. Disch 2002. "On the Long-Term Holistic Development Framework Principle: An Evaluation." CDF Evaluation Secretariat, Operations Evaluation Department, World Bank, Washington, D.C. Amis, P., and L. Green 2002. Survey on Partners' Priorities and Perspectives on Harmonizing Donor Practices. The University of Birmingham, U.K.: International Development Department, School of Public Policy. Anderson, D., and others. 2002. Ownership of SIDA Projects and Programs in East Africa: A Synthesis Report of Kenya, Tanzania, and Uganda. Stockholm: Swedish International Development Authority. Berg, Elliot. 1999. "Why Aren't Aid Organizations Better Learners?" Paper prepared for the Expert Group on Development Issues, Swedish Ministry of Foreign Affairs, Stockholm. Boesen, Nils, Laura Kullenberg, José Antonio Peres A., and Juan Carlos Requena. 2002. Bolivia CDF Evaluation Country Case Study. CDF Evaluation Secretariat, Operations Evaluation Department, World Bank, Washington, D.C. Boesen, Nils, Abena Oduro, Tony Killick, Laura Kullenberg, and Mirafe Marcos. 2002. Ghana CDF Evaluation Country Case Study. CDF Evaluation Secretariat, Operations Evaluation Department, World Bank, Washington, D.C. Bonaglia, F., J. Braga de Macedo, and M. Bussolo. 2001. How Globalisation Improves Governance. OECD Development Centre Technical Paper No. 181. Paris. Chapman, Jake. 2002. Systems Failure: Why Governments Must Learn to Think Differently. London: Demos. CIDA (Canadian International Development Agency). 2002. Canada Making a Difference in the World. A Policy Statement on Strengthening Aid Effectiveness. Hull, Quebec. Collier, Paul. 2001. "Implications of Ethnic Diversity." Economic Policy 5(32): 127-55. Collier, Paul, and D. Dollar. 2001. "Development Effectiveness: What Have We Learned?" World Bank. Photocopy. Collier, Paul, and Cathy Pattillo, eds. 1999. Investment and Risk in Africa. London: Macmillan. Devarajan, S., D. Dollar, and T. Holmgren. 2001. Aid and Reform in Africa. Washington, D.C.: World Bank. Disch, Arne, and Ali Abdel Gadir Ali. 2002. On the Long-Term Holistic Development Framework Principle of the CDF: An Evaluation. CDF Evaluation Secretariat, DECRG (Development Economics Research Group), World Bank, Washington, D.C. Easterly, William. 2002. The Cartel of Good Intentions: Bureaucracy versus Markets in Foreign Aid. Working Paper No. 4. Washington, D.C.: Center for Global Development. _____. 2001. The Elusive Quest for Growth: Economists' Adventures and Misadventures in the Tropics. Cambridge, MA: MIT Press. _____. 2000. "Can Institutions Resolve Ethnic Conflict?" Photocopy. Elbadawi, Ibrahim. 2001. "Social Cohesion, Conflict Management and Economic Growth in Africa." In T. Assefa, S. Rugumanu, and A. Ahmed, eds, Globalization, Democracy, and Development in Africa: Challenges and Prospects. Oxford, U.K.: African Books Collectives. Elbadawi, Ibrahim, and Alan Gelb. 2002. "Financing Africa's Development: Toward a Business Plan." Paper pre- sented at the AERC Senior Policy Seminar, Dar es Salaam, Tanzania. 130 Elbadawi, Ibrahim, George Mavrotas, and John Randa. 2003. The Development Impact of CDF-like Experiences: An Assessment. DECRG (Development Economics Research Group), World Bank, Washington, D.C. Eriksson, John R. 2001. The Drive to Partnership: Aid Coordination and the World Bank. Operations Evaluation Department Study Series. Washington, D.C.: World Bank _____. 1999. Aid Coordination: Moving Toward Partnership--The Challenge of Measurement. Operations Evaluation Department. Washington, D.C.: World Bank. Eriksson, John, David Pedley, Rosern Rwampororo, and Mirafe Marcos. 2002. Uganda CDF Evaluation Country Case Study. CDF Evaluation Secretariat, Operations Evaluation Department, Washington, D.C. Eriksson, John, Lynn B. Salinger, Dumitru Sandu, and Manuela Sofia Stanculescu. 2002. Romania CDF Evaluation Country Case Study. CDF Evaluation Secretariat, Operations Evaluation Department, World Bank, Washington, D.C. EC (European Commission). 2001. Comparative Review of I-PRSP Targets and Conditionalities for HIPC Completion Point. SPA Task Team on Contractual Relationships and Selectivity. Brussels. _____. 2000. Review of Conditionalities Used for the Floating HIPC Completion Point. SPA Task Team on Contractual Relationships and Selectivity. Brussels. Evans, A. 2002. "Bank Strategy in Low-Income Countries: How Relevant and Effective Is the PRSP Process?" Poverty Reduction and Economic Management (PREM) Week. World Bank, Washington, D.C. GAO (U.S. General Accounting Office). 1990. Case Study Evaluations. Washington, D.C.: GPO. Gurr, Ted R. 1974. "Persistence and Change in Political Systems, 1800­1971." The American Political Science Review 68 (4): 1482-1504. Hatry, Harry, and Kerfalla Yansane. 2002. Comprehensive Development Framework Evaluation Results Orientation: An Early Look. CDF Evaluation Secretariat, DECRG (Development Economics Research Group), World Bank, Washington, D.C. Heimans, J. 2002. Strengthening Participation in Public Expenditure Management: Policy Recommendations for Key Stakeholders. Policy Brief No. 22, OECD Development Centre, Paris. Helleiner, Gerry. 2000. Toward Balance in Aid Relationships: Donor Performance Monitoring in Low-Income Developing Countries. Prepared for a Festschrift in honor of Lance Taylor. Herbst, J., and C. Soludo. 2001 Aid and Reform in Nigeria. In S. Devarajan and others, eds., Aid and Reform in Africa. Washington, D.C.: World Bank. Holmgren, T., and C. Soludo. 2002. Implementation of Country-Led Partnership in the Context of the Comprehensive Development Framework. CDF Evaluation Secretariat, Development Research Group and Operations Evaluation Department, World Bank, Washington, D.C. IMF/IDA (International Monetary Fund and International Development Association). 2002. Review of the Poverty Reduction Strategy Paper (PRSP) Approach: Main Findings. Washington, D.C. Jerve, Alf Morten, Ray Mallon, Keiko Nishino, Han Manh Tien, and Laura Kullenberg. 2002. Vietnam CDF Evaluation Country Case Study. CDF Evaluation Secretariat, Operations Evaluation Department, World Bank, Washington, D.C. Jolly, Richard, Giovanni Correa, and Frances Stewart, eds. 1987. Adjustment with a Human Face. New York: Oxford University Press. Kanbur, R., T. Sandler, and K.M. Morrison, 1999. The Future of Development Assistance: Common Pools and International Public Goods. Overseas Development Council Policy Essay #25. Baltimore, MD: Overseas Development Council. Lallement, Dominique, Della McMillan, Kyran O'Sullivan, Patrick Plane, and Kimsey Savadogo. 2002. Burkina Faso CDF Evaluation Country Case Study. CDF Evaluation Secretariat, Operations Evaluation Department, World Bank, Washington, D.C. Lancaster, Carol, and Heba Handoussa. 2002. Evaluating the CDF: Ownership and Participation. CDF Evaluation 131 Secretariat, DECRG (Development Economics Research Group), World Bank, Washington, D.C. Lancaster, Carol, and Samuel Wangwe. 2000. Managing a Smooth Transition from Aid Dependence in Africa. Washington, D.C.: Overseas Development Council and the African Economic Research Consortium. Lipumba, I.H. 2001 Conditionality and Ownership: A View from the Periphery. In External Comments and Contributions on IMF Conditionality. Washington, D.C.: IMF. Lister, S., and W. Nyamugasira. 2001. The Involvement of Civil Society in Policy Dialogue and Advocacy. London: DFID. Maxwell, S. 2003."Heaven or Hubris: Reflections on the `New Poverty Agenda.'" Development Policy Review 21(1): 5-25. Maxwell, S., and T. Conway. 2000. "Perspectives on Partnership." Operations Evaluation Department, World Bank, Washington, D.C. Marshall, Monty G., and Keith Jaggers. 2000. "Polity IV Project: Political Regime Characteristics and Transitions, 1800­1999, Dataset Users' Manual." Center for International Development and Conflict Management, University of Maryland, College Park. Ministry for Foreign Affairs, Sweden. 1999. Making Partnerships Work on the Ground. Workshop Report, Ulvsunda Castle. Stockholm. O'Connell, S. A., and C. Soludo. 2001. "Aid Intensity in Africa." World Development 29(9):1527-52. OECD/DAC (Organization for Economic Cooperation and Development / Development Assistance Committee). 2001a. DAC Guidelines on Poverty Reduction. Paris. _____. 2001b. Task Force on Donor Practices: Progress Report. Paris. _____. 1999. On Common Ground: Converging Views on Development and Development Cooperation at the Turn of the Century. Paris. _____. 1996. Shaping the 21st Century: The Contribution of Development Co-operation. Paris. OED (Operations Evaluation Department, the World Bank). 1999. 1999 Annual Review of Development Effectiveness. Washington, D.C.: World Bank. Picciotto, Robert. 2001. Development Cooperation and Performance Evaluation: The Monterrey Challenge. Operations Evaluation Department, World Bank, Washington, D.C. _____. 1998. The Logic of Partnership: A Development Perspective. Operations Evaluation Department, World Bank, Washington, D.C. Rodrik, Dani. 1999. "Where Did All the Growth Go? External Shocks, Social Conflict and Growth Collapses." Journal of Economic Growth 4(4): 385-412. Sen, Amartya. 1999. Development as Freedom. New York: Alfred A. Knopf. Soludo, Charles. 2002. "Toward the Next Generation of Reforms of Aid Policies and Global Governance." Paper presented at the ADB/OECD Third International Forum on African Perspectives, Paris. _____. 1998. Monitoring Economic Performance in Africa: Toward the Construction of Performance and Policy Stance Indices. A Methodological Note for the Construction of the UNECA Indices. Discussion Paper Series, ESPD/DPS/98/2. Addis Ababa, Ethiopia: UNECA. Stiglitz, Joseph E. 1998. "Towards a New Paradigm for Development: Strategies, Policies and Processes." 1998 Prebisch Lecture, UNCTAD, October 19, 1998. Geneva. UNCTAD (United Nations Conference on Trade and Development). 2002. The Least Developed Countries Report 2002: Escaping the Poverty Trap. Geneva. United Kingdom. Eliminating World Poverty: A Challenge for the 21st Century. British Government White Paper (1997). London. 132 United Nations. 2001. Road Map Towards the Implementation of the United Nations Millennium Declaration. New York. van de Walle, Nicolas, and Timothy Johnson. 1996. Improving Aid to Africa. ODC Policy Essay No. 21. Washington, D.C.: Overseas Development Council. Williamson, John. 2000. "Country Ownership, Public Participation, and the CDF." Paper prepared for the 2000 Workshop on the CDF Evaluation, October 19-20. World Bank, Washington, D.C. Wolfensohn, James D. 1998. "The Other Crisis." Address to Board of Governors, World Bank, Washington, D.C. _____. 1997. The Challenge of Inclusion. Annual Meetings Address. Hong Kong, China, World Bank. World Bank/International Monetary Fund. 2002. Review of the Poverty Reduction Strategy Paper (PRSP) Approach: Main Findings. Washington, DC. _____. 2001. Tracking of Poverty-Reducing Public Spending in Heavily Indebted Poor Countries. Washington, D.C. _____. 1999a. Poverty Reduction Strategy Papers--Operational Issues. Washington, D.C. _____. 1999b. The Poverty Reduction and Growth Facility (PRGF)--Operational Issues. Washington, D.C. World Bank. 2002a. Review of the Poverty Strategy Paper Approach--Main Findings and Issues for Discussion. International Conference on National PRSPs, Bank-Fund. Washington, D.C. _____. 2002b. Roundtables on Latin American Middle-Income Country Experience. The CDF Secretariat, Operations Evaluation Department, World Bank, Washington, D.C. _____. 2002c. Review of the Poverty Strategy Paper Approach--Early Experiences with Interim and Full PRSPs. International Conference on National PRSPs, Bank-Fund. Washington, D.C. _____. 2002d. External Comments and Contributions on the Joint Bank/Fund Staff Review of the PRSP Approach. Vol. I: Bilateral Agencies and Multilateral Institutions. International Conference on National PRSPs, Bank-Fund. Washington, D.C. _____. 2002e. External Comments and Contributions on the Joint Bank/Fund Staff Review of the PRSP Approach. Vol. II: Civil Society Organizations and Individual Contributions. International Conference on National PRSPs, Bank- Fund. Washington, D.C. _____. 2001a. Comprehensive Development Framework: Implementation Experience in Low-and Middle-Income Countries. The CDF Secretariat, Washington, D.C. _____. 2001b. Comprehensive Development Framework: Meeting the Promise? Early Experience and Emerging Issues. The CDF Secretariat, Washington, D.C. _____. 2001c. Design Paper for a Multi-Partner Evaluation of the Comprehensive Development Framework. CDF Evaluation Secretariat, Washington, D.C. _____. 2000a. Comprehensive Development Framework: Country Experience, March 1999­July 2000. The CDF Secretariat, Washington, D.C. _____. 2000b. Annual Review of Development Effectiveness. Operations Evaluation Department Study Series. Washington, D.C.: World Bank. _____. 2000c. JSA Guidelines at Annex II, PRSP. Progress in Implementation, August 14 (EBS/00/167), Washington, D.C. _____. 2000d. Can Africa Claim the 21st Century? Washington, D.C. _____. 1999. HIPC Initiative--Strengthening the Link between Debt Relief and Poverty Reduction. Development Committee DC/99-24 (E). Washington, D.C. _____. 1996. Access to Education and Health Care in Uganda. Eastern Africa Department and Social Policy Department, Country Operations Division, Washington, D.C.