THE rI IDLD IF LrJJ( rrJr f r !LUJQrJJD E E rPr'?TJAIT aD K -- i 22899 A~~~~~~A A~~~~~~~~~~~~~~~~~~~~~~~AL ~~r a ,\-.._ -t .J4 _ t,s'r.- , * a_ t Applyig the Comprehensive < t Development Framework to f Z USAID Experiences j[ StED~~~~~~~~~~~~~ \IIr;r F12 t ,,a, - n- - WOWRLLBD BOANEK OTPEORAT IOSNS EVA LVUATIOAN DTEIPOANRDTEMPENTT EDN 22899 Applying the Comprehensive Development Framework to USAID Experiences OED Working Paper Series * No. 15 James Fox Summer 2000 The World Bank www.worldbank.org/html/oed Washington, D.C. Copyright 2000 Operations Evaluation Department Partnerships & Knowledge Programs (OEDPK) Email: ecampbellpage@vorldbank.org Email: eline@worldbank.org Telephone: 202-473-4497 Facsimile: 202-522-3125 The opinions expressed in this report do not necessarily represent the views of the World Bank or its member governenits. The World Bank does not guarantee the accuracy of the data included in this publication and accepts no responsibility whatsoever for any consequence of their use. The boundaries, colors, denomina- tions. and other information shown on any map in this volurne do not imply on the part of the World Bank Group any judgment on the legal status of any territory or the endorsement or acceptance of such boundaries. ii Contents v Acknowledgments vii Preface 1 Introduction 3 Republic of Korea: The Making of a Miracle 3 The Nature of U.S. Assistance 4 Aid to Korea in a Holistic, Long-Term Development Framework 5 Ownership 6 Partnership 6 Results Orientation 7 Smallpox Eradication: Foreign Aid's Unequivocal Accomplishment 9 Smallpox in a Holistic, Long-Term Development Plan 9 Ownership 10 Partnership 10 Results Orientation 13 Family Planning: From Donor Unilateralism to Local Ownership 14 Family Planning in a Holistic, Long-Term Development Framework 14 Ownership 15 Partnership 15 Results Orientation 17 Agriculture: Technology Matters Enormously 19 Agricultural Research in a Holistic, Long-Term Development Framework 19 Ownership 19 Partnership 19 Results Orientation 21 Foreign Academic Training: Producing New Leaders 22 Foreign Academic Training in a Holistic, Long-Term Development Framework 22 Ownership 23 Partnership 23 Results Orientation 25 U.S. Assistance to Egypt: Eventual Progress? 26 U.S. Assistance to Egypt in a Holistic, Long-Term Development Framework 26 Ownership 26 Partnership 26 Results Orientation 29 Conclusions 30 Holistic, Long-Term Development Framework 30 Ownership 31 Partnership 31 Results Orientation 33 Endnotes 35 References Box 17 Box 1. Effective Agricultural Technology Development iii Acknowledgments James Fox prepared this Working Paper for the 1999 Annual Review of Development Effectiveness, under the direction of Nagy Hanna. the task manager. James Fox. an independent consultant, was formerly lead economist at the evaluation office of the U.S. Agency for International Development. The Working Paper Series was published in the Partnerships and Knowledge Group (OEDPK), by the Outreach and Dissemination Unit. The task team includes Elizabeth Campbell-Page (task team leader), Caroline McEuen (editor). Kathy Strauss (graphics and layout), and Juicy Qureishi-Huq (administrative assistant). Director-General, Operations Evaluation: Robert Picciotto Director, Operations Evaluation Department: Gregory Ingranm Task Manager: Nagy Hanna V Preface Despite the potential benefits of globalization and technological change. world pov- erty has increased and growth prospects have dimmed for developing countries dur- ing the 1980s and 90s. The Comprehensive Development Framework (CDF) was launched by the World Bank in January 1999 in response to these difficult circumstances. It has evoked considerable interest throughout the development community as an approach that can address the increasingly intertwined challenges faced by development practitioners. Its basic elements are not new. What is new is their joint articulation as a framework to guide development assistance. The first point is that development constraints are struc- tural and social, and cannot be overcome through economic stabilization and policy ad- justment alone-they require a long-term and holistic vision of needs and solutions. Second, policy reform and institutional development cannot be imported or imposed; without domestic ownership. reforms and investments are not sustainable. Third, successful de- velopment requires partnership among government, local communities, the private sec- tor, civil society, and development agencies. And fourth, development activities must be guided and judged by results. In this context, the 1999 Annual Review of Development Effectiveness (ARDE), authored by Nagy Hanna under the guidance of Robert Picciotto, set out to examine development experience through the lens of CDF principles. A number of papers were commissioned to support the ARDE by providing in-depth review of evaluation and re- search findings that assess the relevance of the CDF principles and constraints as well as promising approaches to their implementation. vii Introduction Using the four main tenants of the World Bank's Comprehensive Development Frame- work (CDF)-a holistic, long-term development plan: host country ownership; do- nor partnership; and results orientation-to analyze development activities yields useful insights into successes and failures. Here. I apply it to six U.S. Agency for International Development (USAID) activities. Five were chosen because they are broadly regarded within USAID as conspicuous successes-assistance to the Republic of Korea, family plan- ning, agricultural research, university training, and smallpox eradication. The sixth case- assistance to Egypt-is included because it is typically regarded within the institution as USAID's most conspicuous failure. The analysis sheds light on how the four key elements of the CDF contributed to USAID's performance and offers some suggestions on addi- tional dimensions to be considered in applying the CDF. Two of the case studies-Korea and smallpox eradication-involve programs com- pleted in the late 1 970s for which the outcome is clear and a wealth of evaluative material exists. The other four cases represent ongoing activities, for which'achievement of neither the ultimate outcome nor the specific goal is yet evident. The evidence in these cases is necessarily more tentative. 1 Republic of Korea: The Making of a Miracle Korea was one of the largest recipients of U.S. aid during the 1950s and early 1960s. Many U.S. officials feared that U.S. foreign aid was encouraging a permanent de- pendence on high levels of aid to maintain economic stability. In 1962-63, however, changes in government policy produced rapid export-led economic growth, and dependence on U.S. aid soon came to an end. The Nature of U.S. Assistance The basic facts are straightforward. Korea had been under Japanese domination for 35 years prior to its occupation by U.S. and Russian forces at the end of World War II. Very little economic growth took place during the colonial period, and Korea was among the poorest countries in the world in 1945. Partition after the war had placed the more indus- trial north in the Soviet zone of control and the poorer and more agricultural south in the U.S. zone. In the south lands owned by the Japanese were redistributed by the U.S. mili- tary government after the war: the Korean government also redistributed large Korean- held landholdings. The result was a substantial reduction in rural inequality. In 1950-53 the country was devastated by the war with the People's Republic of Korea. The Republic of Korea received massive amounts of U.S. foreign aid-$12.7 billion between 1945 and 1975 (more than $50 billion in 1998 dollars), which constituted the bulk of foreign aid to the country from all sources. Virtually all aid was in grants, roughly equally divided between economic and military aid. Aid was extremely large relative to the size of the Korean economy, averaging 8.1 percent of GDP during 1953-62 and fi- nancing 69 percent of imports. Exports were insignificant (2 percent of GDP) during this period. and in 1961 ($39 million) were even below the 1953 level. The economy grew modestly during the 1950s, averaging 4.8 percent during 1954-58 and slowing to 3.4 percent during 1959-62. Then beginning in 1963 growth rates rose dramatically, averaging nearly 10 percent a year during 1963-66, a pace maintained for most of the next three decades. Exports took off in 1962. more than quadrupling between 1961 and 1965, when they reached $174 million. The explosive growth in exports mod- erated only slightly over the next decade. By 1975 exports had reached $5.1 billion, or about 30 percent of GDP. The change in economic fortunes followed changes in political leadership. The Park Chung He regime. which took power in a coup in 1961, made exports its economic cen- terpiece. The country's economic dynamism did not falter with Park's assassination in 1972. Rapid growth of exports and the economy continued into the 1990s, leading to the 3 APPLYING T HE COMPREHENSIVE DEVELOPMENT FRAMEWORK TO USAID EXPERIENCES World Bank's description of Korea's development as an "economic miracle." In four de- cades Korea had transformed itself from one of the poorest countries in the world to a major economic power-the world's ninth largest exporter. Korea's early growth and export success continued to surprise external observers. In the mid-1960s, the World Bank thought planned growth rates were too high. In 1971 Cole and Lyman (p. v) argued that Korea's rapid growth was unlikely to continue, due to "structural distortions emerging in the economy ... and ... increasing resistance to taking corrective action." Similar assessments followed the 1997 Asian financial crisis, yet de- spite the structural distortions (in 1999 failure to deal with insolvent banks and bankrupt chaebol), the Korean economy in mid-1999 appeared to be back on the road to rapid economic growth. While Korea's growth record is widely recognized. the influence of U.S. economic assis- tance has long been debated. Eight studies by the Harvard Institute of International Devel- opment, summarized in Mason and others (1970), and Steinberg's analyses (1982, 1985, 1989) for USAID are the key explorations of USAID's experience in Korea. Overall, both Mason and others and Steinberg suggest that foreign aid contributed only slightly to the Korean miracle and that rapid growth was due primarily to a combination of domestic factors and a favorable international environment. They agree that foreign aid before and after the Korean war was "essential to the survival of South Korea as an inde- pendent country" and that it added perhaps 1.5 percent a year to GDP during 1953-63 (Mason and others 1970, 203). Nevertheless, because Rhee's policies maximized the need for aid, U.S. aid during that period-only 15 percent of which went for investment goods- provided support for unsustainable policies and so had no permanent value. Mason and others (1970) argue that aid did little to bring about significant social and political change. The aid was also was an impediment to the mobilization of domestic savings and government revenues. In their view, attempts by the aid donors-principally the United States in the 1950s and early 1960s-to force the Korean government to give greater emphasis to stability (instead of or along with growth) were ineffectual and probably mis- guided in that they diverted attention fronm the real impediments to growth. Finally, they argued that, even once the country began to grow rapidly after 1962, the notion that rapid economic growth would lead to a more open and democratic society " proved unrealistic.'" Aid to Korea in a Holistic, Long-Term Development Framework With U.S. encouragement, the Korean government instituted long-run economic planning beginning in 1953. Korea initially relied heavily on foreign advisors but gradually took over leadership of the planning process. After several earlier attempts. the government was able in 1966 to put forward a five-year economic plan that was generally regarded as economically coherent and supported by government policymakers. Even so, the plan, which called for GDP growth of 7 percent per year, was criticized by the World Bank, which offered "continuing admonitions" that the target growth rate should be held to 6 percent (Cole and Lyman 1971. 219). Yet the central feature of Korean development planning during its high-growth period was not its comprehensiveness, but its focus. The government believed that rapid growth in exports was critical, so this became the focus of economic policies. Krueger (1979, 85) 4 REPIJBLIC OF KOREA: THE MAKING OF A MIRACLE writes that " The details of policies that were adopted appear to have resulted in large part in pragmatic response to the fortunes of exports: when export performance was deemed satisfactory, policies were left unaltered: when, however, it appeared that export growth was faltering. changes were instituted until satisfactory performance was again observed." A telling statistic comes from the Jones and SaKong (1980) survey of Korean entrepre- neurs that compared administrative decisionmaking under Rhee and Park. Entrepreneurs indicated that under Rhee. decisions were "always implemented" 3.2 percent of the time, and " almost always implemented " 17.2 percent of the time, whereas under Park they were always implemented 78.2 percent and almost always implemented 16.6 percent of the time. This comparison suggests that the seriousness of purpose of the government in imple- menting its development planning may be more important than the quality or comprehen- siveness of the plan itself. Ownership Ownership of the country's policy framework was unambiguously Korean for most of the period of U.S. assistance. The Korean position ultimately prevailed whenever there was a disagreement with the United States. which was frequently the case. Neither side was able to walk away from the other. At least until 1965 Korea's economic stability depended on continued U.S. aid of $200 million or so a year. The U.S. political commitment, particu- larly after the Korean war, precluded any unilateral withdrawal from Korea or the taking of any large risks on Korea's political stability. This mutual dependence did not bring harmony. The U.S. and Rhee administrations were usually at odds, with the United States pressing for devaluation of the currency and other orthodox policy measures and " the Rhee Administration staunchly opposed [to] the U.S. policy package" (Amsden 1989, 41). Because the Rhee Administration had the final say on policy and, in the characterization of one economist, sought to maximize aid flows, it perpetuated policies that made high levels of aid necessary. Foremost among them was the maintenance of an overvalued exchange rate. Tensions did not lessen with the arrival of the Park Administration. "By the early 1960s U.S. officials had become extremely gloomy about the prospects for Korean development" (Mason and others 1970, 195). The United States also saw the country as stable enough that high levels of aid were no longer thought to be required to prevent collapse and began to discuss the phase-out of U.S. economic assistance. This effectively increased U.S. leverage: The years 1963-64 witnessed some of the harshest bargaining over aid between the United States and Korean officials of the whole post-war period. Faced with severe food shortages, rising prices, and dwindling foreign-exchange reserves, the Koreans were in a very vulnerable position. The U.S. insisted on resumption of the stabilization program in 1963, calling for curtailing the budget deficit and limit- ing growth of the money supply to 5 percent. The following year similar restric- tions plus a 50 percent devaluation were imposed as conditions for aid. While the Korean government acquiesced to these demands in order to assure an adequate grain supply for the coming months, they also began a realignment of policies and international relationships that would save them from ever being trapped in such a compromising position again. (Mason and others 1970. 196-97) 5 APPLYING THE COMPREHENSIVE DEVELOPMENT FRAMEWORK TO USAID EXPERIENCES Thus the ability of the U.S. government to force policy change had increased by the early 1960s, and withdrawal of aid was a credible possibility. Steinberg (1985, 88-89) ignores this issue when he states that "Korea demonstrates that policy dialogue is useful and important over time, even if its influence cannot normally be quantified, but that it will not produce results unless it somehow furthers the overall direction that the state is taking and is regarded as being in the interest of those in power." It is possible that aid can be a more powerful vehicle for self-reliant growth in cases where the donor demonstrates the capacity to force decisions on the government. A committed government in this case will undertake the actions necessary to reduce its vulnerability to control by foreigners. For Korea, this would come through higher exports. Partnership At least until the early 1960s, USAID was the only significant donor in Korea, so issues of donor coordination did not arise. The aid relationship was directly between the U.S. gov- ernment and the Korean government, apparently with some participation by nongovern- mental organizations. Results Orientation Korea's export strategy is one of the clearest cases of successful results orientation in the developing world. In addition to providing the means for measuring trends in exports, Korea's approach also focused on steadily and systematically improving the policy and institutional climate for exports. Korea established ambitious export goals, including firm-by-firm export targets. Monthly meetings of the president with major exporters and representatives of government agencies provided a regular forum for monitoring progress and identifying problems. Collection of current data on exports was given considerable attention. Independent checks on exports to the United States were provided by the consulates on the West Coast, which sent weekly cables reporting on arrivals. In the early days companies that failed to meet the monthly export target could point to obstacles, such as failure by customs to clear a needed input, or delays in obtaining credit, or some shipping problem. The systematic review of such impediments and the elimination of many of them made exporting steadily easier, thus allowing the rapid growth of exports to continue unabated. The feedback loops thus established probably created external economies for exporters from Korea (Fox 1990). 6 Smallpox Eradication: Foreign Aid's Unequivocal Accomplishment Smallpox has been one of the deadliest diseases to plague humanity. Smallpox fatalities numbered in the billions, including an estimated 300 million in the 20"h century (Hopkins 1989). As recently as the late 1960s there were 12-15 million cases of smallpox a year and more than 2 million deaths (Barquet and Domingo 1997. 637). It is likely that nearly all of the deaths from smallpox in the second half of the 20th century were of poor people. since the well-to-do have long had the means to protect themselves. A decade later, not a single person anywhere in the world was in danger from the disease. Unlike with most donor activities, the source of this achievement is clear and unquestionable. Donors did it. A program funded by USAID in West Africa and carried out primarily by the U.S. Centers for Disease Control (CDC) developed the techniques that made rapid elimination pos- sible. They were subsequently used by the World Health Organization (WHO) to elimi- nate smallpox from Asia and the rest of Africa. The basic mechanism for preventing smallpox through vaccination was discovered more than 200 years ago by Edward Jenner. Routine vaccination managed by national health authorities spread rapidly in more advanced countries and virtually eliminated the disease in those countries by about 1950. Mass vaccination campaigns were also standard practice in poor countries, but the inability to reach remote rural populations and the wasted inocu- lations because of spoiled or ineffective vaccine meant that the disease continued to persist in parts of these countries, leading to large. periodic outbreaks. Thus in 1967 the disease was still endemic in more than 30 developing countries, containing more than half of the world's population. Conquest of smallpox was not a result of major breakthroughs in technology or vastly greater health spending. Rather. it resulted from a commitment to action and to learning by donors. This led to smarter spending by donor agencies, beginning with a new ap- proach developed during the West Africa eradication campaign that began in 1967. Prompted by the continuing threat to developed countries posed by smallpox's persistence elsewhere, health researchers in WHO and CDC began to explore the feasibility of global elimination.' In response to a proposal by the Russian delegate to the WHO assembly in 1958, WHO produced a statement endorsing elimination. Although vague. the statement did stimulate a search for ways this might be done. In 1961 the CDC established a smallpox unit for this purpose. The unit identified a vaccine that would remain stable and potent under tropical conditions, determined the dosage level, and searched for an appropriate injection device. Starting with the jet injector, a mass injection device developed by the U.S. military, the researchers first adapted it to the smallpox vaccine (testing it in 1963) and then eliminated the need for electricity to power the device (testing a foot-powered injector in 1965). 7 APPLYING THE COMPREHENSIVE DEVELOPMENT FRAMEWORK TO USAID EXPERIENCES The CDC then pressed for U.S. government endorsement of smallpox eradication, which the U.S. president approved in 1965. The United States and the Soviet Unionjointly spon- sored a resolution at the 1966 WHO assembly calling for a multilateral campaign to begin in 1967 and to eradicate the disease within 10 years. Though the goal was established, one participant noted that "It is clear, in retrospect, that we didn't know how to eradicate smallpox when the eradication effort began" (Foege 1998, 412). The CDC had also been providing technical assistance to USAID on a West Africa measles campaign. When USAID asked it to lead a major expansion of the program in 1965, the CDC proposed including smallpox eradication as well. CDC and USAID spent the next year generating support among the 18 West and Central African countries that were prospective hosts for the program and working out collaboration with WHO. WHO agreed to provide gasoline and meet some other local costs of the campaign and to man- age the program in countries where USAID was not active. USAID provided foreign ad- visers and vehicles and met other foreign exchange costs. The hope was that the mass vaccination program could interrupt transmission by vaccinating enough people in re- mote areas. WHO, once guided by the rule of thumb that the disease would die out after 80 percent of the population had been vaccinated, was moving toward advocacy of a 100 percent vaccination rate in the 1960s. The early results of the West Africa campaign seemed to confirm this. The campaign ultimately vaccinated more than 150 million people-a number larger than the total population of the region-yet almost total vaccination did not seem to be preventing continued transmission. The conceptual breakthrough that ultimately led to rapid eradication came from a logistics failure in eastern Nigeria. Facing a shortage of vaccine and a civil war that com- plicated re-supply, the teams suspended mass vaccination and used the limited supplv to vaccinate individuals living near people suffering from smallpox. Radio reports from mis- sionaries were used to identify outbreaks. Over the next year. this approach, along with the knowledge that outbreaks were strongly seasonal. became the basis for a major revi- sion in technique. First tested in Sierra Leone, where complete eradication was achieved in nine months, the technique was then used elsewhere in West Africa with similar results. The last case in the program area came in May 1970-less than three and a half years after the effort began. The new approach depended on quick identification of new smallpox cases and rapid deployment of a vaccination team. Good communication of outbreaks was important, and vaccination teams had to be restructured to move quickly. WHO began to offer rewards for identification of cases-relatively easy for smallpox. The intensive approach worked, and the last naturally occurring case of smallpox in the world was diagnosed in Somalia in May 1977-10 years and 4 months after the eradication effort began. Successful eradication has not yet been repeated for other diseases. Why was the small- pox campaign so successful while the malaria campaign was not? USAID and WHO ex- pended far more resources and effort on malaria than on smallpox. WHO (1980). in its assessment of smallpox eradication, concluded that the malaria effort included too little research into the scientific aspects of the disease and its transmission. The smallpox effort included research into various aspects of the disease, leading to new knowledge and changes in approach over time. The malaria effort concentrated entirely on a single approach- 8 SMALLPOX ERADICATION: FOREIGN AIm's UNEQUIVOCAL ACCOMPLISHMENT spraying with DDT-to achieve the goal. Indeed, the leaders in the malaria effort had a long history of hostility to other approaches (WHO 1980, 382). When mosquitoes immune to DDT became dominant, there was a resurgence of the disease. In many countries the malaria eradication group developed a strong esprit d'corps and strong local ownership of the technology provided by the donor community. The problem was in the failure to seek more knowledge about the nature of the disease while the eradication campaign was under way. Smallpox in a Holistic, Long-Term Development Plan Coverage of smallpox in a country's long-term development strategy involved two key problems. First, a comprehensive development framework in any of the 30 or so countries where smallpox was still endemic in the mid-i 960s would have included it as only one among many diseases needing attention. It would have been mainly a poverty issue. Middle- and upper-income children are typically inoculated, so the disease is one that affects pri- marily the poor. The health problem posed by smallpox would not stand out among the numerous other severe health problems facing poor countries. Thus it was not the magni- tude of the problem that warranted a concerted attack on the disease but the belief that such an effort could eliminate it, promising a high pay-off from donor funds. Second, smallpox was more a global than a national concern. Without global eradica- tion, each country would have to continue to spend considerable resources on inoculation and surveillance. Thus the disease warranted a global approach, and a lack of interest at the national level would need to be addressed by multilateral efforts. Ownership The original concept for eradication of smallpox was developed primarily by donor coun- tries and marketed to developing countries. Because the disease crosses national bound- aries, its elimination could not be achieved by working only with governments that were supportive of the program. The WHO mandate provided a basis for claiming that all countries participated in the decision to eradicate the disease, but surely much of this participation was formalistic. Nevertheless, little opposition to elimination would be ex- pected for a disease like smallpox. As for West Africa's involvement, the governments had initially sought a measles vacci- nation program from USAID, which USAID then asked the CDC to implement. The CDC's proposal to include smallpox eradication reflected its priorities rather than those of the West African countries. Even where there was strong local commitment by health officials. that did not neces- sarily translate into government willingness to act in a timely fashion. In Nigeria the quick visit by the CDC representative to formalize Nigerian participation turned into a six- week stay as the health minister, the WHO representative, and the U.S. ambassador tried to obtain the signature of the head of state. Finally, it was an introduction to the head of state's fiancee, arranged through the wife of a friend. that made a meeting possible, and the signature was obtained (Ogden 1987. 35). 9 APPLYING THE, COMPREHENSIVE DEVELOPMENT FRAMIEWORK TO USAID EXPERIENCES Partnership The smallpox initiative is one of the most successfu] cases of partnership among donc,rs. The West Africa initiative was primarily a USAID creation, but WHO provided the politi- cal impetus and collaborated closely with the CDC on implementation. The smallpox initiative was not one of equality among participants, of donor coordination in which each party makes some adjustments to programs so as to conform to a common approach. Rather. it was a case of a convinced group of intellectual and technological leaders- mostly veterans of the West Africa campaign-promoting their plans to other donors and the health leadership in affected countries. Results Orientation Measuring results proved more difficult than expected. Still. a focus on results was crucial to the success of the smallpox effort, particularly in four areas. What Data?The first problem was that implementation of the surveillance and contain- ment approach nearly always led to sharp increases in the number of reported cases. The final WHO report (1980, 476) estimates that the actual number of smallpox cases in 1967 was probably 12-15 million rather than the officially reported 15,000 or less. More careful monitoring of the disease had resulted in large increases in the number of reported cases. Which Results? Until the late 1960s most vaccination campaigns did little to ensure that the vaccine was potent or that inoculations were done correctly. Results were mea- sured by number of vaccinations. Through follow-up surveys and laboratory tests, the CDC was able to establish that large numbers of people were missed in villages where coverage was thought to be complete and that vaccinations sometimes did not "take" because of bad vaccine or poor technique. Through consistent attention to quality control. the campaign became steadily more effective over time. Maintaining Focus. Even after the surveillance and containment approach had proved effective in eliminating smallpox in several countries, the results were not always evident to local health officials. The WHO effort in eastern India faced a crisis in May 1974. after 11,000 new cases of smallpox were detected in the state of Bihar. The Bihar government and health officials pressed WHO to abandon surveillance and containment in favor of mass immunization. WHO was able to maintain its focus only through strenuous debate and by seeking a delay before changing approach (Ogden 1987, 103: Shurkin 1979, 330- 31). Just one year later India recorded its last case of smallpox. Using Economic Rates of Return. The final evaluation of the West Africa smallpox and measles program concluded that the project was a success: it had eliminated smallpox from the region and sharply reduced the incidence of measles. Using a crude methodology. the evaluators estimated the internal rate of return at 10 percent for the smallpox compo- nent and 26 percent for the measles component. The evaluators compared these estimates to much higher estimates of the rate of return to education in Nigeria and suggested that the project may not have been wise compared with alternative uses of the resources. Taken at face value, the evaluation would suggest that further support for smallpox eradication would be undesirable. Viewed in retrospect, the finding of a low rate of return was erroneous. The evaluators did not identify the new surveillance and containment approach as a significant result of the 10 SMALL POX ERADICATION: FOREIGN AID'S UNEQUIVOCAL ACCOMPLISHIMENT project. Only later was this new approach seen to be a key to rapid elimination of smallpox worldwide. In addition. the evaluators assumed that mass inoculation against smallpox would continue in West Africa. so that no benefits were assumed to flow from cost reduc- tions in this area. Only total elimination of smallpox from the world could eliminate this cost. Ultimately, the successful 10-year eradication program cost an estimated $300 mil- lion--$98 million from donors and $200 million from developing countries. The annual cost of smallpox to the world was estimated in 1967 as $1.35 billion a year-including the cost of inoculations in nonendemic areas. surveillance at border crossings, and deaths from inoculations (WHO 1980, 1364). Using these data yields an internal rate of return of 40 percent. In sum, the smallpox case strongly underlines the importance of a results orientation. It also illustrates the difficulty of measuring progress. The true indicators of the underlying phenomena would show that real progress was being made and that eradication would be a high payoff activity for donors. Yet if USAID leadership had maintained a results orien- tation based on the data available to them-the increase in reported numbers of smallpox cases and the low internal rate of return for the West Africa project-they would have made wrong decisions. The quantitative indicators and the monitoring of results were essential tools for the practitioners in this activity. In the hands of the overseers, however, they might have led to mistaken decisions. 11 Family Planning: From Donor Unilateralism to Local Ownership USAID has been the largest bilateral provider of international assistance for family planning, providing grants of more than $6 billion since USAID programs in this area began in 1965. USAID activities initially focused almost exclusively on the distribution of contraceptives, but programs gradually became more varied and diverse. By the mid- 1990s USAID was operating family planning programs in 77 countries and was providing indirect funding through nongovernmental organizations (NGOs) or the United Nations Fund for Population Activities (UNFPA) to still others. These activities have been associated with dramatically increased international discus- sion of population issues and heightened concern about the consequences of continued population growth. Fertility has declined substantially in developing countries, from more than 6 children per woman in the early 1960s to 3.6 children in the late 1990s. The early USAID approach identified lack of access to contraceptives as the major reason for high fertility, so delivery of contraceptives was emphasized. Over time, pro- gram approaches became much more nuanced and country focused. A variety of research, policy discussion, and data gathering approaches were used. and a variety of public and private organizations participated as collaborators. The data gathering, including demo- graphic and health surveys, has been an important source of country-specific information on fertility practices and other health and social issues. Two controversies about family planning programs have never been settled definitively. One concerns the impact of high rates of population growth on development and average living standards. Though the negative impact of faster population growth on per capita income seems self-evident to many. statistical analyses-whether cross-sectional or time- series-have generally yielded ambiguous results. A major review of the evidence (Na- tional Research Council 1986) failed to find any significant relationship. A second controversy relates to the impact of family planning programs themselves. Increased use of contraception to reduce fertility is characteristic of modernizing societies. Some of the most striking cases of sharp fertility decline occurred before organized family planning efforts existed (Gomez 1995 shows this for Costa Rica). and some analysts have used cross-sectional regressions to question whether official family planning programs provide any significant additional increment to fertility reduction (Schultz 1993). Family planning supporters counter with analyses that support a positive role. Bongaarts, Mauldin, and Phillips (1990) estimates that family planning programs reduced world population in 1992 by more than 400 million people from what it would otherwise have been-roughly equal to the reduction in world population in 1995 between the 1968 medium-variant projection by the United Nations in and its 1996 estimate of the actual population.2 13 APPLYING THE COMPREHENSIVE DEVELOPMENT FRAMEWORK TO USAID EXPERIENCES USAID's most extensive evaluation offers a generally positive assessment of the pro- gram, suggesting that program effectiveness has improved over time as better. and more country-focused. techniques have been developed (USAID 1997). It identifies both t:he measuring of results for internal management and a focus on broad measures of effective- ness as important to effective implementation. Family Planning in a Holistic, Long-Term Development Framework In recent years many developing countries have explicitly established population growth goals as part of their long-term development framework. USAID has long used a quantita- tive model (called RAPID) incorporating visual images to dramatize the consequences of alternative population growth rates for such national concerns as school construction, food production. and job creation. The model has frequently been used to encourage govern- ments to adopt new population policies by demonstrating the severe negative consequences of continued rapid population growth. Still, other models could be used to produce differ- ent results, and the ultimate consequences of different rates of growth of population for long-term national welfare have never been settled. Some governments-notably China and India during the 1970s-have used coercive measures to affect long-term national population levels. Many others have chosen indirect means, such as lowering the costs or raising the benefits to families of reducing the number of children. The developmental case for using public resources to ensure that all families have access to modern means of contraception is much stronger than the case for anv specific national fertility target. Ownership USAID involvement in family planning was initially driven more by U.S. concerns about world population growth than by developing country demand for aid in this area. Partly for this reason, U.S. assistance for population has always taken the form of grants, while most other development activities have been partly loan-financed. The expectation was that grants would be subject to less public scrutiny, preventing opponents from mobilizing to oppose the assistance. When USAID program activity began in 1965, only nine developing countries had poli- cies favoring slower population growth and many were officially opposed to such pro- grams. USAID efforts probably helped change the climate of opinion so that by the mid-1990s some 70 countries had adopted such policies (USAID 1997, 3). USAID would identify like-minded people in the government or the private sector and encourage them to mobilize public opinion in favor of the delivery of family planning services. Sometimes USAID would invite health officials to international conferences on population growth. As a result, most USAID family planning activities have created a strong domestic constituency capable of continuing programs in the absence of USAID funding. 14 FAMILY PLANNING: FROM\1 DONOR UNILATERALISM TO LOCAI OWNERSHIP Partnership Family planning is perhaps the most successful case of partnership among bilateral, mul- tilateral, and NGO providers. NGOs such as the International Planned Parenthood Foun- dation (IPPF) and the Pathfinder Foundation were the first to provide family planning assistance. USAID and UNFPA were the first bilateral and multilateral providers of such assistance, and they remain the largest official sources of such assistance. Over the de- cades they and NGOs have continued to work closely, sharing information and research and frequently conducting joint activities. Their shared outlook is particularly striking and helps to explain their easy working relationship. They agree that population growth is a threat to human welfare, that empow- erment of women is especially important, and that control over fertility is a key factor in women's empowerment. This shared vision means that USAID family planning profession- als may share closer ties with like-minded organizations outside of USAID than with other parts of USAID. Results Orientation A results orientation appears to distinguish successful from unsuccessful USAID family planning projects (USAID 1997, 31). At the operational level, managerial systems are needed to ensure that commodities and training are used efficiently to achieve the desired results. The availability of contraceptives to end users at all times is critical to program success. This requires good controls on the flow of products. Demographic and health surveys and censuses have also been crucial for steering pro- grams in the right direction. Information about a program's failure to increase contracep- tive prevalence or to reduce fertility or the discovery that large numbers of families are receiving their contraceptives outside of official programs has helped guide experimenta- tion wvith new approaches and brought to an end programs that failed to achieve the desired results. 15 Agriculture: Technology Matters Enormously A gricultural research is a recent phenomenon in the developing world. It has paid A huge dividends, increasing agricultural yields more in the past 50 years than in the previous 1,000 years. As a consequence, humanity is gradually being liberated from scratch- ing a meager living from backbreaking work on the land. Throughout most of history. growing enough food meant that some 80 to 90 percent of the population worked in agriculture. Today, in the United States, each farmer feeds 77 people. In the early years, USAID professionals believed that improving agriculture was largely a matter of transferring existing knowledge to farmers in poor countries. Agricultural extension services were created along the U.S. model, staffed by local agronomists with the support of U.S. advisers with experience in the U.S. extension service. Only gradually did it become clear that agricultural technology is crop and region specific and that improving agricultural technology in developing countries was far harder than imagined (box 1). USAID has done much to promote agricultural research at the national level. It has also worked with other donors in an international agricultural research effort under the Consul- tative Group on International Agricultural Research (CGIAR), which coordinates the work of 14 international centers for particular crops or climates. The first two of these centers, in the Philippines for rice and in Mexico for corn, produced varieties that allowed dramatic increases in productivity. Box 1. Effective Agricultural Technology Development One of the first efforts lems farmers faced. worked Hayami and Ruttan (1971) tial to convincing practi- of' the USAID evaluation on problems of little inter- were able to demonstrate tioners that their ap- of'fice was to analyze est. The role of extension how critical agricultural proach is producing poor USAID agricultural pro- agent as intermediary be- extension had been to the results. Successful expe- grams in Latin America. tween researchers and farm- development of U.S. and rience in other places or The study showed that the ers, a key feature in the Japanese agriculture-and at other times can pro- core of the program, agri- United States, was absent in by extension to any coun- vide ideas for more cultural extension, was a Latin America. try that desired technical promising approacthes. failure because the exten- This finding could have progress in agriculture. In Following these findings, sion agents had little to of- been used to stop funding this case, as in many oth- USAID and Latin Ameri- fer the farmers. of agricultural research or ers, program evaluation can governments experi- Agricultural research extension, for such evalua- and economic, research can mented with alternative was also of little use be- tions are better at finding closely complement each approaches to bring cause researchers. lacking problems than identifying other. Dissemination of farmers and researchers means to learn what prob- solutions. The same year, evaluation results is essen- closer together. 17 APPLYING THE COMPREHENSIVE DEVELOPMENT FRAMEWORK TO USAID EXPERIENCES Regarding the value of agricultural research. USAID's most recent program evaluation concludes: A single finding from the literature overwhelms all others: investments in agricul- tural research have generated high economic rates of return. indicating that the social benefits of the investments justify the costs in virtually all countries, for a wide variety of commodities, and under diverse agronomic and climatic condi- tions. (USAID 1996,15) Studies of the impact of agricultural research of this type have consistently shown very high rates of return. For 97 studies of the effects of agricultural research in developing countries, the median rate of return was 52 percent. Fifteen of the studies (most often of irrigated rice) found rates of return above 100 percent. The reason that private firms do not do much of this research is that many of the benefits of the research are scientific and cannot be patented or captured by the inventor. Consequently, agricultural research in such cases is inherently a government-financed activity. The 1996 assessment offers an important caveat to the generalization that research always works: it notes that a favorable policy climate for agriculture is a precondition for effective agricultural research or infrastructure development. ' If a threshold level of proper policies is not in place, it is seldom worthwhile for donors to make other investments in agriculture: nor is it worthwhile for farmers to take risks and use new technologies to increase production beyond subsistence levels" (USAID 1996, vii). India in the 1960s illustrates the close links between policy and effective use of new. technology.3 In 1966-67 India was suffering from its second consecutive bad monsoon season. A death toll from famine of as high as 50 million Indians was considered possible in the absence of international help. To prevent mass starvation, the United States and other donors mobilized enormous amounts of aid, enabling India to import 21 million tons of food. The United States provided most of the food, with President Lyndon Johnson playing a large personal role. Johnson believed that Indian policies were harming its agriculture. and he suspended all U.S. aid to India at the height of the crisis. Only after the Indian secretary of agriculture signed a secret agreement with the U.S. secretary of agriculture was Johnson willing to renew shipments. Even then, they were provided on a month-by- month basis, linked to progress on the agreed agricultural policy changes. These included an immediate 40 percent increase in agricultural investment by the Indian government. increased agricultural prices, new credit policies, and increased availability of fertilizer and of new rice and corn varieties. With the reforms, the green revolution came quickly to India. India required no massive emergency food aid program in 1978-79, when a drought similar to that of 1965-66 occurred. Subsequent droughts have been managed without international notice, as a nor- mal fluctuation in weather, to be met by drawing down food stocks (India's grain stocks are the largest in the world). Wheat production has quintupled: rice production has more than doubled. These increases occurred with little expansion of the total area planted to these crops. India is now a net exporter of cereals. and the Indian people are able to eat better than ever before. 18 AGRICULTURE:TECHNOLOGY MATTERS ENORMOUSLY Agricultural Research in a Holistic, Long-Term Development Framework The place of agricultural technology development in a holistic development framework is not obvious. Such a framework is unlikely to be very useful in determining the level of resources that society should spend on agricultural research or even the areas in which the research should be concentrated. Rather, the magnitude and direction of such effort are likely to be determined incrementally, responding to experience over time. Continuity of effort and emphasis on quality and relevance are probably the elements most likely to produce the greatest results. Yet, as with research in other areas, there is no necessary relationship between input and output. Some activities will yield no useful results, while others will pay huge dividends. USAID's 1982 assessment of its agricultural research mentions a trend toward " use of multidisciplinary teams and a more holistic approach to research" (USAID 1982, 1). The study does not characterize the results of the more holistic approach. The next major assessment of USAID agricultural research experience (USAID 1996) does not mention such approaches and suggests that farmers are willing to adopt new technologies that are accessible as long as they offer better risk-adjusted profit prospects. Ownership Given the close linkage between farming systems and the usefulness of agricultural re- search in particular countries or regions, the need for close interaction between farmers and researchers is clear. This cannot happen without extensive local participation. Partnership The CGIAR is surely a case of successful donor partnership. The financial cost of the system of international research centers is too large for any single donor to bear, while the alternative of individual international centers run by different donors is unpromising. Re- search might come to reflect the latest priorities in the donor country's research establish- ment more than those of developing countries. The partnership needed in this case is quite different from the coordinated work under a common vision described in the smallpox and family planning cases. The international centers, rather than the donors. should be deciding the research agendas and approaches. The critical issues for the partners are to ensure appropriate burden-sharing among donors and a high quality of leadership for the individual centers. Results Orientation Because agricultural research is unlikely to result in gains for agriculture in policy envi- ronments that are biased against agriculture and where farmers are unable to profit from the use of better technology, donors should avoid investments in agricultural research in countries with unfavorable policies. 19 APPLYING THE COMPREHENSIVE DEVELOPMENT FRAMEWORK TO USAID EXPERIENCES Evaluations have been less clear about t.he conditions for achieving results from re- search itself. Two elements would seem to be especially relevant. First, research needs to be of high quality and the research institution needs to monitor performance. Second. since research only pays off when linked to production. communication between researchers and users is essential. 20 Foreign Academic Training: Producing New Leaders Donors have long used advanced academic training (usually in the universities of the donor country) as a means of promoting development in poor countries. Such pro- grams, called "participant training" by USAID, have always been a significant feature of U.S. development assistance. Most USAID professionals regard these programs as one of the most successful areas of USAID activity, an assessment generally based on the observation that earlier participants have moved into key positions in government. Former participants are often the most effective interlocutors between the foreign aid agency and the host government. Since U.S. foreign aid began in 1949. it has funded participant training to perhaps 350,000 people from developing countries.4 A review of USAID experience conducted in 1986 found more than 200 evaluations of such projects. Nearly all were concerned pri- marily with operational issues relating to training, and none of them provided a basis for assessing the ultimate benefit of the training to the developing country (USAID 1986a, xi). Most USAID evaluations assess the success of overseas training by looking at whether the participant successfully completed the planned study program, returned home to work, and was satisfied with the training experience. By these criteria, most USAID training projects were successful-in some cases, strikingly so. Muscat (1990, 56) looked at the educational backgrounds of 411 senior officials of the Thai government and found that 162 of them, or nearly 40 percent, had been trained under the USAID pro- gram. There are obvious reasons for the popularity of such training among donor govern- ments, their universities, and participants. The donor country benefits from the partici- pants' presumed favorable attitude toward the country. universities prefer students whose costs are fully paid by others. and participants enjoy a huge educational subsidy. Never- theless, this enthusiasm conceals a certain anxiety about the program's real effectiveness. Did a particular individual become minister because of his U.S. education or his natural talent? Although participants are prominent among government elites, they are still a minority. Did USAID choose the right people? From an economist's perspective, were par- ticipants selected because their education would yield the greatest " social value-added" or because they had already absorbed the cultural and educational values that would make them good students and future leaders? Answering these questions requires an experimental design for participant training that would compare promising students selected for training with a similar group of students who were not trained and with unpromising groups of students, both trained and untrained. After perhaps 40 years of observation, the careers of members of the four groups could be com- 21 APPLYING THiE COMPREHENSIVE DEVELOPMENT FRAME\VORK TO USAID EXPERIENCES pared in terms of social usefulness, and the real effect of participant training could be scien- tifically determined. Clearly, it is unlikely that any donor would undertake the experiment. At the country level Valdes (1995) describes an interesting case study of the experience of Chilean economists trained at the University of Chicago under a USAID project in the late 1950s and early 1960s. These economists later presided over the development of the " Chil- ean model" of development, important aspects of which were adopted by other Latin Ameri- can governments in the 1990s. This project had several key features. First, participants' academic mentors were of unusual stature: three (Milton Friedman, George Stigler, and Theodore Schultz) later won Nobel prizes. Second, the program's goal was not to train economic policymakers but to develop a strong academic economics department at the Catholic University of Chile. Like their U.S. mentors, the Chilean participants (later re- ferred to as the "Chicago boys") expected their influence to come through the quality of their economics research and teaching. It was only in the mid-1970s that an accident of politics thrust them into a policymaking role. It would be interesting to calculate the social value added for the welfare of Latin Americans of the influence of this group of economists on policies in the region and to compare it with the value added that would have resulted had they remained in academia as expected. (Valdes does not address this issue.) Foreign Academic Training in a Holistic, Long-Term Development Framework Foreign academic training seems to belong more to an earlier era, when some believed that satisfactory long-term development would require predictable numbers of people with particular academic specializations. Given the long lags between education and full pro- fessional competence, planners would program training funding to ensure that the country would have, say, the right number of mechanical engineers 20 years later. Such ideas no longer hold sway, as experience has suggested that neither the demand nor the supply side can be projected with any confidence. Both exhibit remarkable flexibility in the presence of free labor markets. Consequently. there is no easy answer to the question of how much foreign academic training a country should have. Rather, a future-oriented coun- try might want to provide some general subsidy for foreign (and domestic) academic training in the expectation that it would pay dividends in the long term for society. Ownership Peculiar features of training projects give them some self-enforcing mechanisms for encour- aging ownership and effectiveness. Training is typically very demanding for the individual concerned, usually requiring learning of a new language and academic performance at a high level. Failure would be harmful to the program and the individual. And the substan- tial subsidy provided by such programs, and the obvious private benefits to people who complete them successfully, make it easy to find promising candidates. Country ownership is also apparent in the lives of returned trainees, since the donor no longer has control over how participants use their skills (beyond the typical requirement that participants must return to their native country or pay for the training). Trainees have been empowered by the donor to use their skills as they see fit. 22 FOREIGN ACADENMIC TRAINING: PRODUCING NEW LEADERS Partnership USAID's participant training programs have been unilateral and have involved no coordi- nation with other donors. Activities of other donors appear to be similar, since programs specialize in training in the donor country. Results Orientation Training programs appear to have the main features of a results oriented approach at the output level, as reflected in the three features mentioned above: selection of suitable candi- dates, monitoring of their return to the country following training, and interviews with returned trainees following completion of training. 23 U.S. Assistance to Egypt: Eventual Progress? A mong USAID professionals, the aid programs in Egypt and Israel have long had the reputation of being the least effective in development terms. Israel is a special case, since there has been little pretense of expected development impact from U.S. assistance. In Egypt, however, USAID has always maintained a large presence, including several hun- dred U.S. employees and contractors. The program combines project and sector program and balance of payments assistance, either through concessional sales of agricultural com- modities or through a general commodity import program. Altogether, the United States has provided Egypt with more than $25 billion in economic assistance since U.S. aid resumed in 1975 (more than double that in 1999 dollars). Despite all the assistance, Egypt has not performed particularly well. After rapid eco- nomic growth during 1976-82. fueled by high oil prices and the reopening of the Suez Canal, Egypt went through a long period of economic stagnation. In the early 1990s the government undertook significant economic reforms, including liberalization of the ex- change rate and control of the fiscal deficit, resulting in relatively rapid growth since 1995. While USAID has not done any long-term assessment of the impact of its aid to Egypt, other studies have concluded that the aid had little impact. Sullivan (1996) attributes the ineffectiveness to lack of a cohesive growth-oriented approach on the Egyptian side and to the inability of USAID-because it was overruled by the State Department-to demand performance. USAID evaluations of particular projects show the usual mix of success and failure, but those dealing with policy issues are almost uniformly negative until the early 1990s. A 1981 General Accounting Office (GAO 1981) report concluded that USAID's agricultural project in Egypt had produced little. A 1983 USAID assessment of the large program to provide agricultural commodities on concessional terms concluded that the program had had a significant disincentive effect on Egyptian agricultural production and had reinforced already heavy government control over food distribution. And a 1994 evalu- ation of USAID funding of investment projects concluded that the investments had yielded a low rate of return because of the distortions in the Egyptian policy environment. Overall, the project evaluations for Egypt (Montrie and Diamond 1999 contains summa- ries of more than 200 USAID projects) conclude that social sector projects have been rela- tively successful while economic projects have generally fared poorly, because of a lack of scope for market forces to operate or for the Egyptian government to follow through on promised liberalization or privatization. Some improvement in performance has been evi- dent in recent years, beginning with the agricultural sector in the late 1980s and followed by macroeconomic policies, which improved after a successful IMF-World Bank stabilization and adjustment program. These changes are reflected in more successful USAID projects. 25 APPLYING THE COMPREHENSIVE DEVELOPMENT FRAMEWORK TO USAID EXPERIENCES Despite the slow growth of GDP and the negative assessments of the country's economic performance, broad measures of material well-being show a more favorable picture. Life expectancy has increased by about 12 years since 1975, and the share of the population with access to water and electricity has increased dramatically. The evidence that ordinary Egyptians live much better than they did in 1975 is quite compelling (Fox 1999). This dichotomy suggests that the link between per capita economic growth rates and the well- being of the mass of the population is less direct than assumed by economists. U.S. Assistance to Egypt in a Holistic, Long-Term Development Framework The Egyptian government has long used planning as a development tool and has sought to promote the country's development by organizing all sectors of society under government direction and ensuring that everyone participates in the benefits of modernization. This mobilization of society takes place through government-controlled institutions. Reports of various human rights groups, such as Freedom House, paint a picture of an all- pervasive government that leaves very little space for civil society. Ownership The Egyptian government had clear ownership of the programs carried out with U.S. funds. Indeed, this was the major source of frustration with the program for USAID profes- sionals. USAID would seek to condition assistance to progress on economic policy goals bv the Egyptian government. When the government failed to perform. USAID would be forced to renegotiate performance targets or simply to accept the shortfall.5 This case differed from the early Korean situation. There, the high levels of aid were critical to Korean stability and to U.S. foreign policy goals. In Egypt the government could have forgone U.S. assistance without a threat to the country's economic stability. Partnership For most of the period USAID has been the dominant donor in Egypt, and its programs typically did not involve much collaboration with other donors. Rather, it was the other donors who felt the need to take USAID programs into account in the design of their activities. Results Orientation From USAID's perspective the Egypt program has never been results oriented. Senior USAID officials have periodically sought to reallocate the Egyptian funds to other, more promising, development programs.' In a few projects some capacity to withhold or to reallocate funds probably influenced some attention to a results orientation. This may explain why social sector activities (in which USAID was largely autonomous) were more successful than activities relating to economic policy, in which State Department involve- ment was more likely. 26 U.S.ASSISTANCE TO EGYPT: EVENTUAL PROGRESS? The most extreme evidence of the absence of a results orientation is the approval of a project proposal containing an economic analysis showing a negative expected rate of return (USAID 1 989a). Surely this is unique in the annals of donor project proposals. (The proposal was for an electric power plant, an investment deemed necessary because of the underpricing of electricity in Egypt.) 27 Conclusions Several general conclusions flow from the case studies. First. USAID's decentralization of authority to the country level played a role in most of the cases. Mission manage- ment had the flexibility and authority to adapt programs to country conditions and to speak authoritatively about the conditions under which funds would flow. The typical USAID mission includes a large staff of U.S. employees and contractors, who can develop and approve smaller projects (within an overall country program) without seeking ap- proval from Washington on the design or implementation characteristics of the project. This flexibility was absent in the Egypt case. Important policy differences between USAID and the host government were invariably referred to Washington, which frequently over- rode country mission positions. Second, successful programs are not always easy to identify, particularly in the short term. In all of the successful cases, there were uncertainties about whether real progress was being made, yet in retrospect, success was evident. Still, even in the longer term, the contribution of aid to the ultimate outcome is completely unambiguous only in the case of the smallpox eradication program. In all the other cases the inability to be sure about what would have happened in the absence of aid (the counterfactual) meant that skeptics could always raise questions about the impact of the assistance. On the other hand, even in the two cases regarded as failures-aid to the Republic of Korea before 1963 and aid to Egypt-some important positive trends or results were evident. The complexity of the processes that foreign aid is meant to address means that there will always be some ambi- guity about results.7 Third. new knowledge was critical to the success of most of the cases. This suggests that the issue of technology should be recognized more explicitly in the CDF and in each of its elements: * Greater effectiveness can be expected from a holistic, long-term framework that incorpo- rates the most current knowledge about development. Korea's focus on exports succeeded because it rejected the export pessimism that had characterized most development plans in that era and incorporated newer empirical evidence on international trade, emulating recent Japanese experience. Egypt's development plans largely ignored this evidence, and the results were far less favorable. * Host country ownership that also focuses on adapting the best technology to country circumstances will get better results. In the family planning case, programs that contin- ued to experiment with newer birth control technologies often gained more new users than programs applying older approaches. 29 APPLYING I HE COMPREHENSIVE DEVELOPMENT FRAMEWORK TO USAID EXPERIENCES * Partnerships in which donors share research with each other and discuss new findings collegially are likely to improve the methods used by each partner. Family planning is a good example. * Activities with a results focus that continually scrutinizes the adequacy of the measure- ment techniques and the variables being measured will be more effective. The success of the smallpox program was due in no small part to changes in how results were moni- tored and success was measured and to a continuing search for better approaches. The failure of the much larger malaria effort was due partly to the failure to question the effectiveness of an approach that appeared to be successful and had produced great early results even as the real situation deteriorated. Holistic, Long-Term Development Framework The need for a long-term focus is evident from all of the cases. How to do it is less clear. In the Korean case development success was related to the government's long-term vision. But while the 1965 development plan offered a holistic vision, its strategy was narrowly focused. Achievement of rapid economic growth and economic independence was the top priority. Once export growth was achieved, the other elements of development success- social sector expenditures. increases in wages for workers, and broadening of the political system-would fall into place. Over the longer term, this belief has held true. The danger evident in an approach that advertises itself as holistic-particularly when wide public participation is invited into the planning process-is that it can become a "Christmas tree" that includes too much. If the effort to broaden the constituency for the approach means that difficult choices are not made, the program will fail for lack of coherence and focus. The development strategy also should not distract from investing in promising activities or targets of opportunity that seem likely to have high payoffs. In some of the successful cases, the impact came not because the activity rose to the top in some comprehensive framework. but because it was an excellent investment. Smallpox eradication was desir- able because it was possible at a reasonable cost. Foreign training of people from develop- ing countries pays off because they contribute, in ways that cannot be foreseen clearly, to the development of the country. Agricultural research is desirable because it reduces the real cost of agricultural production. freeing people to work on other things. A final concern relates to implementation. The difference in implementation between the Rhee and Park regimes in Korea is so striking as to suggest that this may have been as important as differences in the quality of their vision of the country's future.8 Ownership Host country ownership is an important factor, but the cases argue against too narrow an interpretation of ownership. Donors often equate government ownership with country own- ership. There is a need to recognize the diversity of perspectives within a country or its government and to ensure a sense of ownership among those involved in a project. In many cases governments embraced family planning as a national strategy only after pri- vate groups had changed the climate of opinion in the country. Had USAID waited until 30 CONCLUSIONS the government endorsed the concept. millions of people would have been denied access to modern family planning techniques for years or decades. Country ownership should not mean that donors accept country programs that are inadequate. Hard conditionality proved developmentally productive on two occasions in the case studies-Indian agriculture in the mid-1960s and Korean economic policy in 1962-63. The U.S. government pressed for reforms, and the government ultimately took decisive action that substantially improved the country's long-term situation. Partnership Partnership was important in three of the six cases considered: family planning. smallpox eradication, and agricultural research. In the first two it was a shared vision that was critical to success. When donors share a common view of the problem, partnership is easy. The danger in partnership approaches comes when donors have different perspectives on the problem or the solution. Then there is a risk that efforts to coordinate will mean that the most effective approaches are shunned in the search for common denominators. In the case of agricultural research, partnership took the form of mutual encouragement of ap- propriate burden sharing. Results Orientation A results orientation was the most important factor in project effectiveness in the cases studied. But the cases also suggest that a results orientation is more difficult to achieve in practice than in theory, for several reasons. First, the results that are most important for monitoring and evaluation may not be readily available, while those that are available may be misleading. The smallpox case shows that regularly reported data by WHO were wrong by a large margin. Implementers may need to develop new data sets for tracking actual performance. Second, people far from the scene of activity may not be able to monitor progress effectively. This suggests that decentralized management may produce better results. The complexity of the problems being addressed should also be a warning to avoid oversimpli- fication when measuring results. It may be that the development business shares the char- acteristics of what James Q. Wilson (1989, 158) calls a craft organization, in which the top layer of the organization has to rely heavily on the judgments of the professionals on specific issues. 31 Endnotes 1. This narrative is drawn mainly from Shurkin (1979), WHO (1980). and Ogden (1987). 2. This comes from comparing the 1968 and 1996 editions of the UN's World Population Prospects, adjusting the earlier baseline data for consistency with the later estimate. 3. This case draws on Lele and Bumb (1995) for general data and on Pillsbury's luridly titled (1999) study for the specifics of the negotiations. 4. USAID (1986a) reports that USAID had trained 240,000 participants until that time and was currently training about 8,000 people a year. Assuming that this annual level continued through 1999. this would bring the total to around 350.000. 5. Another explanation derives from Nasser's standing as the leading figure in Egyptian eco- nomic and political independence. Although Nasserite socialism never lived up to the promise originially held for it. subsequent political leaders derived their legitimacy in part frorn their link to Nasser. so repudiation of his policies was difficult. Consequently, Nasser's followers could move only incrementally away from previous policies, even when they were considered inefficient. 6. Development was not the primary purpose of the Egyptian aid program. The State Department's view has been that the Egypt program was aimed at promoting stability in the Middle East. Within this constraint. economic development is a desirable, though not necessary, feature. 7. Even in the case of the Republic of Korea. the country's overall development success into the middle 1990s has been questioned by some. Only slightly before the recent famine in the People's Republic of Korea. one reputed Cambridge scholar maintained that development in the Republic of Korea had not clearly outpaced that in the People's Republic of Korea (Sanderson 1995). 8. A recent paper blames the donor community's most conspicuous failure-Tanzania-on that country's capacity to design programs that donors found very attractive, combined with an inabil- ity to implement programs that had looked so good on paper (Bigsten 1999). 33 References Amsden, Alice. 1989. Asia's Next Giant: South Korea and Late Industrialization. New York: Ox- ford University Press. Barquet, Nicolau, and Pere Donminlgo. 1997. "Smallpox." Annals of Internal Medicine 127 (Octo- ber): 635-42. Bigsten. Arne. 1999. "Aid and Reform in Tanzania." Economic and Social Research Foundation, Dar es Salaam. Bongaarts, John. WV Parker Mauldin. and Jamnes F. Phillips. 1990. "The Demographic Impact of Family Planning Programs. ' Studies in Family Planning 21-6: 299-310. Cho. Lee-Jay, and Yoon Hyung Kim, 1991. Economic Development in the Republic of Korea: A Policy Perspective. Honolulu: University of Hawaii Press. Cole. David C., and Princeton Lyman. 1971. Korean Development: The Interplay between Politics and Economics. Cambridge. Mass.: Harvard University Press. Das Gupta. Monica. 1999. " Liberte, Egalite. Fraternite: Exploring the Role of Governance in Fer- tility Decline." Journal of Development Studies 35(5): 1-25. Fletcher, Lehman B.. ed. 1996. Egypt s Agriculture in a Reform Era. Anmes: Iowa State University Press. Foege, William H. 1998. " Confronting Emerging Infections: Lessons from the Smallpox Eradica- tion Campaign. " Emerging Infectious Diseases 4 (3): 412-13. Fox, James W. 1990. " Feedback Loops and Econornies of Scale: A Strategy for Export-Led Growth in the Caribbean Basin." USAID/LAC Staff Paper 2. USAID. Washington. D.C. . 1999. "Trends in the Quality of Life in Egypt. 1975-99." Paper prepared for USAID, Egypt, Cairo. GAO (General Accounting Office). 1981. "U.S. Assistance to Egyptian Agriculture: Slow Progress after Five Years." Report ID-81-19. Washington, D.C. Gomez. Victor. 1995. "Population and Family Planning in Costa Rica." USAID Costa Rica Evalu- ation Background Paper, USAID/CDIE, Washington. D.C. Hasan. Parvez. 1976. Korea: Problems and Issues in a Rapidly Growing Economy. Baltimore. Md.: Johns Hopkins Press. Hayami. Yujiro, and Vernon Ruttan. 1971. Agricultural Development: An International Perspec- tive. Baltimore, Md.: John Hopkins Press. 35 APPLYING THE COMPREHENSIVE DEVELOPMENT FRAMEWORK TO USAID EXPERIENCES Hopkins. Donal(d R. 1983. Princes and Peasants: Smallpox in History. Chicago: University of Chi- cago Press. Hopkins, Jack W. 1989. The Eradication of Smallpox. Boulder; Colo.: Westview Press. Jones. Leroy, and Il SaKong. 1980. Government. Business and Entrepreneurship in Economic De- velopment: The Korean Case. Cambr-idge, Mass.: Harvard University Press. Kim, Ching-yumn. 1994. Policymaking on the Front Lines: Memoirs of a Korean Practitioner, 1945- 79. Washinlgton, D.C.: World Bank. Krueger, Annie 0. 1979. The Developmental Role of the Foreign Sector and Aid. Cambridge. Mass.: Harvard University Press. Lele, Uma, and Balu Bumb. 1995. "The Food Crisis in South Asia: The Case of India." In The Evolving Role of the World Bank. Washington, D.C.: World Bank. Mason, Edward S., Mahn Je Kim, Dwight H. Perkins. Kwang Suk Kini, and David C. Cole. 1970. The Economic and Social Modernization of the Republic of Korea. Cambridge. Mass.: Harvard University Press. Mazarr, Michael J. 1991. "Investing in Security: U.S. Economic Assistance and Noneconomic Goals in Korea." Center for Strategic and International Studies, Washington, D.C. Montrie, Charles, and Charles Diamond. 1999. "Program Evaluation Study Plan for the USAID Program and Its Impact on Egypt Policy Reform Programs." TAPR/USAID, Cairo. Muscat, Robert. 1990. Thailand and the United States: Development, Security and Foreign Aid. New York: Columbia University Press. National Research Council. 1986. Population Growth and Economic Development: Policy Ques- tions. Washington, D.C.: National Academy of Sciences Press. Oehmke, James F. 1997. "Agricultural Technology Developrnent and Diffusion: A Synthesis of the Literature." In Luther G. Tweeten and Donald C. McClelland, Promoting Third-World Devel- opment and Food Security. Westport, Conn.: Praeger. Ogden, Horace, G. 1987. CDC and the Smallpox Crusade. U.S. Department of Health and Hunilan Services, Publication (CDC) 87-8400. Oldstone. Michael B.A. 1998. Viruses, Plagues, and History: New York: Oxford University Press. Pillsbury, Michael. 1999. Secret Successes of USAID. Washington, D.C.: National Defense Univei- sity Press. Rock, Michael T. 1993. " Can Export Services Assistance Make a Difference? The Korean Experi- ence." AID Technical Paper 7. USAID, Washington, D.C. Sachs, Jeffrey. 1996. "Achieving Rapid Growth: The Road Ahead for Egypt." Egyptian Center for Economic Studies, Cairo. Sanderson, Stephen. 1995. A General Theory of Historical Development. London: Rowman & Littlefield. Schultz, T. Paul. 1993. "Mortality Decline in the Low-Income World: Causes and Consequences.' American Economic Review 83 (2): 337-4 1. 36 REFERENCES Shurkin, Joel N. 1979. The InvisibleFire: TheStory ofMankinds Victoryover the Ancient Scourge of Smallpox. New York: G.P. Putnam's Sons. Steinberg. David I. 1982. "The Economic Development of Korea: Sui Generis or Generic?" USAID Evaluation Special Study 6. Washington, D.C. ---. 1985. "Foreign Aid and the Development of the Republic of Korea: The Effectiveness of Concessional Assistance." AID Special Study 42. Washington. D.C. 1989. The Republic of Korea: Economic Transformation and Social Change. Boulder, Colo.: Westview Press. Sullivan. Denis J. 1996. " American Aid to Egypt 1975-96: Peace without Development." Middle East Policy (October): 36-49. USAID (U.S. Agency for International Development). 1966. "West Africa Smallpox/Measles." Tech- nical Assistance Paper. Washington. D.C. 1969. " Audit Report on Examination of the Field Operations of the West Africa Smallpox Eradication/Measles Control Project 625-11-510-16." Audit Report 70-68. Washington, D.C. 1971. "Evaluation: Smallpox Eradication and Measles Control Progranm." Bureau for Af- rica, Washington. D.C. ---. 1979a. "Study of Family Planning Program Effectiveness." AID Program Evaluation Dis- cussion Paper 5. Office of Evaluation. Washington. D.C. ---. 1979h. "Family Planning Effectiveness: Report of A Workshop." AID Progr-am Evaluation Report 1, Office of Evaluation, Washington. D.C. ---. 1982. "AID Experience in Agricultur'al Research: A Review of Project Evaluations." AID Program Evaluation Discussion Paper 13. Washington, D.C. 1983. "PL-480 Title I: the Egyptian Case." AID Project Inmpact Evaluation Report 45. Washington, D.C. ---. 1986a. "Review of Participant Training Evaluation Studies." AID Evaluation Occasional Paper 11. Washington, D.C. 1986b. "An Analysis of AID Participant Training Projects." AID Evaluation Occasional Paper 12. Washington. D.C. ---. 1989. "Egypt Shoubrah El Kheima Thermal Power Plant." Project Paper, Amendment 3 (Project 263-0030). Washington. D.C. . 1989. "The Role of Participant Training in Building Social Science Capabilities in Asia." AID Evaluation Occasional Paper 32, Washington, D.C. ---. 1994. "Capital Projects: Egypt Case Study." AID Evaluation Technical Report 20, USAID/ CDIE. Washington, D.C. ---. 1996. "Investments in Agriculture: A Synthesis of the Evaluation Literature." Program and Operations Assessment Report 15, USAID/CDIE, Washington. D.C. ---. 1997. "USAID's Population and Faimily Planning Program: A Synthesis of Six Country Case Studies." Program and Operations Assessment Report 16. USAID/CDIE, Washington. D.C. 37 Valdes, Juan Gabriel. 1995. Pinochet's Economists: The Chicago School in Chile. Cambridge: Cam- bridge University Press. Wilson. James Q. 1989. Bureaucracy: What Government Agencies Do and Why They Do It. N'ew York: Basic Books. WHO (World Health Organization). 1980. "The Global Eradlication of Smallpox." Final Report of the Global Conriiission for the Certification of Smallpox Eradication. Geneva. 38