______ _______ W P S I POLICY RESEARCH WORKING PAPER 1689 Is Economic Anialysis Are the tools for economic Isf Economlc allalyslsanalysis of projects - of Projects Still Useful? developed nearly a quarter century ago - still relevanrt? Yes, but they need to annswver Pedro Belli nwetos new questions. The World Bank Operations Policy Department Decemnber 1996 POLICY RESEARCH WORKING PAPER 1689 Summary findings The methodology for economic analysis of projects is as * Assess whether all of the main actors have the relevant today as it was 25 years ago, reports Belli, but economic and financial incentives to implement the the focus of analysis should shift. project as designed. Project analysts need to: * Take advantage of new measurement techniques to * Make full use of project information, especially that try identifying the project's external effects - as well as embedded in the difference between economic and the benefits of health and education projects. financial prices and in the difference between economic * Take advantage of advances in personal computing and financial flows. to provide a more systematic assessment of risk. * Look at a project from the perspective of the main stakeholders, especially the implementing agency, the government, and the country. This paper - a product of the Operations Policy Department - is part of a larger effort in the department to improve economic analysis of projects. Copiesof the paper are available free from the World Bank, 1818 H Street NW, Washington, DC 20433. Please contact Kristyn Schrader, room MCIO-354, telephone 202-458-2736, fax 202-522-3253, Internet address kschrader@worldbank.org. December 1996. (14 pages) The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about - development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should he cited accordingly. The findings, interpretations, and conclusions expressed in this paperare entirely those of the author. They do not necessarily represent the viewof the World Bank, its Executive Directors, or the countries they represent. b Produced by the Policy Research Dissemination Ceniter Is Economic Analysis of Projects still Useful? (by Pedro Belli) The tools of economic analysis of projects were developed nearly a quarter of a century ago. Since then, the world economy and the economic paradigms guiding economic development have undergone major changes. Are the tools of economic analysis of projects, developed for a different world and a different development paradigm still relevant? Devarajan et. al. (1995) argue that the basic tools-standard conversion factors, sectoral conversion factors, border pricing-are less relevant in today's world and that the important questions concern the proper role of government in the provision of goods and services and the project's fiscal impact. This note argues that while these are important concerns, we need to go way beyond them and assess not only the project's fiscal impact, but its sustainability, the impact on various groups in society, and its risks, and that the traditional tools are useful in answering these questions. The note also argues that the tools can be usefully extended to the evaluation of environmental externalities, and applied to projects that normally have not been the subject of economic analysis. The Questions of the 1970s The methodology for economic analysis was developed in the late 1 960s and early 1970s, when government involvement in the production of goods and services was common. At the time, governments often attempted to influence economic activity through public ownership of assets, price controls, entry and exit of firms, taxes and subsidies, quotas, and discretionary allocation of resources. As a result, the policy environment in which projects were being implemented was rife with distortions and economic analysis was intended to guide government investment decisions in the production of goods and services in highly distorted environments. The methodology attempted to cut through the maze of distortions and answer the question, Will this investment increase welfare for society as a whole? Since market prices were not a good guide to the economic cost of resources, they were of little use in answering this question. An important focus of the methodology, therefore, was on techniques to calculate "shadow" prices that reflected social opportunity costs, and the analysis emphasized "getting the prices right" to compensate for the impact of distortions on the assessment of the net benefits of the project. Economic analysis also had to "get the flows right;" that is, economic analysis had to take into account all of the explicit and implicit transfers to or from the project entity, regardless of whether they entailed a monetary flow. The results of the analysis were summarized in a single measure, the economic rate of return (ERR), as opposed to the financial, or private rate of return. 2 The New Questions Distortions are neither as prevalent nor as pronounced as they were in the 1970s and the role governments in the production of goods and services is shrinking. Is there is still a need for economic analysis, or can we rely on market prices to assess a project's contribution to the welfare of a country? Although the policy environment has improved in many countries, distortions still exist and will continue to exist as long as governments tax and markets are imperfect. Therefore, we still need to get the prices and the flows right. But we need to go beyond that. First, we need to broaden the scope of our analysis and pay more attention to the financial aspects of projects. A project may contribute substantially to the economic welfare of a country, but if the implementing institution lacks the funds to finance it, project implementation will suffer. Therefore, we need to look not only at the project's contribution to economic welfare, but also at its financial aspects. In particular, we need to look at the annual cash flows to ensure that there are no critical years in which the cash flow is so negative that it places the entire project in jeopardy. We cannot, therefore, divorce the economic (or social) from the financial (or private) evaluation. Second, as Devarajan et. al. argue, we need to look at the project's fiscal impact. One of the most important lessons that the World Bank has learned is that counterpart funds play a vital role in project success. Projects often fail because the funds that governments are supposed to provide are not provided on time, or, in the worst of cases, are never provided. To increase the likelihood of success, it is important to ensure that the project does not place an unduly high financial burden on any government agency. Therefore, we need to use the tools to help us assess the project's impact on public finances and to assess it in relation to some relevant magnitude, such as the central government's budget, or the local government's budget, or a ministry's budget, depending on who is financing it. The whole point of the exercise is to assess whether, when the chips are down, the government entity charged with financing the project is likely to have the wherewithal to do it. Third, we need to extend the analysis to sectors that traditionally have not been subjected to rigorous economic analysis. Projects in education and health are increasing as a share of the World Bank's lending program. In 1970 only 9 percent of all loans were for projects in these two sectors, but by 1990 that percentage had nearly double the share. These projects as well need to be subjected to the rigors of economic analysis. In particular, we need to provide quantitative measures of the benefits, whether in monetary or non-monetary units, to choose among alternatives and improve project design. Fourth, we need to take externalities into account more systematically. The presence of externalities has been one of the major sources of divergence between private and social benefits of projects. The effects of projects on pollution, the effects of dams on downstream fishing activities, the effects of wells on the water table, and the effects of irrigation schemes on health, are standard examples of costs of projects that are not always reflected in the money accounts of the implementing agency. Even though the early guidelines recognized the need to take external effects into account, there were no 3 satisfactory ways to measure them in monetary terms. Fortunately, there are now various techniques that enable us to measure external effects Finally, we need to assess project risks to improve project design, identify the critical variables that should be followed during implementation, and reduce the risk of project failure. The conceptual framework for risk analysis has been around for a long time. At the World Bank, Pouliquen used risk analysis as far back as 1970. But risk analysis never became standard practice for lack of a convenient and cost-effective way of assessing risk. Until recently, the use of Monte Carlo techniques for risk analysis, for example, was time-consuming, expensive, and difficult. With the advent of personal computers and canned risk analysis prograns, risk analysis has become as convenient to use as a spreadsheet. The new technology has made it possible to bring risk analysis into the mainstream. Economic Analysis for the 1990s To answer the new set of questions we need to integrate the project's financial and economic evaluation, and keep track of the sources of divergence between financial and economic costs and benefits. Whenever economic and market prices differ, some group in society, other than the project entity, is either paying a cost of the project or enjoying some of its benefits. Similarly, if a flow of benefits accrues to society but not to the project entity, someone other than the project entity is enjoying a project benefit or bearing a project cost. We need to identify (a) the source of the divergence between market prices and economic costs as well as the source of the divergence between economic and private flows, and (b) the group that pays the cost or enjoys the benefits. This information enables us to identify gainers and losers, likely project supporters and detractors, and fiscal impact. Full use of the information available A substantial amount of the information needed to extend the analysis is available either as part of the project profile, or in the data used to calculate economic prices. We do not need to gather more information, we only need to make full use of the information available. Consider the information embedded in two of the most important economic prices used in project analysis, namely the exchange rate, and in the price of a imported good. Take the calculations of the economic cost of foreign exchange for Cyprus done by Jenkins and Savvides (1991). Because of import taxes and export subsidies, the economic price of foreign exchange was some 14 percent above the market rate. In this case, the existence of a premium stemrning from these distortions meant that the government lost revenues whenever it entered the foreign exchange market and diverted a unit of foreign exchange towards a project. The revenue loss was equivalent to 14 percent of the amount diverted. To the extent that the imports for the project paid import duties, the revenue loss was compensated by the duties collected. Box 1 illustrates how this mechanism works through a hypothetical example. 4 Market imperfections also generate rents. In Cyprus, for example, Jenkins and Savvides estimated that the financial price of an imported automobile was about 5,000 Cyprus pounds, compared to an economic price of about 3,382. Of the difference, the net fiscal impact accounted for 1,328 (1,660 pounds in import duties, less 332 pounds lost from the premium on foreign exchange). Monopoly rents accruing to automobile distributors accounted for another 290 pounds. The distribution of taxes and rents was as follows: Project entity Financial cost 5,000 - Import duties 1,660 - Monopoly rents 290 = Border price 3,050 + Foreign exchange premium 332 = Economic cost 3,382 Distribution of (costs) and benefits Project entity (5,000) Government 1,328 Distributors 290 Economy (3,382) Similar breakdowns can be done in every instance where the financial and economic prices and financial and economic flows differ. In both of the examples the importation of a good for the project had a fiscal impact, and in both cases it was possible to assess the fiscal impact properly only by pinpointing the source of the divergence between the economic and financial prices. If we want to assess the fiscal impact of projects, then, it is not possible to ignore border prices and standard conversion factors. On the contrary, since the policy framework has improved, price controls, QRs and other favorite instruments of old (which diverted rents to groups outside the tax authority) are less likely to be the causes of divergence between economic and financial prices, but taxes and subsidies (which have direct fiscal implications) are more likely to be the culprits. More than ever then, in today's policy environment we need to pinpoint the sources of differences between economic and financial prices and economic and financial flows if we want to assess the fiscal impact of projects correctly. Identifying the sources of difference between economic and financial prices and flows not only helps assess the fiscal impact, but also helps identify who gains and who loses from a project. And as mentioned before, the financial analysis of the project is an important source of information for assessing sustainability. If we put together the financial information, the fiscal implications of the project, and the distribution of costs and benefits among the various actors in the economy, the analysis becomes a lot richer 5 and more informative. Such an analysis, summarized in Box 2, was done in the case of the Mauritius Higher and Technical Education project. Box 1: Who gets the Premium on Foreign Exchange? Take a country with a uniform import duty of 15 percent and no taxes or subsidies on exports. Let us say that in this country the exchange rate is market determined and that it is 5:1 with respect to the US dollar. For every dollar of imports, every importer surrenders 5.75 units of domestic currency (5 units to purchase dollars plus 15 percent to pay for import duties). Exporters, on the other hand, receive 5 units of domestic currency for every dollar of exports. The import duty introduces a distortion that drives a wedge between what importers must pay in order to import one dollar's worth of goods and what exporters receive when they export one dollar's worth of goods. Because of this difference, the economic price of foreign exchange is not equal to the market rate.' Let us assume that the economic price of foreign exchange is 5.60, i.e., that there is a premium on foreign exchange of 12 percent over the market rate. A project that uses foreign exchange will cost the economy 5.6 units of domestic currency for every dollar of exports, yet importers will only pay 5.0, What happens to the difference? In this particular case, it can be shown that the difference is a loss of revenue to the government. Of course, since all imports pay 15 per cent duty, for every unit of foreign exchange imported by the project, the government will recover 15 cents. The net fiscal impact would be a positive 3 cents in foreign currency (or 15 cents in domestic currency). In all cases where the premium stems from taxes (and subsidies) on international trade, the premium accrues to the government. But if the premium stems from other sources, it could accrue to other groups in society. If the premium stems from QRs, for example, it would accrue to those who enjoy the benefits of the QRs. Economic Evaluation of Education and Health Projects Like projects in other sectors, education and health projects involve flows of expenditures and benefits over time. Unlike other sectors, however, the measurement of benefits is particularly difficult. Since the early 1960s, and increasing number of economists are treating expenditures on education as investment in human capital. Viewed as such, education can be subjected to the same type of economic analysis as any other type of investment that has identifiable costs and benefits that are measurable in monetary terms. Education increases people's productivity. If the labor market works well, higher productivity will result in higher income. Comparing the earnings of individuals with say, a university education to the eamings of an appropriate control group with only a high school education, then, would be one way of measuring the benefits of a university education in dollar terms. In general, the net benefits of an education for an individual can be measured by assessing the value of his/her incremental earnings over his/her It is important to note that as long as there are taxes and subsidies on foreign trade, a difference between the financial and economic cost of foreign exchange could exist even in a country with a market- determined exchange rate. 6 working life, and subtracting the direct cost of the education (tuition, books, etc.) plus the income forgone while attending school. Psacharopoulos (1995) provides a handy formula: NPVi = (Wu-Ws)t (Ws+Cu) t=1 (1 + r)' t=l (1 + r)' where (Wu - Ws) stand for the earnings differential between a university (subscript u) and a high school graduate (subscript s) and Cu for the costs of providing a university education. The formula presumes that the students take m years to obtain a university degree and that upon graduation they stay in the work force for n years. The benefits, then, are given by the present value of the incremental income, while the costs are given by the present value of the forgone income plus the present value of the cost of the education (tuition, books, etc.). This formula can be easily extended to a group of individuals. This formula can be applied to project evaluation, but adjustments and certain simplifying assumptions are needed. First, it is convenient to assume that the benefits of a project are confined to students that graduate. Thus, even though non-graduates probably enhance their productivity by attending school, the data required to assess the incremental income are not usually available and are expensive to gather; it is simpler to assume that only graduates earn more income. Second, it is also convenient to assume that present income differentials hold throughout the life of the project. Income differentials do not remain constant over time. For example, the earnings gap between engineers and high school graduates in the United States is wider now than two decades ago. There is no guarantee that today's gap will remain constant through time, but since we do not know how that gap is likely to evolve, it is usually assumed that it will remain constant through time. The final simplifying assumption is that the two groups are similar in every respect but earnings, that is they live just as long, they work just as long, they fall ill just as often, etc. Under these conditions, the benefits of an education project are equal to the number of graduates times the income differential per graduate projected for the remaining working life of the graduates. Take a project that produces, say, 50 graduates per year for five years. The benefits of the project in terms of graduates would be as follows: Year 1 2 3 4 5 Number of graduates: 50 50 50 50 50 The monetary value of this stream of benefits in each year would be equal to the present value of the incremental earnings of the first year graduates, plus the present value of the incremental earnings of the second year graduates, plus the present value of the incremental earnings of the third year graduates, etc. The total stream of benefits, then, would be: Year 1 2 3 4 5 Benefits ($1,000): 100 100 100 100 100 7 and the present value of the benefits of the project would be the discounted value of this stream, or about $380,000 if we use a 10 percent discount rate. In short, to assess the benefits of the project, we need to discount the benefits twice: once to assess the benefits accruing in a given year and again to assess the present value of the benefits of the entire project at a single point in time (usually at the beginning of the project). Since the benefits in the ith year are equal to the present value of the incremental earnings ($200 in this case) received every year for 40 years, times the number of graduates in the ith year, the present value of the stream of benefits can be expresses as follows: PV(benefits) = E t( ( ) = (Wu-Ws)iN t=O (+rt=)t where (Wu - Ws)i stands for the present value of the average incremental earnings of an individual from the time of graduation to the time that they exit the labor force, Nt stands for the number of graduates in year t, andj for the number of years that the project lasts. The costs of the project, of course, would include the incremental capital (K) and recurrent (R) costs, as well as the forgone income of the students: PV(costs) = j (Nt)(Ws) + (Kt + Rt) In some projects the unit costs of graduates are of interest. In these cases the equation above can be modified as follows: xz (Kt + Rt) Unit Costs (l + r) Nt ( + r)' and we can calculate the ratio of the present value of costs to the present value of benefits, the latter measured by the discounted number of graduates produced by the program. 8 Box 2: Gainers and Losers: Mauritius Higher and Technical Education The main objective of this project was to increase the quantity and quality of the students coming out of the institutions of higher education in Mauritius in order to increase the productivity of the labor force. The main measure of the benefits of the project was the incremental income of the additional graduates. Given Mauritius efficient labor market and full employment situation, the incremental earnings of the graduates was deemed to be a good measure of the value of the graduate's incremental productivity. Table 1: Summary of Costs and Benefits, Net Present Value as of 1995 (million Mauritius rupees) Higher education Students institutions Government Society (1) (2) (3) (4) Benefits Incremental income 2,204 945 3,149 Costs Forgone income (943) 0 (238) (1,181) Tuition & fees (249) 259 0 0 Investment costs 0 (343) (10) (353) Incremental recurrent costs 0 (144) 0 (144) Transfers from gov 't. 0 487 (487) 0 Total costs (1,202) 259 (734) (1,678) Net benefits 1,002 259 210 1,471 The table above presents the distribution of costs and benefits of the project in terms of the present value of the main items. Each column shows the benefits and costs from a particular stakeholder's point of view, and each row shows the distribution of a benefit (or cost) across the different stakeholders. The first column presents the project from the students' point of view. For them, higher education increases their expected lifetime earnings by 2,2 billion Mauritius rupees. After deducting tuition fees and the value of the forgone income while attending school, the present value of the net benefits amounts to about 1,0 billion. The second column presents the project from the point of view of the institutions of higher learning. The third column presents the government's point of view, that is, the fiscal impact of the project, assessed at 210 million. The final column, which is the algebraic sum of the first three columns, presents the country's point of view, or economic assessment of the project. The project is expected to increase society's income by 1.5 billion rupees. In this case, all of the main stakeholders gain from the project: students would be better off, the institutions of higher education would also be better off, the project would have a positive fiscal impact, and Mauritius would also be better off. This presentation makes full use of the information available about the project, including the information embedded in economic prices and flows. Full use of the information available enables us to integrate the fiscal, economic, and financial analyses and assess project's costs and benefits from the perspective of any number of stakeholders, including the implementing agency, the fisc, and, of course, the country. It also makes it possible to shed light on the questions that are relevant to today's concerns: there is likely to be demand for the project, as students would be better off. The fiscal impact is positive, hence likely to be supported by the government, and the institutions of higher learning are better off both in prestige and financially, hence the project is likely to be sustainable. This information suggests that since there are no apparent losers, no one is likely to oppose the project. 9 Additional benefits The social benefits of education transcend income differentials and there are non- monetary effects as well. It is claimed, for example, that more educated people are healthier and live longer because they live in more sanitary conditions. Other benefits ascribed to education include lower crime rates, higher social cohesion, faster technological change, and fewer unwanted pregnancies. Most of these benefits, however, are difficult to quantify in monetary terms and are not usually included in the calculations of rates of return to education. Using income differentials as the sole measure of benefits, then, generally undervalues the benefits derived from education.2 Measuring Benefits in Health Projects The problem of measuring benefits arises in the health sector with a vengeance because there is an overwhelming and nearly universal reluctance to attach monetary values to health benefits and especially to value life in monetary terms. As a result, the outcomes of health projects are not usually measured in monetary terms, but in some other unit and the measure of effectiveness relate non-monetary benefits to costs. That is, we do not usually calculate the NPV of a project, but rather assess the cost-effectiveness of the intervention. The simplest measure of outcomes in health projects is in terms of the outcome that the project seeks, as in the number of vaccines delivered. The main shortcoming of such measures is that they focus the attention on the intervention itself and not on the results of the intervention. Vaccines are delivered in order to prevent illness and premature death, and they are effective only in so far as they achieve their ultimate aim. A more appropriate measure of the benefits of a project that delivers vaccines is its effectiveness in preventing illness and premature death. Years of poteptial life gained (YLGs) is a measure of benefits that more accurately measures the effects of interventions and are calculated as the difference between the expected duration of life with and without the intervention. YLGs, however, do not take into account the benefits obtained from the prevention of illness. Healthy years of life gained (HYLGs) take into account the gains stemming from preventing illness as well as premature death and they are defined as the sum of years of life gained on account of reduced morbidity and mortality, with morbidity adjusted for disability. Other measures of benefits HYLGs count a year of life gained by a young person the same as a year of life gained by an old person. DALYs (disability adjusted life years gained), on the other hand, weight HYLGs by age. The weights vary by age group, are highly subjective, and may vary across cultures and social contexts. Consequently, DALYs are controversial. QALYs (quality adjusted life years) count a fully functional year of life as one and 2 It is sometimes argued that unemployed university graduates may become disgruntled and cause harm to society. 10 dysfunctional years as fractions. QALYs are a standard tool in cost-effectiveness analysis in OECD countries. Any of these measures lend themselves readily for project analysis. The mechanics are similar to those mentioned above for the evaluation of education projects. The benefits of some interventions are spread out over a number of years and hence must be discounted back to a base year, normnally the year in which the intervention occurs. If the intervention occurs over several years, then the benefits must be discounted again. If HYLG stands for the present value of the benefits of an intervention in year i, then the value of the benefits in the year in which the project is being evaluated will be: PV(benefits) = E ( L)I (1) t=O (1+ r)' Box 3: The Mechanics of HYLGs Consider a disease that affects one person in 20 thousand every year. Suppose that the disease usually strikes at age 15 and that of those stricken, 70 percent recover fully after 90 days of an illness; 10 percent become chronically disabled and fall ill some 30 percent of the time during the rest of their lives; and finally 20 percent die from the disease after one year of illness. What would be the benefits from a treatment that prevents this disease? First, we calculate the days lost to the illness, given that a person falls ill. Those who die, on average die at age 16 (disease strikes at age 15 and lasts for one year). To find out how many days of life those who die lose, we need to know their life expectancy. Say that in this country it is 62 years. An individual who is stricken, then, loses 46 years of expected healthy life: one to illness and 45 years of life. This means that on average, we can expect to lose 9.2 years of healthy life, or 3,358 days (46 x 0.2 x 365). Those who recover fully lose 90 days: on average we can expect to lose 63 days of healthy life (90 x 0.7). Finally, days lost to illness and chronic disability amount to 5,127: 90 days of illness plus 30 percent of time lost due to illness from age 16 to age 62 (90 + 46 x 0.3 x 365). On average, we would expect to lose 513 days (5,127 x 0.1). The expected number of days lost to illness, given that a person falls ill, would then be given by the sum of all three effects: 3,358 + 63 + 513 = 3,934. Second, we calculate the expected number of days lost due to illness for a given population. This number would be equal to incidence of the disease times the expected number of days lost to illness: (0.05 x 3,934) = 197 per thousand. Finally, we calculate the benefits of preventing the disease. The benefits from any treatment that reduces the incidence of the disease would be given by the number of healthy years of life gained, which in turn would depend on the extent of coverage and efficacy of the treatment. Assuming a 95 percent effectiveness with 80 percent coverage, treatment would save [.95 x .80 x 197] 150 days per 1,000 population per year. Is a life saved today worth more than a life saved tomorrow? Equation (1) implies that a healthy year enjoyed today is more valuable than a healthy year enjoyed in ten years. This is in turn implies that a life saved today is worth more than a life saved tomorrow. The justification for this position is two-fold. First, we need to make the obvious point that when we talk about saving lives, we are really talking about prolonging lives. Second, we must also make the obvious point that life is valuable 11 because we enjoy being alive. Enjoyment today is more valuable than enjoyment tomorrow. Hence, if an activity prevents enjoyment to be shortened today as opposed to tomorrow, that activity is more valuable than an activity that prolongs enjoyment in the future rather than today. Another reason for valuing prolongation of life in the future less than prolongation of life in the present is as follows. Suppose that a program costs $1,000 and will avert premature deaths at $10 per person. We have two options. First, we can spend $1,000 this year and avert 100 deaths, or we can invest the $1,000 for one year at, say, 3 percent and have $1,030 next year, allowing us to prolong 103 lives next year. If we value premature deaths averted in the future as much as those averted today, we will take the second option. But next year we will be faced with a similar choice and we will make a similar decision, as we would be able to save 106 lives in the third. Obviously, according to this logic, as long as we can invest the money at some positive real rate and save more lives in the future, we would rather invest than saves lives. This leads to the absurd conclusion that we should never save lives. If we accept the premise that improving health status in the present is more valuable than improving health status in the future, then the benefits from health interventions need to be discounted just like the benefits of any other project. Environmental Externalities The need to take into account environmental externalities was recognized since the inception of economic analysis of projects, but until relatively recently, the measurement problems were extremely difficult, if not impossible, to overcome. Since 1980 a number of techniques have become increasingly popular (see the review by Dixon et. al. (1994), for example). The techniques for measuring the values of externalities fall into two broad categories, those that rely on objective measurements and those that rely on subjective assessments, as revealed in real or hypothetical market behavior. The first set of techniques assesses the impact of externalities through the use of a production function that relates the level of activity to the degree of damage (or benefit). For example, we may relate the level of production of a factory to the level of pollutants in the air or to the soot in adjacent buildings. Or we may relate the level of exposure to pollution to health effects. Because most of the environmental externalities produce "bads" instead of "goods," these production functions are usually referred to as "damage functions," but they are also referred to as "dose-response" functions. Damage or dose-response functions are estimated from both field studies and controlled experiments. A major advantage of damage and dose-response functions is that some of thenm are transferable among countries. Objective measurements techniques have been use(d to estimate the value of soil erosion, the value of reduced fish catch, the value of tourist attractions, the impact of pollution on mortality and morbidity, etc. Subjective valuation approaches are based on real or hypothetical market behavior. For example, people sometimes incur costs to avert an undesirable consequence, such as boiling water before consuming it to prevent transmission of water- 12 borne diseases, or constructing dikes to prevent soil erosion. The cost of boiling water may be taken as an indication of the willingness to pay to prevent a disease. Kim and Dixon (1986) estimated the costs incurred by farmers in building dikes to prevent waterborne eroded soil from silting up their fields and damaging their crops. The costs incurred by farmers in building dikes were taken to be at least equal to their subjective valuation of the benefits of preventing soil erosion. Examination of housing markets has revealed in many cases that property values are higher in areas where air quality is good than in areas where it is bad. The difference in property values has been taken as a proxy for willingness to pay for good air quality. In some cases not even indirect indications of the value of goods are available and it is necessary to construct hypothetical markets. Contingent valuation methods are useful in these situations. These techniques involve direct questioning of consumers to determine their reactions to hypothetical situations. For example, an interviewer may describe a good to respondents, say improved water quality, and ask them for the maximum amount that they would be willing to pay for the good, or for the minimum compensation that they would be willing to accept to do without the good. The responses are then averaged and extrapolated to arrive at an aggregate willingness to pay or at an aggregate level of compensation that the population is willing to receive. Contingent valuation methods have been used to estimate the value of rural water supply in India, of ambient surface water in Rio de Janeiro, of a marine park in Netherlands Antilles, etc. Risk Analysis Sensitivity analysis and switching values have been the traditional tools of risk analysis at the Bank. The former identifies the variables that most influence a project's net benefits and quantifies the extent of their influence. In particular, it assesses the effects on the net benefits of the project of varying the values of critical variables by an arbitrary percentage. The latter determines percentage by which a variable must depart from its posited value in order for the net benefits of a project to disappear. Sensitivity and switching value analyses have two major limitations. First, they do not take probabilities into account. Second, they do not take correlations into account. Thus, sensitivity and switching value analyses may tell us, for example, that a if given variable departs by more than 25 percent from its posited value, the project will go bust. This information is of limited use because it does not tell us how likely such an event may be. The major shortcoming of both types of analyses, however, is the disregard for correlations. The usual technique of varying one variable at a time is justified only if the variable is uncorrelated with all the other project variables. Unfortunately, when it comes to projects, correlations can be devastating because if one thing goes wrong, it is likely that many things go wrong. For example, if demand for the project falters because the expected economic growth does not materialize, then counterpart funds may also be in short supply. If the variables are correlated, then varying one variable at a time may lead us to believe that a weak project is robust. Monte Carlo analysis overcomes these limitations. It takes into account probabilities and correlations, it identifies the likely impact of each variable on project 13 outcomes. It can also take into account delays and other events that may impinge on project outcomes. More importantly it correctly calculates expected net present value of the project, the probability distribution of the outcome, and the probability of failure of the project. In Monte Carlo simulation, the computer acts as if we were implementing the same projects hundreds or even thousands of times under the specified conditions. It then pools the results and estimates averages as well as probability distributions of the random variables, including the probability that the project will result in a negative net present value. By ranking the variables not only in terms of their impact on project outcomes, but also in terms of probability of occurrence, Monte Carlo simulation can help design better projects and zero-in on the variables that are worthwhile tracking during project performance. Monte Carlo techniques have been around for many years, but until recently the computational requirements to apply it to projects were formidable. With the advent of spreadsheet programs, the technique is neither time consuming, nor expensive, nor difficult to use. Several canned packages that can be used in combination with spreadsheets are commercially available. Summary In summary, while the questions that we ask of projects today as opposed to 25 years ago have changed, the methodology for economic analysis of projects is as relevant today as it was 25 years ago. The focus of the analysis needs to shift and we must make full use of project information, especially of the information that is embedded in the difference between economic and financial prices and of the difference between economic and financial flows. In addition, project analysts need to look at the project from the perspective of the main stakeholders, principally the implementing agency, the government, and the country. They should also assess whether all of the main actors have the economic and financial incentives to implement the project as designed. They should also take advantage of advances in technology and attempt to identify and measure any external effects of projects, as well as the benefits of education and health projects. Finally, they should take advantage of the advances in personal computing to provide a more systematic assessment of risk. 14 Bibliography Devarajan, S., L. Squire, and S. Suthiwart-Narueput. 1995. "Reviving Project Appraisal at the World Bank," Policy Research Working Paper, 1496, Washington, The World Bank. Dixon, J., L.F. Scura, R.A. Carpenter and P.B. Sherman. 1994. Economic Analysis of Environmental Impacts, 2nd ed., Earthscan Publications Ltd., London. Jenkins, G. P. and A.C. Harberger. 1992. Manual: Cost Benefit Analysis of Investment Decisions, mimeo. Little, I.M.D. and J.A. Mirrlees. 1974. Project Appraisal and Planning for Developing Countries, Basic Books, Inc., New York. Pouliquen, L. 1970. "Risk Analysis in Project Appraisal," World Bank Staff Occasional Paper, 11, Washington, The World Bank. Squire, L. and H.G. van der Tak. 1975. Economic Analysis of Projects, Published for The World Bank, The Johns Hopkins University Press, Baltimore. World Bank. 1992. Economic Analysis. of Projects: Towards a Results-oriented Approach to Evaluation. Operations Policy Department, The World Bank. World Bank. 1995. A Review of the Quality of Economic Analysis in Staff Appraisal Reports for Projects Approved in 1993. Operations Evaluation Department/Operations Policy Department, The World Bank Policy Research Working Paper Series Contact Title Author Date for paper WPS1665 How Important Are Labor Markets Andrew D. Mason October 1996 D. Ballantyne to the Welfare of Indonesia's Poor? Jacqueline Baptist 87198 WPS1666 Is Growth in Bangladesh's Rice John Baffes October 1996 P. Kokila Production Sustainable? Madhur Gautam 33716 WVPS1667 Dealing with Commodity Price Panos Varangis October 1996 J. Jacobson Uncertainty Don Larson 33710 WIPS1668 Small is Beautiful: Preferential Trade Maurice Schiff October 1996 M. Patena Agreements and the Impact of 39515 Country Size, Market Share, Efficiency, and Trade Policy VVPS1669 International Capital Flows: Do Punam Chuhan October 1996 T. Nadora Short-Term Investment and Direct Gabriel Perez-Quiros 33925 Investment Differ? Helen Popper NVPS1670 Assessing the Welfare Impacts Dominique van de Walle October 1996 C. Bernardo of Public Spending 31148 WPS1671 Financial Constraints, Uses of Asli Demirgu,c-Kunt October 1996 P. Sintim-Aboagye Funds, and Firm Growth: An Vojislav Maksimovic 37644 International Comparison VVPS1672 Controlling Industrial Pollution: Shakeb Afsah October 1996 D. Wheeler A New Paradigm Bemoit Laplante 33401 David Wheeler WPS1673 Indonesian Labor Legislation in a Reema Nayar October 1996 R. Nayar Comparative Perspective: A Study 33468 of Six APEC Countries '/PS1674 How Can China Provide Income Barry Friedman October 1996 S. Khan Security for Its Rapidly Aging Estelle James 33651 Population? Cheikh Kane Monika Queisser \NPS1675 Nations, Conglomerates, and Branko Milanovic October 1996 S. Khan Empires: The Tradeoff between 33651 Income and Sovereignty WPS1676 The Evolution of Payments in David B. Humphrey October 1996 T. Ishibe Europe, Japan, and the United Setsuya Sato 38968 States: Lessons for Emerging Masayoshi Tsurumi Market Economies Jukka M. Vesala VWPS1677 Reforming Indonesia's Pension Chad Leechor October 1996 G. Telahun System 82407 Policy Research Working Paper Series Contact Title Author Date for paper WPS1678 Financial Development and Economic Ross Levine October 1996 P. Sintim-Aboagye Growth: Views and Agenda 38526 WPS1679 Trade and the Accumulalion and Pier Carlo Padoan November 1996 M. Patefna Diffusion of Knowledge 39515 WPS1680 Brazil's Efficient Payment System: Robert Listfield November 1996 T. Ishibe A Legacy of High Inflation Fernando Montes-Negret 38968 WPS1681 India in the Global Economy Milan Brahmbhatt November 1996 S Crow T. G. Srinivasan 30763 Kim Murrell WPS1682 Is the "Japan Problem" Real? Shigeru Otsubo November 1996 J. Queen How Problems in Japan's Financial Masahiko Tsutsumi 33740 Sector Could Affect Developing Regions WPS1683 High Real Interest Rates, Guarantor Philip L. Brock November 1996 N Castillo Risk, and Bank Recapitalizations 33490 WPS1684 The Whys and Why Nots of Export Shantayanan Devarajan November 1996 C. Bernardo Taxation Delfin Go 37699 Maurice Schiff Sethaput Suthiwart-Narueput WPS 1685 Macroeconomic Crises afnd Poverty Gaurav Datt November 1996 P. Sader Monitoring: A Case Study for India Martin Ravallion 33902 WPS1686 Institutions, Financial Markets, and Asli Demirgu,c-Kunt November 1996 P, Sintim-Aboagye Firms' Choice of Debt Maturity Vojislav Maksimovic 37644 WPS1687 Regionalism versus Multilateralism L. Alan Winters November 1996 A. Kitson-Walters 33712 WPS1688 Risk, Taxpayers, and the Role of Michael Klein December 1996 S. Vivas Government in Project Finance 82809 WPS1689 Is Economic Analysis of Projects Pedro Belli December 1996 K. Schrader Still Useful? 82736