76596 BEYOND RATE OF RETURN: REORIENTING PROJECT APPRAISAL Shantayanan Devarajan Lyn Squire Sethaput Suthiwart-Narueput Traditional approaches to project appraisal fail in practice to address two fundamental questions: whether a project belongs in the public or the pri- vate sector; and what effect any external assistance associated with the project has on the country's development. The first issue is of general interest to both national policymakers and international donors. If the government provides a good or service that would otherwise have been provided by the private sector, the net contribution of the public project could be low. The second issue is of particular concern to donors. If financial resources are fungible, the project being appraised might well have been undertaken with- out external financing. In this case, donor funds are actually financing some other, unappraised project. Both cases argue for a shift in the emphasis of project evaluation away from a concern with precise rate-of-return calcula- tions and toward broader sectoral analyses and public expenditure reviews. In this context, three areas critical for proper project appraisal include a consideration of the rationale for public intervention, the fiscal impact of the project, and the fungibility of external assistance. A quarter century ago, several groups of economists (Little and Mirrlees 1969,1974; Dasgupta, Marglin, and Sen 1972; Harberger 1973; Squire and van der Talc 1975) developed methods for appraising investment projects financed from public revenues. Extending the principles used by the private sector in making investment decisions, these economists advocated the use of "social cost-benefit analysis," a test that weighs a public project's costs and benefits in terms of its contribution to national (social) welfare. If the proj- ect's social benefits exceeded its social costs, the recommendation was that the government should undertake the project. Although there are important differences in the various methods for evaluat- ing the costs and benefits, they have at least three elements in common. First, The World Bank Research Observer, vol. 12, no. 1 (February 1997), pp. 35-46 O 1997 The International Bank for Reconstruction and Development / THE WORLD BANK 3 5 they emphasize that a project's inputs and outputs should not necessarily be valued at current market prices because the market price may not reflect the social opportunity cost of the resource—that is, its cost in forgone benefits to society. Say, for example, that a project hires an unemployed worker. Even though his wage is a cost to the project, it does not represent the social cost, which would be the supply price of labor (that is, the price at which a person would be willing to work). When there is unemployment this usually means a price (or wage) that is below the prevailing—or actual—wage. Thus, rather than use market wages as the cost of labor, for example, the evaluator is instructed to use a set of "shadow prices" that reflect the social opportunity cost of the project's inputs and outputs. Second, policymakers should evaluate every component of a project relative to a counterfactual—that is, what would have happened without the project? The example above also illustrates this principle: because the worker in this case was unemployed, no national output is forgone when he is drawn into the project. Finally, the methodology dictates that because the project's benefits and costs occur at different points in time, they should be combined in some summary statistic, such as the project's net present value or internal rate of return. Today, a charitable assessment would have to acknowledge that the practical application of these principles has been limited. Governments and international agencies (some of which developed social cost-benefit analysis) use the tech- niques only sporadically. In a retrospective paper Little and Mirrlees (1991, p. 376) concluded "that the extent to which [social cost-benefit analysis is] used and [has] real influence is not great, even in the World Bank." Although there are many reasons for this decline in the use of social cost- benefit analysis, Little and Mirrlees (1991) suggest four: the growth of nonproject lending; internal incentives in lending institutions; new concerns such as pov- erty, women, environment, and "sustainability"; and the complexity of the meth- odology. In addition, the traditional approach may often fail to address the fun- damental questions of concern to policymakers and donors today. Among these questions two of the most important are whether a project belongs in the public or private sector and what the effect has been on development of the external assistance (if any) associated with the project? The first question is of general interest to both national policymakers and international donors. The world has changed substantially in the past twenty- five years. Governments, and donor agencies are now reconsidering the role of the state. Instead of asking if the project generates a positive net social benefit, governments and agencies are asking if there is a rationale for public provision of that good or service. Whether the standard approaches to project analysis are suited to answering the latter question is not clear. For instance, a recurring example in Little and Mirrlees' (1974) book is an industrial project in India—a project whose rationale for public intervention would be dubious today. By con- trast, projects with a strong public-sector rationale, such as vector control and immunization (described in Hammer's companion article in this volume), are 36 The World Bank Research Observer, vol. 12, no. 1 (February 1997) likely to be those with large positive externalities (that is, their value to the community is greater than their price). The resulting improvement in welfare, although apparent, is difficult to quantify in net present value or rate-of-return calculations.1 The second issue is of particular concern to donors. If financial resources are fungible, it is unclear what a project's rate of return conveys about the loan's effect on development. The project being appraised might well have been under- taken without external financing. In this case, the donor's funds are actually financing some other project that would not have been carried out otherwise. What is the role of project analysis in a world in which the public-private boundary is the relevant issue? This article shows that the principles underlying social cost-benefit analysis can incorporate this dimension but that the tech- nique must be modified. Because the projects under consideration by policymakers and donors are public projects, an appropriate counterfactual is what the pri- vate sector would have provided in the absence of the project. What is required is a sectoral anlysis to identify market failures (that is, instances where the mar- ket over- or under-provides goods or services relative to the socially desirable levels) that warrant public intervention. Reorienting project appraisals in this direction also leads policymakers to focus on the project's fiscal impact. The article addresses the problems that arise with standard net present value and rate-of-return calculations when financial resources are fungible and argues that policymakers should not rely on such techniques. A more appropriate analysis is a review of public expenditure programs, both to improve the quality of projects and to ensure that the impact of foreign aid on development is favorable. The Rationale for—and Cost of—Public Provision Traditional cost-benefit analysis, as presented by Little and Mirrlees and many others, addressed the following question: Will the project under consideration result in a net benefit to the economy? This is an important question. No one wants to invest in projects that impede overall development. But the answer to this question says nothing about whether the project ought to be in the public or private sector. Twenty-five years ago that concern may not have been so great. At the time governments were expanding public investment rapidly, and much of that in- vestment was in industrial sectors. Indeed, the first version of Little and Mirrlees' book, published in 1969, was titled Manual of Industrial Project Analysis. But now countries worldwide are redrawing the boundary between the public and private sectors and paying more attention to whether or not a project ought to be in the public sector. The principles underlying the manuals on project evaluation can be applied to the changed circumstances and can help to formulate a new set of questions about the appropriate role of government. In particular, all the approaches to Shantayanan Dcvarajan, Lyn Squire, and Sethaput Suthtwart-Narueput 3 7 cost-benefit analysis require the project evaluator to specify the counterfactual. This principle is perfectly general; it applies even when the alternative to public provision is a private project. To illustrate, assume that the government is contemplating an investment that produces private goods—a shoe factory, say. One possible judgment re- garding the counterfactual is that, in the absence of the public-sector project, nothing would have happened. In this case, the analyst should focus on a com- parison of the costs incurred by the project and the benefits it is expected to yield. Assuming that the evaluation points to a positive net present value (that is, the benefits are higher than the costs), the decision would be made to go ahead with the project. But the appropriate counterfactual might be that the private sector would have produced the shoes anyway (assuming the enterprise is profitable). In this case, the relevant magnitude is the net contribution of the government shoe factory. The net present value of the public-sector project over and above that of the private-sector project (evaluated at shadow prices) may well be zero. Specifying the counterfactual is not always easy. The relevant counterfactual to public provision includes the private market outcome considering taxes, sub- sidies, and regulations. But some principles can be invoked. First, if the project is producing a private good that is profitable at market prices, there is good reason to believe that the private sector will undertake it. In this case there is no advantage to public provision, and the point can probably be established with- out a serious evaluation. Second, at the other extreme is the case of pure public goods. Here there is no prospect of private provision and hence no need to worry about a private-sector counterfactual. But pure public goods—defense, for example—are relatively rare. The third possibility is the need to redistribute income to the poor or correct for market failures, such as externalities and public goods. In such areas private markets will not yield a socially desirable outcome, and a case can be made for government intervention (even though the private-sector counterfactual will not necessarily be zero). Because many projects are likely to fall under this category, we consider it in more detail. Consider the case of a product with a positive externality—a benefit from producing a good or service that is not fully captured in the product's price. The private sector will provide some of the good, but because producers are not paid the full social value of it, they will produce less than the socially optimal amount. In figure 1, the positive externality is shown as the marginal social benefit, which describes how much society is willing to pay for each unit of the good. That amount is greater than the marginal private benefit—or private demand, which is the price that individuals are willing to pay. The marginal social and private benefit lines are drawn as perfectly elastic for ease of exposition. An example would be secondary education, which many would claim has significant positive externalities, although in many countries private secondary schools exist along- side public ones. In figure 1 the private sector would provide up to Q of the good 38 The World Bank Research Observer, vol. 12, no. 1 (February 1997) on its own. The socially optimal amount, however, is Q*. (In the shoe factory example referred to earlier, the marginal social benefit and marginal private benefit curves coincide, so that Q = Q*.) In evaluating projects such as these, the analyst needs to establish three points. First, public provision should result in a greater supply of the good than would have occurred with just private-sector provision (that is, overall supply should exceed Q). It is also possible that private provision up to Q is not forthcoming because, for example, of capital-market imperfections. But if that were the case, direct intervention in the capital markets would be better than addressing the problem indirectly through public provision. It is important to ascertain pre- cisely why private supply is not forthcoming. The current level produced by the private sector could also represent that amount of output that can be profitably produced and sold given the costs imposed by government-induced distortions. In that case the appropriate comparison should be between the net benefits of the project and the costs of removing those distortions. If the state displaces private investment, it would simply be "crowding out" private providers. The amount produced by the private sector in the absence of the project (the counterfactual) would be equal to the amount produced by the project (assum- ing costs are the same), and thus the net present value of the public-sector project would be zero. Second, the project should not result in total supply of the good beyond Q* (the point at which marginal costs exceed marginal social benefits). Thus, the relevant range for public provision in this sector is between Q and Q*. Figure 1. Public Provision and the Private Sector Counterfactual Marginal cost Marginal social benefit Price Marginal private benefit Note: Marginal cost (benefit) is the cost of (benefit from) an additional unit of output. Source:Authors' calculations. Shantayatum Devarajan, Lyn Squire, and Sethaput Suthhvart-Narueput 39 Third, although the presence of spillovers and market failure may justify pub- lic intervention, it should not be presumed that public provision is required. The relevant counterfactual to public provision could well be the private market outcome under appropriate taxes, subsidies, or regulation. In figure 1, for ex- ample, an alternative to public provision up to the optimal quantity Q* would be a subsidy equal to height AB, the size of the positive externality at Q*. Figure 1 is drawn as though the exact size of the externality were known so that the optimal amount the public sector should provide—Q*—could be deter- mined. In reality, the true value of the externality will rarely be known. Indeed, that is true of most justifications for public intervention, whether the rationale is a market failure or redistribution. Until better estimates of externalities are available, there is no real solution to this problem. Thus the risk of over- or under-supply cannot be eliminated. The costs of public-sector intervention, however, can be measured much more easily. They can be approximated by calculating the additional burden the project imposes on the budget when consumers are charged the full amount they are willing to pay. In figure 1 the area under the marginal cost between Q and Q* is the total additional cost, but consumers are paying BCQQ* to the government; thus the additional cost of the project is shown by the area ABC. Of course, this area can be identified only because the size of the externality is known. Assume that the size of the externality is unknown—that is, the mar- ginal social benefit line in figure 1 cannot be located. In fact, that line could lie anywhere above the line depicting the marginal private benefit. If the govern- ment still decides to supply Q*, then the area ABC can be interpreted as the additional cost incurred in order to realize the (unknown) benefit implicit in the existence of the externality. Such cost calculations can provide a useful "reality check" on proposed interventions. Whatever the true size of the exter- nal benefits, the government must judge that at a minimum the external ben- efit exceeds this cost for the intervention to be worth undertaking.2 To illus- trate, CNCA (Caisse Nationale du Credit Agricole), a development bank serving rural areas in Morocco, received subsidies that could conceivably be justified on the grounds that the bank operated in an underserved rural credit market and reached poor people. Although these benefits are hard to quantify, assess- ing the cost of the subsidy is one way to ask whether this subsidy is a good use of scarce public resources and to think about alternative uses. In this case, CNCA's annual subsidy amounted to about 20 percent of the recurrent budget for primary education and 160 percent of the recurrent budget for basic health care. And this in a country where social indicators are quite unsatisfactory— primary enrollment is around 70 percent, and under-five mortality is about eighty deaths per thousand live births. It should be emphasized that while Q is an unobserved counterfactual, it is routinely assessed implicitly during project appraisal. Most project appraisals include calculations of both the financial net present value (using market prices) and the economic net present value (using shadow prices). A positive financial 40 The World Bank Research Observer, vol. 12, no. 1 (February 1997) net present value strongly indicates that the private sector could undertake the project (because it generates profits).3 Such a positive value is thus an argument against public-sector investment in the project. (This does not mean that the project should necessarily be undertaken from a social standpoint, simply that it should not be undertaken by the public sector. A private project that generated negative externalities—pollution, for example—could be privately profitable but socially undesirable.) Note that this conclusion is exactly the opposite of current practice where a high and positive financial net present value is used to justify a public-sector project. An estimate of what consumers are willing to pay provides an indication of how much the private sector would have provided, because a private company should be willing to provide goods as long as it is profitable to do so. An ap- praisal of the Leyte-Luzon Geothermal Project in the Philippines, a $1.3 billion project, illustrates the idea. Consumers were willing to pay 6.8 cents a kilowatt- hour (kWh) for electricity, based on current bulk energy rates in Luzon, com- pared with an estimated long-run marginal cost of 5.2 to 5.8 cents/kWh to oper- ate the project. Because consumers were willing to pay more than the cost to produce the electricity, it would have been profitable for a private company to undertake the investment. Public provision here crowded out at least this quan- tity of private provision. The same argument holds for projects that aim to redistribute income. For these projects the government charges less than consumers are willing to pay and bears a fiscal cost. Although the incidence of such redistributive transfers may be easier to quantify than externalities are, the government still needs to judge whether the value of the redistribution outweighs its fiscal cost. As the public sector shifts the composition of projects in favor of those with a dear public-sector rationale—on either market failure or redistributive grounds— these projects will usually imply a subsidy. To put the same point differently, these projects place a burden on the budget. Another change in circumstances or, more accurately, in the appreciation of what conditions support develop- ment, is the recognition that macroeconomic stability is an essential prerequi- site. Since a prudent fiscal policy is central to macroeconomic stability, it fol- lows that projects that require government funding need to be reviewed with care. Fiscal balance requires that the government recoup the costs of these projects through some other tax instrument, which in turn will introduce distortionary costs somewhere in the economy. These costs should be included in the evalua- tion of the project. If they are omitted, and public costs and private benefits are treated equally, the net present value of these projects will be systematically overestimated (Squire 1989). Although policymakers do not have a precise mea- sure of the marginal cost of funds for most countries, the net impact of the project on the government's budget—a minimum measure of the true cost—is still worth showing. Because of the uncertainty surrounding the marginal cost of public funds, it is useful to reestimate the project's net present value for a range of values of the marginal cost of funds (box 1). Shantayanan Devarajan, Lyn Squire, and Sethaput Sutbiwart-NanupuX 41 Box 1. Revisiting the Marginal Cost of Public Funds How does the government generate funds to cover the fiscal cost of a project? Even if the funds are borrowed, ultimately the government's only source of revenue is the domestic tax base. Every tax instrument, however, imposes a cost on the economy because it creates a distortion. This additional cost should be applied to the project. How much does it cost? Ballard, Shoven and Whalley (1985) estimate that it costs seventeen to fifty-six cents in the United States to raise a dollar of extra revenue (yielding a marginal cost of public funds of $1.17-$1.56). This suggests that public projects should produce marginal benefits of more than $1.17 per dollar of cost. In developing countries, the marginal cost of funds is likely to be even higher to the extent that these countries have access to a more limited set of tax instruments (trade taxes, for example), which are highly distorting. To ensure that public-sector projects recover as much of their costs as possible from the private sector, an appropriate pricing strategy is important. Figure B-l uses the geothermal project in the Philippines to show how a project's net present value changes with different pricing policies and different levels in the marginal cost of funds. If the electricity is sold at 80 percent of the market value and there is no distortion associated with increased taxation (that is, the marginal cost of funds is one), the net present value for the project is $29.7 million. But with even a small premium of twenty cents to raise each dollar, the project's net present value becomes negative. With a marginal cost of funds of 1.6 (roughly the upper range estimated for the United States), the project's net present value drops to -$93.3 million. Figure B-l. Leyte-Luzon Geotbermal Project: Prefect Net Present Value at Discounted Price for Electricity Net present value 40.0 29.7 20.0 0.0 1.00 1.20 -20.0 -11.3 -40.0 -60.0 -80.0 -100.0 Marginal cost of public funds Note: Assumes 80 percent of market rate for electricity charges and 10 percent for discount rate. Source: Project documents; author calculations. 42 The World Bank Research Obttrvtr, vol. 12, no. 1 (Februiry 1997) The Fungibility of Aid Project-specific appraisals can (at best) only assess the project's rate of return or its acceptability. This approach is problematical for two reasons, both of which are important for multilateral lending agencies and donors interested in the impact not only of aid-financed projects but also of aid itself. First, as noted earlier, financial resources are fungible at least to some extent (Feyzioglu, Swaroop, and Zhu 1995; Pack and Pack 1990, 1993). It is unlikely that the projects evaluated by the World Bank, for example, are so marginal that they would otherwise not have been carried out. For the ninety-nine projects evalu- ated in 1993, the World Bank (1994) found an average economic rate of return of 21 percent—a return too high to indicate marginal projects. Second, even if the project would not have been undertaken without external funding, there is no guarantee that it was the best of all the projects under consideration. Yet that is the relevant question. One practicable approach is to require sectorwide reviews before project- specific appraisals and financing decisions are made. For example, Humplick and Paterson (1994), who studied the infrastructure sector in Peru, calculated the economic rate of return of expanding each of the major road links in the country. The results varied widely. The report recommended funding only those projects with a rate of return above 12 percent. On this basis several road ex- pansion projects were dropped, cutting $275 million from the government's road construction program. Donor-financed road projects may have had high ap- praised rates of return, but if these projects were already included in the government's prospective investment program, the development impact of do- nor financing would have allowed funds to be shifted to one of the projects with a rate of return below the 12 percent cutoff rate. These reviews actually go beyond setting a good foundation for subsequent project appraisal. They also improve the overall quality of the sectoral invest- ment program. Moreover, if the result is satisfactory, the specific project fi- nanced and appraised by the donor agency becomes less important. The donor could have more impact on development by associating itself with that project where its knowledge and technical expertise is likely to be of most value. Alter- natively, the agency could finance a "rime-slice" of a specified expenditure pro- gram, an option that is receiving greater attention (Harrold and Associates, 1996). (Time-slice lending finances a certain percentage of public expenditures for a certain number of years rather than the whole project.) Sectoral expenditure reviews can also shed light on the hypothetical no- program (or counterfactual) state by identifying areas where die private sector al- ready provides, or can provide, the goods and services in question. For instance, a review of public expenditures in Malaysia (World Bank 1992) noted that 62 percent of the Ministry of Health's funds went to medical care (mostly private goods) and only 23 percent to public health, although the latter clearly had a higher marginal impact on heakh.4 Although sectoral expenditure reviews do not provide data on Shantayamtn Dtvarajan, Lyn Squire, and Sethaput SuthiuHtrt-Harueput 43 the marginal cost of public funds, public expenditure reviews regularly conducted by die World Bank that cover a government's entire budget could in principle do so. At the very least, public expenditure reviews should identify those countries in which die gains from applying the marginal cost of funds in project evaluation—or the losses from not doing so—are the greatest. Conclusion Altogether governments in developing countries typically spend about 4 percent of gross domestic product, or a total of about $190 billion on public investments every year. Overseas development assistance runs at about $45 billion annually. Even marginal improvements in assessing investment projects could therefore have very high payoffs. At the same time, die analytical skills to undertake careful evalu- ations are limited and die information required is often lacking. To some extent, the changed circumstances justify the relatively limited at- tention that government and donors have paid to the appraisal process (World Bank 1995). In this context, how should the scarce analytical resources that are available be allocated to improve the quality of projects and achieve the maxi- mum impact on development? Public Expenditure Reviews Governments and donors should routinely review die entire public expendi- ture program or its sectoral components before embarking on the appraisal and financing of specific projects. Governments would benefit because such a re- view would improve the overall quality of the public investment program; do- nors would benefit because the program would provide some assurance diat aid flows were being well utilized even when financial resources were fungible. Once an expenditure review is in hand, aid programs should be designed to ensure that the recommendations of the review are implemented and that finan- cial assistance is conditioned on a satisfactory program of public spending. The choice of lending instrument—single project, sectoral investment loan, or gen- eral budgetary support—should be made according to which vehicle contributes most effectively to the abjective. Where the investment program is less than satisfactory, donors would be well advised to focus their efforts on technical assistance and to limit their financial assistance to projects that are likely to be nonfungible. Project Evaluation Whether In the context of a single project or a public expenditure review, project evaluation still has a vital ro.le to play. But the traditional tool analysts use—the rate of return—will be less and less relevant given die nature of the 44 The World Bank Research Observer, vol. 12, no. 1 (Febmtry 1997) projects that governments and donors will be pursuing. In these new circum- stances, two questions should be routinely addressed in evaluating every project, even when it is not possible to measure all the benefits, a characteristic that is likely to describe a growing number of public investment programs. First, what is the rationale for the public sector's involvement? Since govern- ments and donors are still financing projects that appear to be producing private goods, a greater effort should be made to establish the rationale for public pro- vision. Policymakers should routinely assess private-sector alternatives, includ- ing improved regulatory or price (tax-cwm-subsidy) policies. Even where the rationale for public ownership is clear, the objective should be to ensure that total supply is greater than would be the case with just private provision. Second, what is the cost of public provision? If countries and donors system- atically implement the preceding recommendation, the net impact on the budget will be negative. Where this fiscal cost arises from an expansion in supply be- yond what would have been forthcoming from the private sector, it represents the price that society has to pay to reap the benefits underlying the rationale for public intervention. If the government is not charging the maximum amount that the private sector is willing to pay, there is an additional fiscal cost—a transfer. Both the expansion and the transfer constitute additional burdens on the budget. To the extent that governments have to rely on (distortionary) taxa- tion, raising the required revenue will entail real costs. These costs, as well as the marginal cost of public funds, need to be incorporated in project appraisal wherever possible. Are our proposals feasible? Technically, yes. For example, the rationale for public-sector provision can be identified at the same time as the public expendi- ture review is conducted or early in the project cycle, and without amassing all the information usually associated with a full-blown cost-benefit analysis. Simi- larly, analysts should be able to provide reasonable estimates of the fiscal costs of projects without knowing all the benefits. Finally, both of these tasks can be undertaken for all projects. Notes Shantayanan Devarajan is division chief, Lyn Squire is director, and Sethaput Suthiwart- Nameput is an economist in the World Bank's Policy Research Department. The authors wish to thank Mark Baird, Pedro Belli, Jeffrey Hammer, Anandrup Ray, and several anony- mous referees for their insightful comments on an earlier draft. 1. In this article rate of return refers to both rate of return and net present value calcu- lations. 2. Note that this is true for any intervention up to Q*. At Q*, for example, the addi- tional cost ABC is strictly less than the size of external benefits beyond the market out- come, ABCD. For interventions beyond Q*, this may no longer be the case. 3. The qualification that capital-market imperfections may prevent the private sector from undertaking the project may apply here, as does the response, namely, that the ap- propriate intervention would be to remove the capital-market imperfection. Shantayanan Devarajan, Lyn Squirt, and Sethaput Suthiwart-Narueput 45 4. Analyses of infant mortality showed that expenditures on safe water and immuniza- tion had a much higher effect than expenditures on government-employed doctors (World Bank 1992). References The word "processed" describes informally reproduced works that may not be commonly available through library systems. Ballard, Charles, John Shoven, and John Whalley. 1985. "General Equilibrium Computa- tions of the Marginal Welfare Costs of Taxes in the United States." American Eco- nomic Review 75(l):128-38. Dasgupta, Partha, Stephen Marglin, and Amartya K. Sen. 1972. Guidelines for Project Evaluation. New York: United Nations Industrial Development Organization. Feyzioglu, Tarhan, Vinaya Swaroop, and Min Zhu. 1995. "The Impact of Foreign Aid on Public Spending." World Bank, Policy Research Department, Washington, D.C. Processed. Harberger, Arnold C. 1973. Project Evaluation: Collected Papers. New York: Macmillan. Harrold, Peter, and Associates. 1996. The Broad Sector Approach to Investment Lending: Sector Investment Programs. World Bank Discussion Paper 323. Washington, D.C: World Bank. Humplick, Frannie, and William Paterson. 1994. "Peru: Infrastructure Public Expenditure Review." World Bank, Latin America and the Caribbean Region, Washington, D.C. Processed. Little, I. M. D., and J. A. Mirrlees. 1969. Manual of Industrial Project Analysis. Paris: OECD Development Centre. . 1974. Project Appraisal and Planning. London: Heinemann. . 1991. "Project Appraisal and Planning Twenty Years On." In Proceedings of the World Bank Annual Conference on Development Economics 1990. Washington, D.C: World Bank. Pack, Howard, and Janet Rothenberg Pack. 1990. "Is Foreign Aid Fungible? The Case of Indonesia. Economic Journal 100(399, March): 188-94. . 1993. "Foreign Aid and the Question of Fungibility." Review of Economics and Statistics 75(2, May): 258-65. Squire, Lyn. 1989. "Project Evaluation in Theory and Practice." In Hollis Chenery and T. N. Srinivasan, eds., Handbook of Development Economics vol. 2. New York: Elsevier Science Publishing Company. Squire, Lyn, and Herman G. van der Tak. 1975. Economic Analysis of Projects. Baltimore, Md.: Johns Hopkins University Press for the World Bank. World Bank. 1992. "Malaysia: Fiscal Reform for Stable Growth." Report 10120-MA. Asia Region, Washington, D.C Processed. . 1994. 1993 Evaluation Results. Washington, D.C. . 1995. "A Review of the Quality of Economic Analysis in Staff Appraisal Reports for Projects Approved in 1993." Operations Evaluation Department, Washington, D.C. Processed. 46 The World Bank Research Observer, vol. 12, no. 1 (February 1997)