Report No. 26443 Efficient, Sustainable Service for All? An OED Review of the World Bank’s Assistance to Water Suply and Sanitation September 1, 2003 Operations Evaluation Department Document of the World Bank I Table o f Contents Foreword ........................................................................................................................................ 1 Summary ....................................................................................................................................... ... 111 1. Introduction............................................................................................................................. 1 Sector Context ..................................................... .... 1 Objectives of Study ................. .................................. 2 Study Scope and Methods .......................................... 3 2 . Bank Sector Assistance ........................................................................................................... 4 Trends and Patterns ............................. ....... ............... .... 4 Analysis of the Dedicated Water Supply and Sun ........................... 5 Internal Consistency o f Dedicated Projects ........................................................... 5 Policy Consistency Between Dedicated and Non-dedicated Projects ...................5 First Sector Objective: “Providing Service-For-All” ............................................ 6 Second Sector Objective: “Providing Sustainable Service” .................................. 7 Third Sector Objective: “Providing Efficient Service” ......................................... 8 Project Performance ............................................ Ratings o f the Operations Evaluation D Ratings o f the Operations Evaluation Department-Non-Dedicated Projects ...... 9 Water and Sanitation Project Ratings Compared to Related Sectors .................. 10 3 . The Pursuit of Sector Reform .............................................................................................. 11 The Old Assistance Paradigm ................ ................................. 11 The New Assistance Paradigm .............................................. 12 Evaluation of the New Assista Progress I n Creating Sector Progress I n Applying Pricing Policies . ................................ 15 Progress I n Private Sector P Bank Analytical and Advisory Assistance to PSP............................................... 22 OED Ratings o f Completed Projects with PSP ................................................... 23 Performance Indicators Under Bank-Assisted Projects, With and Without PSP 24 ................................................................................ 25 Efficiency o f Service-Operating Efficiency ......................................... 25 Sustainability o f Privately Operated Services-Financial Sustainability26 Environmental Sustainability ................................................................. 27 Comparing the Performance o f Projects With and Without PSP ........................ 27 4 . Findings and Conclusions .................................................................................................... 28 Progress Has Been Uneven . . . . But More Remains to be Done .......................................................... The Way Forward ................................................ . Annex A Glossary ........................................................................................................................ 33 Annex B. Dedicated Water Supply and Sanitation Lending, by Country, 1970-2001 ..........35 Annex C. Dedicated Water Supply and Sanitation Lending, by Sector, 1970-2001 ..............38 Annex D. Dedicated Water Supply and Sanitation Lending. by Region, 1970-2001 ............. 39 Annex E. Non-dedicated Water Supply and Sanitation Lending, by Country, 1990-2001 ...40 Annex F. Non-dedicated Water Supply and Sanitation Lending, by Sector and Year, 1990-2001 ............................................................................................................................... 42 Annex G. OED Ratings for Dedicated Water Supply and Sanitation Projects. 1970-2001 ..43 Annex H. Performance Indicators f o r Water Supply and Sanitation Projects ........... 45 Annex I. Bank Economic and Sector Work in Water Supply and Sanitation Sector ............47 Annex J. Lessons from Water Supply and Sanitation Project Cohort. FY98-2001 ..............48 Boxes Box 1. Performance Indicators o f Results on the Ground ................................................................ 3 Box 2 . Bolivia I s Promoting Demand-driven Rural Community Development .............................. 8 Box 3 . Chile Represents Best Practice in Pricing Policies ............................................................. 20 Box 4 . Bolivia Offers a Cautionary Lesson in Concessions .......................................................... 22 Box 5 . Colombia Shows the Way in Better U s e o f Existing Capacity .......................................... 26 Table 1. Dedicated and Non-dedicated Water Supply and Sanitation Lending. 1990-2001 ...........4 Table 2 . Objectives. Components. and Conditionality o f Water and Sanitation Lending. 1990-2001 ................................................................................................................................. 5 Table 3 . Outcome Ratings for Dedicated Water and Sanitation Projects ........................................ 9 Table 4 . ID1 Ratings for Dedicated Water and Sanitation Projects ................................................. 9 Table 5 . Sustainability Ratings for Water and Sanitation Projects .................................................. 9 Table 6 . Outcome, Sustainability, and ID1 Ratings for Project Exits, fiscal 1990-2001 ...............10 Table 7 . Few Bank Client Countries Have Regulatory Regimes for Water Supply and Sanitation ................................................................................................................................. 14 Table 8 . Choices in Pricing, Tariffs, and Subsidies Affect Achieving Bank Policy Objectives .... 17 Table 9 . PPIAF I s an Important Supporter o f PSP in Water Supply and Sanitation ...................... 23 Table 10. The Riskiest o f A l l Infrastructure Sectors ...................................................................... 23 Table 11. The Few Closed Projects with PSP Have Generally Performed Well ........................... 24 Table 12. And They Are Rated Higher Than Projects without PSP ............................................. 24 Table 13 . Projects With and without PSP Both Show Improvement ............................................ 25 Figure Figure 1. Percentage o f Population Connected to Piped Water versus Income Levels. Lima .........6 ... 111 Foreword This OED study, which i s primarily a desk review, updates the findings o f a 1992 OED review o f the Water Supply and Sanitation (WSS) sector and follows up on selected key sectoral issues identified as part o f OED’s more recent review o f the Bank’s overall Water Resources Management Strategy. Accordingly, it focuses selectively on the performance o f the WSS portfolio and the impact o f Bank assistance on regulatory and pricing reform as well as private sector participation. The study finds that portfolio performance, although improving , has remained below average and that the Bank has had limited success in promoting deep and lasting regulatory and pricing reform in the sector. It also concludes that private sector participation, where it was promoted under Bank-supported projects, has by and large shown promising results in terms o f coverage and quality o f service but that it cannot be .the panacea to the sector’s structural problems. The study’s findings point, inter alia, to the need for clarifying and consistently implementing the Bank’s pricing policy in the WSS sector, for country-focused sector strategies geared to the achievement o f the ‘ Millennium Development Goals, and for a pragmatic approach to private sector participation. Gregory K. Ingram Director-General Operations Evaluation Summary 1. From a modest start in the 1960s, World Bank water supply and sanitation lending grew to reach 9 percent o f total commitments in 1979. Subsequently, dedicated’ lending decreased to about 3 percent o f lending, but the decline was partly offset by water supply and sanitation components under non-dedicated lending categories, such as the environment, social investment funds, and urban devel- opment. Currently, the active portfolio consists o f about 100 dedicated water supply and sanitation projects, and another 1 50 non-dedicated projects with significant water supply and sanitation compo- nents. Lately the sector has attracted renewed interest because improved access to safe water and ade- quate sanitation are among the central goals established by the U ” s Millennium Summit. Three tar- gets under the Millennium Development Goals (MDGs) depend on success in expanding coverage and improving the quality o f water supply and sanitation. 2. OED last fully evaluated the sector in 1992. That evaluation concluded that the relevant sec- tor objectives had only been partially met and prompted a reformulation o f the objectives. Although the sector has not been covered by an explicit Bank strategy for the sector, in practice i t s objective has been to provide efficient, sustainable service for all, with emphasis on the poor. However, the instruments to reach this goal have been changing. From having previously worked through gov- ernment water supply and sewerage agencies the Bank adopted in the 1990s a paradigm that seeks to create regulatory capacity in the sector and encourage private sector participation. More recently, OED evaluated the implementation o f the World Bank Water Resources Strategy that was approved in 1993 and updated in 2000 as Operational Policy 4.07. That evaluation2 concluded, inter alia, that the water supply sub-sector had not sufficiently documented i t s effect on alleviating poverty; that the experience from sector regulation had not been evaluated; that the sector pricing policies were applied inconsistently; and that the sustainability o f private sector participation remained to be demonstrated. 3. Accordingly, the present evaluation, which i s primarily a desk review, follows up on these issues by focusing on four specific topics: (i)a review o f portfolio performance to detect possible effects o f the sector’s current lending paradigm; (ii)an evaluation o f regulatory reform promoted with Bank assistance: (iii)an analysis o f how sector pricing policies have been applied; and (iv) an assessment o f the effectiveness o f private sector participation (PSP) in contributing to achieving the goal o f “efficient, sustainable, service for all.” 4. This evaluation notes that OED ratings o f the outcome, institutional development impact, and sustainability o f closed projects have improved over the 1990s although the sector continues to rank low among all sectors rated by OED. Proactive portfolio management has also helped improve the performance o f the active portfolio. The proportion o f satisfactory OED outcome ratings has risen from about half to about two-thirds for projects that closed in the 1995-2001 period. The ratings for institutional development impact (IDI) and sustainability have also improved, but remain at low levels: only one-third o f the closed projects have ID1ratings o f “substantial” and less than half have sustainability ratings of“likely.” Most projects that closed in the 1990s were designed with the lending paradigm o f the 1980s, however, and it remains to be seen if the detected improvements w i l l continue for projects that were designed with the new lending paradigm o f the 1990s. 5. OED concludes that the Bank‘s efforts to introduce effective regulation have failed to produce lasting results. There are few well-functioning water supply and sanitation regulatory agencies in 1. Dedicated water supply and sanitation lending i s under prqjects mapped to the Water and Sanitation Sector Board. Non- dedicated projects are mapped to other sector boards within the World Bank. 2. OED, 1jrin’giH.p 7?0l/h/cd Il’rilers .Assessinp / / r e ICorid Hirnfi IV’otcr, Resources Strate.w.Washington DC: World Bank, 2002. vi developing countries, and practically none in the Middle East and North Africa, Europe and Central Asia, and South Asia Regions. The best-performing regulatory agencies, such as in Chile and Colombia, were establishedwithout any Bank assistance. Both quality regulation and economic regulation are largely absent elsewhere. OED finds that the eight sector performance indicators adopted in 1999 by the Bank Water and Sanitation Sector Board have not been effectively implemented by the Bank and its borrowers. 6. The absence o f tariff regulation in Bank member countries can partly be explained by the Bank’s inconsistent application o f its long-standing pricing policies (recently reiterated in the Water Resources Management Policy, OP 4.07). These three objectives-economic efficiency, financial cost recovery, and targeted subsidies to the poor-are rarely met. Often pricing policies are internally inconsistent, and water supply and sanitation operators lack the capacity and incentives to apply them effectively. Prevailing pricing policies have not served the poor well. The poor would best be served by universal connection coverage to the public water supply systems, but their low incomes make it necessary to subsidize connections. Where water supply and sanitation are under-priced, the wealthy, who consume the most water, are the most subsidized. In contrast, the poor who consume little water and are often unconnected, receive lower or no subsidies, both in relative and in absolute terms. There i s a general consensus regarding what needs to be done-universal metering, charging tariffs that reflect future costs in accordance with economic principles, subsidizing connections to the poor, and targeting consumption subsidies to the poor-but these steps have been applied only erratically. 7. Preliminary results o f the Bank’s efforts to promote private sector participation (PSP) are encouraging. About one-third of projects approved in the 1990s had PSP as an objective. OED rated 86 percent o f the PSP projects that closed in the 1990s as having a “satisfactory” outcome versus 60 percent for projects without PSP, 57 percent as having “substantial” ID1 (compared to 28 percent), and 71 percent as having “likely” sustainability (versus 3 5 percent). Data compiled for a limited sample o f water utilities before and after PSP also point to improvements in performance. However, many projects without PSP recorded similar improvements in the performance indicators. Clearly, PSP cannot be considered a panacea to the sector challenges, nor the only option. Bank assistance to the sector should consider solutions within the continuum between fully public and fully private provision, which are most suited to individual country circumstances, as both private and public approaches have been shown to produce good results if properly designed and supervised. 8. OED concludes that, despite the noted shortcomings, the sector has made some, albeit un- even, progress during the 1990s by adopting a more relevant sector development paradigm that fo- cuses on “efficient, sustainable service-for-all.” A number o f emerging findings are relevant in the design and implementation o f strategies to meet the three MDGs that involve the provision o f safe water and adequate sanitation. Finding One: Monitoring Performance Indicators and Evaluation Systems Will Be Necessary to Track the Bank’s Progress in Helping Its Member Countries Achieve the MDGs 9. The MDGs are broad and general and cannot serve as a primary basis for planning and imple- menting detailed country/sector strategies. Instead, the Bank should revisit and adapt the performance indicators adopted for the water supply and sanitation sector in 1999 and use them to monitor pro- gress toward achieving the MDGs. Without continuous estimates o f service coverage and quality, ef- ficiency, and likely sustainability, it will be difficult to formulate credible development strategies. A shift in resources i s necessary to assist the Bank‘s member countries in creating monitoring systems that will help make service providers and financing agencies accountable for progress toward meeting the MDGs. vii Finding Two: Regulation Must Move from Prescription to Implementation 10. A decade after beginning the effort to create regulation, only a handful o f countries have put in place well-functioning systems o f quality and economic regulation in water supply and sanitation. A n enhanced system o f performance indicators i s needed to provide incentives for countries to strengthen their regulatory frameworks, since well-designed quality regulation would enable plan- ners, politicians, and managers to benchmark their systems or sectors in terms o f coverage, quality o f service, and efficiency. 1 1. A push toward economic regulation i s equally overdue. The review o f pricing policies under Bank projects has revealed that predictable tariff regulation through the consistent application o f transparent rules does not exist in any country that i s an active Bank borrower. In the few countries where de j u r e tariff regulation does exist, it can rarely withstand pressure to conform to short-term political expediency. This diminishes its value to investors and operators since the tariff can no longer be predicted with any measure o f confidence. The performance o f Bank loans would benefit from a policy o f requiring either institutionalized tariff regulation or covenanted tariffs as a condition o f lending. A minimum requirement should be to adjust the tariff annually to compensate for inflation. Finding Three: Ensuring that the MDGs are Achieved Will Require Translating them into Imple- mentable Sector Development Strategies 12. The development community expects to meet the MDGs by 2015, that is, 12 years from now. Despite the relative imminence o f the target year, most countries lack sector strategies that describe the present situation, and analyze policy reforms, pinpoint priorities, and pre-identify priority pro- jects. The Bank i s well placed to assist its member countries in preparing country-specific sector strategies geared to achieving the MDGs. But this will likely require a shift o f resources toward operational economic and sector work. I n the past decade, 1990-2001, about 90 pieces o f economic and sector work (ESW) were prepared by the Bank, but only one tenth were sector studies directly aligned with sector development. Future ESW will have to be more closely linked to lending. Finding Four: Private Sector Participation has Shown Promising Results and Remains an Impor- tant Tool to Improve Coverage and Quality 13. The review concludes that private sector participation has been effective in raising service coverage and quality. However, it has not been a panacea to the sector’s problems, which has prompted a recent reassessment o f its use. The wholesale questioning o f the justification for PSP i s premature, however, but shows that the unrealistically high expectations from clients and Bank staff must be tempered. A continued careful evaluation o f each PSP-led program i s needed to identify fac- tors o f success and failure. Continued promotion o f private sector participation i s sensible as PSP of- fers the promise o f more transparent performance by operators; faster gains in coverage, quality, effi- ciency, and sustainability; and a further stimulus o f regulation. Finding Five: Operators Require Special Incentives to Serve the Poor 14. A n early lesson from the experience o f all types o f contracts, including those with PSP, i s that they will mostly be evaluated by the stakeholders based on how service improved for the poor. To this end, governments will have to ensure effective regulation and provide transparent subsidies linked to service provision to the poor. Output-based aid i s one promising model, the results o f which will need to be carefully evaluated. 1 “When you can measure what you are speaking about and express it in numbers, you know something about it; but when you cannot measure it, when you cannot express it in rmmbers, your knowledge is of a meager and unsatisfactory kind” Lord Kelvin, 19th century physicist 1. Introduction SECTOR CONTEXT 1.1 From a modest start in the 1960s World Bank water supply and sanitation sector lending grew to reach 9 percent o f total commitments in 1979. Subsequently, dedicated lending decreased to a level o f about 3 percent o f total commitments. The relative decline has been partly offset by non-dedicated lending for water supply and sanitation under a number o f other sector categories, such as the environment, social investment funds, and urban development. At present, the active portfolio consists o f about 100 dedicated water supply and sanitation projects, and another 150 non-dedicated water supply and sanitation projects. 1.2 Lately the sector has attracted renewed interest because improved access to safe water and adequate sanitation are among the goals established by the U ” s Millennium Summit and con- firmed and strengthened at the 2002 Johannesburg Summit. Three o f 18 targets under the Millen- nium Development Goals (MDG) depend on expanded and improved water and sanitation services: 0 Reducing by two-thirds, between 1990 and 2015, under-five mortality rates (Target 5); 0 Halving by 20 1 5 the proportion o f people without sustainable access to an improved water source (Target 9); and Halving by 20 15 the proportion o f people without access to improved sanitation (Target 10 as amended in the Johannesburg Summit in August 2002). Interest in supporting the sector has also benefited from recent studies showing that water supply, sanitary excreta disposal, and health education are cost-effective ways to reduce the incidence o f diseases4 In particular, the importance o f sanitary excreta disposal and improve hygiene habits i s confirmed as critical for achieving lasting health improvements. 1.3 The Bank’s sharpened focus on poverty alleviation as its main mission i s evident from the sector objectives listed in the previous OED evaluation o f the sector in 1992. At that time, the first three objectives were: to help governments achieve least-cost solutions to infrastructure needs; to foster institution building; and to help institutions achieve financial viability. Only the fourth objective stated that projects should “ensure the availability o f a minimum supply o f safe water to the poor.” 1.4 Operational Manual Statement 3.72, “Energy, Water Supply and Sanitation, and Telecommunications,” o f September 1978 i s the last comprehensive statement o f what should guide Bank lending in this sector. The OMS has not been updated, but the Bank’s sector agenda 3. Traditionally, World Bank sector lending has been divided into dedicated, exclusive water supply and sanitation lending that i s managed by water supply and sanitation divisions and non-dedicated lending that i s managed by other divisions and where water supply and sanitation i s part o f multi-sectoral projects. The definitions are somewhat imprecise but nevertheless useful. 4. As cited in Kseniya Lvovskq. / / m / d i urid k h i r o n r ~ g g Report ~ No. 24096, Washington DC: World Bank, 2001. 2 has evolved as reflected in Bank-sponsored conferences, training events and the 1994 World Development Report on Infrastructure. A consensus has emerged on the key principle underlying the Bank’s approach to the water supply and sanitation sector: to promote the provision o f efficient, sustainable service-for-all. The implications would be that the World Bank strives to design and finance projects that will meet demand at the least possible economic cost (“efficiency”); that it will try to ensure that services will remain operational and produce economic benefits for the foreseeable future (“sustainable”);and that the entire population, including the poor, will be served by the water supply and sanitation utilities (“service-for-all”). 1.5 I n 1993, the Bank issued a Water Resources Management Policy Paper that became the basis for the Bank’s Operational Policy 4.07. Although i t s main focus i s on water resources management, OP 4.07 reiterates the importance o f pricing and incentive policies to achieve cost recovery and water conservation. It also underlines the necessity o f establishing a strong legal and regulatory framework to meet social concerns, protect the environment, and prevent monopoly pricing. A recent OED evaluation o f the implementation o f that policy, Bridging Troubled Waters: Assessing the Bank’s Water Resources Strategy Since 1993 (OED 2002), identifies several water supply and sanitation sector issues that warrant further evaluation: The sector has not documented its effect on ensuring safe water and adequate sanitation to the poor. 0 The experience froin sector regulation in developing countries i s not properly evaluated. 0 The pricing policies in the sector are ambivalent between satisfying efficiency and financial performance and facilitating the consumption o f the poor. The long-term sustainability o f private sector participation and its effect on meeting the needs o f the poor are not demonstrated. OBJECTIVES OF STUDY 1.6 The starting point for the evaluation would normally be to evaluate the implementation o f the Bank policy that applies to the water supply and sanitation sector. However, such a benchmark i s not available because there i s no formal comprehensive and updated policy and strategy for the sector. The closest approximation to a policy may be the implicit objectives o f Bank assistance that emerged over the 1990s, namely to put the sector on the path to “effective, sustainable service for all.” (See Annex A for a Glossary o f Terminology). Such objectives are equivalent to ensuring that projects meet demand at the least possible economic cost (“efficiency”); that they will be able to provide services over the foreseeable future (“sustainability”); and that the entire population, including the poor, will benefit from the services. (“service-for-all”). The first objective o f the study, therefore, i s to assess whether Bank assistance has indeed helped its client countries in moving towards “efficient, sustainable service-for-all.” 1.7 Additionally, the study aims at following up on several key issues identified in OED’s recent report Bridging Troubled Waters, namely: 0 Whether the Bank‘s assistance has promoted effective sector regulation in i t s member countries; 0 Whether OP 4.07 has been implemented as regards pricing policies under water supply and sanitation projects; and 0 What the results have been o f the Bank’s effort to support private sector participation. 3 STUDY SCOPE AND METHODS 1.8 This evaluation was designed primarily as a desk study. I t i s based on an analysis o f existing OED databases, data generated by OED project evaluations, and data obtained through electronic mail surveys. The evaluation has been divided into four main areas. The first part o f the evaluation i s a portfolio review that tracks quantitative and qualitative trends in the portfolio and in projects that have been rated by OED or by the Quality Assurance Group (QAG) o f the World Bank. The portfolio review comprises 304 water supply and sanitation projects o f which 142 dedicated and 162 noli-dedicated, representing the totality o f projects approved in the 1990-2001 period. The portfolio review includes both urban and rural water supply and sanitation projects but the focus o f the evaluation i s predominantly urban.' Similarly, the evaluation focuses more on water supply aspects than on sanitation aspects given that water supply lending has been many times larger than sewerage .and sanitation lending. (See Table 1.) Special challenges faced by sanitation projects may be assessed in the context o f future OED evaluations. 1.9 The portfolio review was complemented by collecting project performance indicators prompted by the adoption o f the Comprehensive Development Framework in 1999. (See Box 1.) The source o f the performance indicators has been OED assessments o f 13 individual projects and a special OED review o f the Bank's Assistance to the China urban water supply and wastewater sector covering 1 8 urban water supply and wastewater projects.6The project assessments were prepared during the 1999-2001 period. Box 1. Performance Indicators of Results on the Ground Following the 1999 launch o f the Comprehensive Development Framework, the Bank's water supply and sanitation sector board developed performance indicators to enable monitoring o f progress toward "efficient, sustainable service-for-all," primarily in urban areas. A similar set o f indicators was also elaborated for rural and peri-urban areas where connections often are too expensive and where low-cost on- site technologies are often more appropriate in the early stages o f development. The urban water supply and sanitation performance indicators comprise: Service-for-A 11: Share o f population connected to public water supplies Share o f population connected to public sewerage Share o f population with continuous service Share o f population with bacteriologically safe water Efficiency of Service: Percentage unaccounted water Number o f employees per thousand households connected to public water supplies Sustainability of Service: Working ratio (cash operating costs/cash operating revenue)-financial sustainability Share o f wastewater treated-environmental sustainability. 1.10 The second part o f the evaluation maps the Bank's success in bringing about an improved lending paradigm comprising (i)effective and predictable regulation; (ii)transparent pricing 5. The present evaluation bas limited mainly to urban water supply and sanitation projects supported by the World Bank Group as OED had recentl) evaluated the segment o f rural water supply and sanitation projects, the findings o f which are reported in i i u i d Ilh/cJr/'in;cc~.s.&SSOIIS /rcim OElI Eixziunfions. OED Working Paper No. 3, Washington DC: World Bank, March 2000. 6. "Chinii-Rcvicw of thc I3;ink'.:&sistance to thc I.!rban Water S i i m l y and Wastewater Sector." OED Report No. 24979, Washington DC: World Bank, October 16, 2002. 4 policies that promote efficient water consumption and production, favor financial autonomy, and provide the poor with access and a minimum lifeline consumption. The evaluation o f regulatory arrangements benefited from a separate study conducted by Norwegian consultants under a grant from the Government o f Norway. The consultants reviewed the portfolio o f Bank water supply and sanitation projects with regulation components, interviewed Bank staff, and visited the Philippines and Zambia, both o f which had received Bank assistance to create regulation in the water supply and sanitation sector. A n intended visit to Gaza and the West Bank to study the monitoring o f a Bank water supply project there proved impossible for security reasons. 1.1 1 The evaluation o f the implementation o f the pricing policies contained in OP 4.07 used a questionnaire that was mailed to the respective Bank task managers responsible for 24 dedicated and 24 non-dedicated water supply and sanitation projects with questions pertaining to the inclusion and implementation o f OP 4.07 with regard to pricing. A total o f 15 replies were received and form the basis for the evaluation o f the pricing policies. 1.12 The Bank's success in assisting its client countries in widening and deepening private sector participation (PSP) in the water supply and sanitation sector has also been evaluated. The sources o f data for this evaluation were drawn from: (i)all projects with PSP that had been closed and assessed by OED since 1997; and from (ii) a special mail questionnaire that was sent out to the task managers o f 20 water supply and sanitation projects that had received Bank financial and technical assistance in support o f PSP. A total o f 1 1 complete replies were received to this mail questionnaire. 2. Bank Sector Assistance TRENDS AND PATTERNS 2.1 Total dedicated and non-dedicated Bank (IBRD and IDA) lending for water supply and sanitation in the 1990-2001 period i s shown in Table 1. Bank commitments for water supply and sanitation amounted to about $13.3 billion under 304 projects- $9.8 billion (74 percent) under 142 dedicated projects and $3.5 billion (26 percent) under 162 non-dedicated projects. A portion o f the latter are non-dedicated mostly in name. For instance, the China water supply and sanitation portfolio i s categorized as environmental but has been managed like a dedicated portfolio. The situation may be similar for some o f the urban development WSS projects. Table 1. Dedicated and Non-dedicated Water Supply and Sanitation Lending, 1990-2001 ,_^ --.I_L _yx_c I_(*I * _ L- 1- . x __--I*_ # Projects as ~ ~ Sector # Projects Commitments Commitments as share of total (%) (Constant US$ share of total (%) _I__- million) Dedicated WSS Urban Water Supply 52 17 1% 4,063 30 7% Sewerage 26 8 6% 1,821 13 7% Water San Adjustment 24 7 9% 1,628 12 3% Other WSS 16 5 3% 1,142 8 6% Rural WSS 24 7 9% 1,079 8 1% Total Dedicated 142 46.7% 9,733 73.5% Non-Dedicated WSS Urban Development 57 18 7% 1,646 12 4% Environment 21 6 9% 954 7 2% Social Protection 55 18 0% 533 4 0% Agriculture 17 5 6% 154 12% Other Sectors 12 3 9% 225 17% Total Non-dedicated 162 53.3% 3,512 26.5% GRAND TOTAL wss I l I " " cl l I *-*.-. 305 I 100.0% 13,292 100.0% 5 Table 2. Objectives, Components, and ANALYSIS OF THE DEDICATEDWATER SUPPLY AND SANITATION PORTFOLIO Conditionality o f Water and Sanitation Projects, Loans and Credits Approved in 1990-2001 .,”yI-.l.^-. 2.2 The analysis o f the dedicated and ~ Dedicated Non-dedicated non-dedicated W-SS portfolio attempts to WSS Projects WSS Projects . .. __ Share of Total Share of Total answer the following questions: Totd 100% 100% OBJECTIVES What have been the sector WSS Expansion 75% 78% Institutional Development 74% 52% objectives and to what extent Better Cost Recovery 40% 15% have they supported efficient, Private Sector Part 31% 0% Policy Reform 17% 6% sustainable service-for-all? Regulatory reform 8% 0% What have been the project COMPONENTS components? WSS Expansion 98% 99% Institutional Development 85% 76% What has been the conditionality Cost Recovery 76% 47% o f loans? TA for Policy Reform 43% 14% Private Sector Part. 38% 3% What has been their performance? TA for Regulation 14% 0% COVENANTS Each o f the 142 dedicated projects has WSS Covenants 100% 40% Tariff Covenants 52% 22% been analyzed within this framework and Institutional Development 33% 15% the composite analysis i s summarized in Improved metering, 19% 2% billings & collections Table 2. Private Sector Part. 17% 0% Policy Reform 15% 1% Improved financial 11% 6% Internal Consistency o f Dedicated management systems Projects Regulatory reform _“*----- 1% 0% Note. Percentages do not total to 100% under each category as many projects have multiple objectives. 2.3 It i s reasonable to expect consistency between stated objectives, loan components, and loan conditionality because objectives usually need to be supported by sufficient financing and loan conditionality to stand a chance o f being achieved. However, the analysis indicates that such internal consistency i s not always the case. Objectives are often quite general and lack legal covenants to underpin project implementation. For instance, better cost recovery i s a stated objective for 40 percent o f the dedicated water supply and sanitation projects, and although 76 percent o f the projects provide financing to raise cost recovery only 52 percent have tariff covenants and only 19 percent have covenants to help improve metering, billings, and collections. Similarly, 8 percent o f the projects list the establishment o f regulation as an objective, 14 percent provide financing to make it happen, but only 1 percent has legal covenants to support the effective implementation o f sector regulation. 2.4 The consistency between project objectives, components, and covenants with the objectives o f “efficient, sustainable service-for-all” i s also weak. “Efficient” sector development i s facilitated by higher cost recovery, but only 40 percent o f the dedicated projects contain this objective. “Sustainable” sector development requires institutional strengthening, higher cost recovery, and regulation but only 74 percent, 40 percent, and 8 percent o f the dedicated projects, respectively, have associated covenants to assist and enforce changes in these three areas. Finally, although the Bank’smain objective i s to promote ”service for all,” and particularly for the poor only 75 percent o f the projects list explicitly the objective o f raising coverage and improving service quality. Policy Consistency Between Dedicated and Non-dedicated Projects 2.5 Policy consistency between dedicated and non-dedicated projects i s similarly lacking. For example, private sector participation with regulation seems to be pursued only under dedicated projects. Likewise, policy and strategy reform ranks considerably higher among the objectives, 6 components, and conditionality for dedicated projects compared to the non-dedicated ones. The reason for the differences could be that a non-dedicated loan covers a number o f sectors and each sector may receive less attention than it would in dedicated, stand-alone projects. The discrepancies point to the inconsistent policies where water supply and sanitation investments could have widely varying lending conditions and policies depending whether they are financed under dedicated or non-dedicated projects. The unintended result could be to undercut those projects that pursue a more ambitious reform agenda as distinct from those that limit themselves to financing physical works. Lack o f data prevents testing this hypothesis, however. Assuming the trend toward a growing share o f non-dedicated sector lending continues, the challenge o f ensuring policy consistency across the entire water supply and sanitation sector w i l l intensify. First Sector Objective: “Providing Service-For-All” 2.6 The overarching sector objective i s to provide “service-for-all” and i s practically a universal component o f both dedicated and non-dedicated projects. (See Table 2.) It i s less often listed as an objective, showing up in only about one-quarter o f either category o f projects. This could lead to a weakness in project monitoring because unstated objectives are less likely to be tracked through the use o f monitoring indicators. 2.7 The same lack o f precision applies to service provision to the poor. Even though the focus o f a project may be stated as connecting poor households or improving their service quality, the objective i s not often made explicit in terms o f number o f additional poor consumers served. In general, i t i s true that increasing connection rates benefit the poor proportionately more. Wealthier households can be expected to be connected before poorer households. Figure 1 shows how connection rates increase with average income in the city o f Lima, Peru. The curve shows how connection rates for the lowest income segments are around 70 percent and how the connections rates then rise to reach 100 percent froin a certain household income level and onwards. I t follows that the urban poor will be the last to get connected. In order to track service levels o f the urban poor it i s therefore essential that each project track total service connection rates and that each project clearly state how many more consumers are expected to be connected. Figure 1, Percentage of Population Connected to Piped Water versus Income Levels Lima, Peru 70 30 J so 00 50 00 I 40 00 10 00 20 00 I O 00 , 5b0 00 ‘00’1 03 150C 00 2000 00 2500 00 Ibg‘Pra ram E 7 2.8 Arguably, poor consumers benefit most of all making water service continuous 24 hours a day and from improving the bacteriological quality o f the water.’ All consumers incur costs to cope with unsatisfactory service. For those who can afford it, the coping costs include storage tanks, alternative water supplies, and pumping and treatment facilities. The poor who cannot afford these investments pay through the time they spend fetching water from public taps. Compounding the difficulties o f the poor i s the fact that such water often risks being contaminated and the cost in time and disease often exceeds what wealthier neighbors pay for water from house connections. Coping costs provide a lower boundary to willingness-to-pay for water but are seldom quantified. 2.9 The kind and level o f investments that a household makes to cope with intermittent service differ substantially by income and by alternative water sources. A study in Dehra Dun, a city o f 300,000 inhabitants in the state o f Uttar Pradesh, India, has confirmed that families in all income brackets pay significant costs to cope with the inconvenience o f intermittent supplies. The average household spent $34 per year, o f which $22 was coping costs and $12 was payments to the utility. In contrast, poor households spent an average $69 per year to cope, almost entirely in the form o f time spent drawing “free” water from public taps. 2.10 The review o f project completion reports reveals that Bank projects fail to clearly quantify how many consumers are expected to move from intermittent to continuous service by project completion. Under some projects “improved service quality” i s listed as an objective but i s rarely quantified. Consequently, the outcome o f completed projects may be rated “satisfactory” by OED although on average only four hours o f service i s provided.’ The “satisfactory” rating i s likely the result o f the failure to include continuous service among the project objectives. All major cities in India report daily water availability to consumers in the order o f four hours.” The failure to pursue a target o f continuous service after three decades o f Bank assistance begs the question why this has not been better documented in the Bank’s economic and sector work.” Second Sector Objective: “Providing Sustainable Service” 2.1 1 A number o f objectives aim at ensuring sustainable and safe services, Le., uninterrupted operations and adequate maintenance. Well-maintained water supply and sanitation systems will be able to provide benefits over very long periods, certainly 50 years or more. Four o f the six most frequent objectives contribute to improving sustainability: institutional development, better cost recovery, private sector participation, and regulatory reform. A clear distinction emerges between dedicated and non-dedicated projects, where dedicated projects pay greater attention to those aspects that underpin sustainability. (See Table 2.) 7. Some utilities may face a trade-off between (a) increasing the number o f connections, and (b) maintaining 24-hour service and high water quality. Revenue and sustainability improve with more connections, which may drive demand above the capacity to provide 24-hour service. The issues are not as straightforward as the public health trade-off. Water in a pipe i s less contaminated than what a vendor sells, but a vendor can supply more households if the utility restricts the number o f connections to what i t can serve with a 24-hour supply. 8. Kyeong Ae Choe, Robert C.G. Varley, and H.U. Bilani, (’oi?ine wilh In~ernzi/tentWater Sunrilv: Problenzs and Prospects, Environmental Health Prqject, Activity Report No. 26: USAID, October 1996. 9 . For example, in the Hyderabad Water Supply and Sewerage Project, Loan 3 181-IN and CR 21 15-IN o f 1990. 10. /uciia--(:oicntri, I;r~me\~o,-/i in Infrustructurc, paper presented at the Conference on tor Priviite P~i.ricil?iition Private Sector Participation in India Infrastructure, November 12-13, 1999, New Delhi, sponsored by the World Bank and Public-Private Infrastructure Advisory Facility, page 27. 1I. Some estimates have been made though. e.g.. in C. Brandon and K. Hommann, “The cost o f inaction: valuing the economy-wide cost o f environmental degradation in India,” paper presented at the Modeling Global Sustainability conference, United Nations University, Tokyo: October 1995. 8 2.12 Sustainability i s most o f all about the quality o f the institutional arrangements, in both urban and rural projects. Two recently closed rural water supply and sanitation projects in Bolivia offer a relevant comparison o f the virtues o f creating sound institutions and policies in order to raise the probability o f sustainability (Box 2). Box 2. Bolivia I s Promoting Demand-driven Rural Community Development In the 1990s the Bank supported rural water supply and sanitation investments under two projects. The first was the non-dedicated Social Investment Fund (Credit 2532-BO); the second was the dedicated Rural Water and Sanitation Project (Credit 2806-BO). Initially the two projects had quite different strategies to improve rural water supplies and sanitation. The Social Investment Fund concentrated on rapidly responding to demand by building systems, while PROSABAR devoted substantial resources to first organizing and training communities before authorizing construction under the Social Investment Fund as the implementation agency. In the end, the PROSABAR method proved the superior approach for achieving sustainability, so much so that the sub-projects that had been built by the Social Investment Fund were retrofitted with additional resources spent on community development and training, using the PROSABAR methodology. Once more the principles o f best practice for successful rural water supply and sanitation projects were demonstrated: investments need to be driven by demand and match the community’s willingness to pay for operations and maintenance; and each beneficiary community has to be trained to operate and maintain the system; and for full benefits to accrue, safe water, sanitary excreta disposal, and health education need to be provided as a package. These findings confirm the conclusions o f earlier OED impact assessments o f rural water supply and sanitation projects, contained in the report Rural Water Projects: Lessons from OED Evaluations, OED Working Paper No. 3, March 2000. Third Sector Objective: “Providing Efficient Service” 2.13 Service efficiency matters because lower costs facilitate sustainability when beneficiaries pay tariffs to cover costs. However, higher efficiency does not appear among the six most important project objectives. (See Table 2.) I t is often included though as an auxiliary expected outcome resulting from institutional development, improved metering, reduced unaccounted water and the like. In fact, only 15 percent o f the 142 dedicated loans and credits in the 1990- 2001 period l i s t efficiency and reduced water losses among their objectives or components. The low proportion might reduce efficiency gains. With regard to the 162 non-dedicated projects approved in the same period, efficiency i s mentioned only once (in relation to collections efficiency). In short, efficiency seems to be o f little concern under dedicated water supply and sanitation projects, and o f practically no concern under non-dedicated water supply and sanitation projects. The absence o f covenants stipulating higher efficiency i s juxtaposed with covenants expecting customers to pay higher tariffs in 52 percent o f the dedicated and 22 percent o f the non- dedicated projects. The risk i s that consumers may end up paying for inefficiencies. PERFORMANCE PROJECT Ratings o f the Operations Evaluation Department-Dedicated Projects 2.14 The majority o f projects prepared in the 1990-2001 period have not yet closed, which makes i t difficult to evaluate if the lending paradigm o f greater private sector participation and regulation has improved ratings for project outcome, institutional development impact, and sustainability. The majority o f projects that did close in 1990-2001 were prepared in the late 1980s and the associated OED ratings for dedicated water supply and sanitation projects are summarized in Tables 3-5. 9 2.15 The OED outcome ratings for closed projects Table 3. Outcome Ratings for show a slight increase from ratings o f 58 percent in the Dedicated Water and Sanitation 1980s to 64 percent in the 1990s (Table 3). Project Projects ambitions and complexity have undoubtedly increased over time and it has become more difficult for projects to Decade 198G-89 1990-01 achieve satisfactory outcome ratings. Satisfactory 58% 64 % Unsatisfactory 39% 34 % 2.16 Similarly, the OED ratings o f institutional development impact have stabilized with roughly one- Not Available 0% 2% third o f the closed projects being rated “substantial” on Not Rated 3% 0% IDI, about 40 percent being rated “modest,” and about one-quarter being rated ”negligible” (Table 4). These ratings should be seen in the light o f the difficulty and fragility o f achieving large institutional changes during a Table 4. ID1 Ratings for project’s relatively short median implementation period Dedicated Water and Sanitation o f six years. I t should also be noted that OED did not Projects start rating ID1 and sustainability separately until the Decade 198G89 199&01 mid-1980s. For this reason, trends in ID1 ratings are only Substantial 32% 32% meaningful for those projects that were evaluated in the Modest 30% 43% latter part o f the 1980s and the 1990s. Negligible 24% 23% 2.17 For the same reasons, the OED ratings o f Not Rated 14% 2% sustainability can be tracked in a meaningfLl fashion only Total 100% 100% for projects that OED evaluated in the 1980s and the 1990s. I t shows roughly 40 percent o f the projects closed in the 1990-2001 period having a sustainability rating o f Table 5. Sustainability Ratings “likely,” approximately 25 percent having an ”unlikely” for Dedicated Water and rating, and the remaining 3 5 percent having an Sanitation Projects “uncertain” or “not rated” sustainability (Table 5). Decade 198&89 199&01 Sustainability i s undoubtedly the most difficult aspect to Likely 32% 40% rate in a consistent fashion, and i s also the most difficult Uncertain 36% 26% to use for comparisons with the corresponding ratings for Unlikely 19% 30% other sectors. Not Rated 13% 4% 2.18 Quality at entry and for quality o f supervision are Total 100% 100% rated on annual basis by the Quality Assurance Group (QAG). The small sample size o f fewer than five projects rated annually explains why annual QAG ratings vary considerably from year to year. I t i s more representative to use multi-annual averages as a basis for quality at entry and for quality o f supervision. The resulting quality-at-entry rating o f the Bank performance in preparing water supply and sanitation projects was 91% satisfactory for the average o f F Y 97,98.99 and 01 as compared to the Bank-wide average for all sectors o f 88%. Analogously, the four-year average rating for the Bank performance in supervising projects under implementation was 75% for the water supply and sanitation sector as compared to the sector-wide average o f 78 percent. Quality-at-entry i s highly correlated with satisfactory ratings o f the outcome o f completed projects and it might be expected that the convergence o f QAG quality-at-entry ratings for the water supply and sanitation sector with those o f other sectors will in time translate into a similar convergence in the ratings for project outcome. Ratings o f the Operations Evaluation Department-Non-Dedicated Projects 2.19 The corresponding ratings o f outcome, IDI, and sustainability are not readily available for water supply and sanitation investments financed under non-dedicated projects because ratings 10 are not provided for individual components under multi-sectoral projects. Nevertheless, a sample o f 41 non-dedicated project completion reports over the 1990-2001 period were selected for textual analysis to infer ratings. Many proved impossible to rate, further reducing the effective size o f the sample, so the results are only indicative. On this basis, 46 percent o f non-dedicated projects would have an outcome rating o f “satisfactory,” compared to roughly 60 percent for dedicated projects. The ID1 ratings diverge much more: only 7 percent for non-dedicated projects were rated “substantial” for ID1compared to 32 percent o f the dedicated projects. The higher ID1 for dedicated projects is not surprising given that 74 percent o f the dedicated projects had ID1as an objective, compared to 52 percent o f the non-dedicated projects. (See Table 2.) Similarly, 24 percent o f the non-dedicated projects were rated as having a “likely” sustainability, compared to 40 percent o f the dedicated projects.’* Again, the difference in ratings i s readily explained by that fact that dedicated projects include objectives such as cost recovery, policy reform, private sector participation, and regulatory reform more frequently than non-dedicated projects do. Water and Sanitation Project Ratings Compared to Related Sectors 2.20 OED’s three core ratings, outcome, sustainability, and institutional development impact (IDI), are tabulated in the Annual Review o f Development Effectiveness (ARDE) and can be compared for different sectors. The ratings for projects exiting the portfolio in the 1990s are shown in Table 6 for selected sectors that share characteristics with the water supply and sanitation sector, either because they share the characteristics o f economic goods with the water sector (the power sector) or the social goods characteristics (environment, HNP, and social sectors). The ratings vary much with the annual sample sizes in each o f the five categories studied. I n order to smooth the variations three cohorts o f project closings have been analyzed: FY 90-94; FY 95-98; and FY 99-01. Out o f the 480 projects rated in the ARDE over the fiscal 1990-2001 period, the power sector accounts for 3 8 percent, the environment sector for 6 percent, the health, nutrition, and population (HNP) sector for 22 percent, the social sector for 11 percent, and the water supply and sanitation sector for 23 percent. As could be expected, the smaller sample sizes for the environment and social sectors result in larger variations in cohort ratings whereas the large sample sizes for the power and HNP sectors result in more stable ratings. The variance o f the water supply and sanitation sector ratings i s in between: i t s outcome ratings have varied considerably while those o f sustainability and ID1have been more stable. Table 6. Outcome, Sustainability, and ID1 Ratings for Project Exits, fiscal 1990-2001 x_-____ 1 - 1 1 y“~-”_.l Sector - - Exit FY 90=_94 ___lll_ __ ____Outcome SustaEabiljty ID[ Power 64% 57% 30% 66% Environment - - HNP 69% 45% 17% 65% Social Sector 75% 50% 75% 83% Water&San --*-- 70% Legend -- “ ** ” .uI y 33% _u ID1 = Institutional Development Impact HNP = Health. Nutrition and Population Outcome Ratings refer to share o f project exits rated “satisfactory” or higher Sustainability Ratings refer to share o f projects rated “likely” or higher sustainability ID1 Ratings refer to share o f projects rated “substantial” or better 12. A recent OED evaluation o f social funds ( S i c i d b‘7,nds .A.sscwiw Ef7i;ctivenes.y. Washington DC: World Bank, 2002) concluded that the uater supply and sanitation sub-prqjects faced greater problems than those o f other sectors in reaching sustainability. In particular, the evaluation reported greater problems with respect to the quality o f construction and with respect to inadequate mechanisms to collect funds to pay for the costs o f operations and maintenance. 11 2.21 The water supply and sanitation sector’s dual “utility/social” nature has a series o f implications for the way governments set tariffs. The sector’s social and political nature may result in financially unsustainable tariff policies, which make it difficult to attain financial and institutional autonomy and viability. Therefore, the sustainability o f a utility that i s unable to cover its operation and maintenance costs from revenue i s usually rated as unlikely because OED includes financial res; lience as an important component in its sustainability ratings. In effect, this means high or likely sustainability i s harder to achieve in the water supply and sanitation sector than in sectors that depend on government budgets to pay for recurrent costs (e.g., environment, HNP, and social sectors, in the sample above). I n the end and although the absolute OED ratings for the water supply and sanitation sector improved from the fiscal 1995-98 cohort to the fiscal 1999-2001 cohort (“satisfactory” outcome rose from 49 percent to 64 percent; “likely” sustainability rose from 37 percent to 47 percent; and “substantial” ID1rose from 23 percent to 33 percent), the sector continues to rank last or next to last relative to other sectors. 3. The Pursuit o f Sector Reform THE OLD ASSISTANCE PARADIGM 3.1 The Bank assistance paradigm to the sector changed l i t t l e during the 1970s and the 1980s. I t was a development model that relied on the public sector to meet consumer needs at the least possible cost and in a sustainable manner. Gradually the disadvantages o f relying exclusively on the public sector became apparent. The faltering performance o f the Bank portfolio was evident from the many deviations from sound sector policies and management practices. Some governments may have been prompted to search for alternatives to the old sector development paradigm by these risk factors: 0 The risk that the monopoly power inherent in allowing only public sector agencies to implement and operate Bank water supply and sanitation projects could lead to inefficient operations, and inefficient and excessive investment costs, in part due to corrupt procurement practices; 0 The risk that utilities controlled by political office-holders could invite political interference in project selection and in the day-to-day operations o f the water supply and sewerage agencies; 0 The possibility o f unsafe service because o f the absence o f effective supervision by knowledgeable outsiders. Where regulation and operational responsibility were both vested in a monopoly provider there could be a tendency to adjustment performance standards to actual performance rather than the other way around; 0 The temptation o f preferring supply-driven solutions rather than considering demand- driven development and programs o f demand management; 0 The negative experience from opaque and politicized tariff setting that has lead to unsustainable finances o f the public sector service providers; and 0 The risk that the professional standards o f utilities could stagnate because o f the politicization o f the appointment o f key staff and because o f the excessive turnover o f staff. 3.2 Conventional Bank projects were fairly ineffective in combating the flaws in the public sector development model. I n tnany instances,-the effect was just the opposite o f the intended. For instance, the granting o f Bank financing to a monopoly service provider could favor the provider whenever financing was scarce. Similarly, there was little Bank task managers could do to control excessive political interference in operations and management in agencies where the boards and 12 management were largely appointed on account o f their political affiliations and loyalty. Service quality and efficiency suffered for lack o f explicit performance contracts for the public water supply and sewerage agencies and for lack o f outside, statutory control. A t any rate, performance contracts with public sector companies (“contra plans”) were found to be largely ineffective for Besides, sanctions in the form lack o f incentives to improve and for lack o f effective reg~1ation.l~ o f fines for contract violations would only be paid out o f consumers through higher tariffs or through budget subsidies. Tariff setting was a source o f constant friction where consumers thought that they were asked to pay for the inefficiencies o f poor service and utility managers could not plan ahead because they could not be certain that needed tariffs would be raised in time. Finally, the Bank’s efforts in institution building could be ruined over night when staff who had been trained under Bank projects were replaced by a new manager because they owed their appointment and loyalty to competing politicians. In short, the old assistance paradigm suffered from lack o f correct incentives and for lack o f accountable service providers. THE NEWASSISTANCE PARADIGM 3.3 Given the challenge o f achieving project sustainability and institutional development under the old assistance paradigm, starting in the early 1990s the Bank began exploring alternatives to the old assistance paradigm o f public management. The paradigm that emerged had three cornerstones: 0 Quality regulation to ensure accountability o f service providers and the separation between the regulation and the operating responsibility. The expectation was that the collection and analysis o f data on service coverage and quality, and on the efficiency and sustainability would stimulate competition between different service providers to strive to approach the best practice in the sector. Such “comparative regulation” or “yardstick regulation” i s practiced, for instance, in Chile (see Box 3), in the United Kingdom through the offices o f OFWAT, and was promoted in Brazil under the Bank-financed Water Sector Modernization project that set up the Sistema Nacional de Informagoes sobre Saneamento (SNIS) for the entire water supply and sanitation sector. 0 Economic regulation based on explicit and predictable pricing policies where tariff levels and structures were predicated on the efficient costs o f service and where an effort was made to track which consumers received how much o f subsidies. The present study showed that such economic regulation i s practically non-existent in countries with active Bank water supply and sanitation projects. Instead, the study attempted to evaluate the implementation o f the Bank’s own pricing policies under the projects it has financed under the presumption that these pricing policies would substitute for the absence o f the client countries’ economic regulation for the projects in question. Private sector participation to avoid the instability o f public water supply and sanitation agencies and to offer competition to the public monopoly providers. The evaluation attempted to assess the effect o f PSP on the performance indicators (see Box 1) that relate to the provision o f “efficient, sustainable service for all.” Similarly, the analogous indicators were collected and analyzed for a sample o f projects without PSP in order to compare whether there was any significant difference in performance between the two groups. 13. See, for instance, M a r y Shirley, ” W h \ Pcrfortnancc Contracts for State-Owned HnterDrises Haven’t Worked.” Public Policyfor the Private Sector, N o t e N o . 150: August 1998. 13 OF THE NEW EVALUATION ASSISTANCE PARADIGM 3.4 Ideally, an evaluation o f the new paradigm the evaluation should have analyzed a sample sufficiently large to permit identifying the separate effect o f each o f the three key factors: (i) quality regulation; (ii)economic regulation; and (iii)private sector participation. This proved unfeasible mainly because, as will be discussed below, there has been little institutionalized regulation implemented under either the closed or active water supply and sanitation projects. In the few cases where quality regulation was developed under Bank-financed projects (see Table 7) it proved impossible to establish a statistically significant correlation between the existence o f quality regulation and project performance. 3.5 The link between PSP and regulation could take two forms: (i) PSP in projects in countries with institutionalized quality and economic regulation; and (ii) PSP in projects where the contract itself specified performance criteria and certain tariff levels. The former group o f projects (with institutionalized regulation) contained very few projects. Either the Bank i s no longer active in the water supply and sanitation sector with the most developed regulation (Chile) or the existing regulation had been over-ridden by the political realities (Bolivia, Colombia, and Peru). The latter PSP-group (with contract regulation) contains more projects but the contractual commitments, either on performance targets or on tariff levels, proved insufficient to withstand the macro-economic and political pressures. 3.6 I n summary, whereas the study could not evaluate the full effect o f quality and economic regulation on the performance o f either private or public water supply and sanitation operators, it did assess the extent to which quality and economic regulation has been implemented under those projects that have received Bank assistance. Similarly, the study did survey the extent o f private sector participation under Bank-assisted projects and was able to compare the evolution o f performance indicators under these projects. Thus, it points to ways in which the new assistance paradigm could be better implemented in future Bank projects. A tentative conclusion i s that the new assistance paradigm remains valid since it has forced the sector to address how to measure progress explicitly, how to mitigate risk, and how to take advantage o f both regulation and private sector participation as means to improve sector performance rather than ends in themselves. PROGRESS I N CREATING SECTOR REGULATION 3.7 Regulation in the water supply and wastewater sectors was developed began in the late 1980s in Chile and in the United Kingdom with a separation o f the three key sector functions: 0 legislating laws and policies; 0 regulating that laws and policies are correctly applied; and 0 providing the services within the given legislation and under regulation. Best practice in the sector assigns each function to different branches o f the government and service providers. The legislative authority i s part o f the political process whereas the regulatory authority i s best exercised independently o f the legislative process. The separation between legislative and regulatory duties i s considered necessary to lend greater stability and predictability to the sector and avoid the problem o f a regulator who adjusts rules to match fit the actual performance o f service providers. In turn, service providers should be given incentives to perform efficiently and be subjected to regulation by results and not by day-to-day micro-regulation with the subsequent loss o f autonomy. The regulator should rather limit his role to periodically monitoring whether the private operator i s meeting his contractual obligations to provide a given service coverage and quality within the tariff authorized. 14 3.8 The World Bank has assisted regulatory reform in the sector in a number o f ways. First, analytical and advisory assistance (AAA) have been provided to i t s client countries to define and prepare for possible further technical and financial assistance. Second, the actual establishment o f regulatory agencies has been promoted under adjustment lending or in parallel to investment lending. Third, the Bank has financed and disseminated research on the experience o f regulation in the sector. 3.9 The OED portfolio review found that 20 projects (14 percent o f the total) included sector regulation as a component o f the project, and 1 1 projects (8 percent) included regulation among the objectives o f the project. A l l projects with an objective or component to improve regulation also had private sector participation, and it can be inferred that PSP i s a main driver o f Bank action to demand or convince the client o f the need for regulation. On the other hand, 54 projects (38 percent) promoted privatization, and it can be deduced that 34 projects (24 percent o f the total number o f projects) relied on contract regulation to monitor and regulate contract compliance. The evaluation o f those projects with regulation yields a number o f findings that are relevant for the Bank’s future assistance the sector. 3.10 First, development o f the legislative and regulative framework for the water supply and sanitation sector i s a new concept on the political agenda o f many developing countries, where the need for regulation i s often not understood. Alternatively, it i s thought that governments already are exercising it if the service provider i s a public agency. The imperfect understanding o f what regulation i s about confirms the wisdom o f the Bank’s substantial effort to disseminate the concept o f regulation and its common forms. However, the Bank must tailor i t s message o f what regulation i s and what it can accomplish to what i s realistic in developing countries. Regulation can be o f two types: (i) quality regulation that tracks the quality and efficiency o f individual service providers; and (ii)economic and tariff regulation that applies objective criteria for setting tariffs. Bank clients have only accepted quality regulation under Bank-assisted programs. Economic regulation has never been succes@ly applied for an entire sector under Bank assistance. Where it has been applied, as in Chile and Colombia, the Bank played no role in i t s design and implementation. 3.1 1 Second, the novelty o f the Table 7. Few Bank Client Countries Have Regulatory concept and the difficulties have Regimes for Water Supply and Sanitation produced only a handful o f functioning . “ “ ” .- I water supply and sanitation sector Country Quality Economic Bank Assistance ~ regulators among the Bank’s client - Rmlation Regulation for Creation LATIN AMERICA countries, and none at all in the Middle Argentina Yes No No Easmorth Africa, Europe and Central Bolivia Yes Yes Yes Brazil Yes No Yes Asia, and South Asia regions. Even Chile Yes Yes No fewer have had Bank assistance in the Colombia Yes Yes No creation o f their regulatory regimes Peru Yes Yes No AFRICA (Table 7)’‘ Mozambique Yes Yes Yes Zambia Yes No Indirectly 3.12 Third, as noted above, Bank EAP support to establish regulation has -. The ‘ .. -Philippines * ”.I - Yes, Manila Yes, Manila Yes, Manila mainly been a consequence o f the 14. In many countries with limited private sector participation it i s not clear that dedicated regulatory agencies are the most appropriate answer to the need for regulation. A more cost-effective alternative may be contract regulation, involving private audits o f compliance with contract covenants. Similarly, Brazil’s Sistema Nacional de Informagoes sobre Saneamento (SNIS), supported by the Bank, may represent best practice for “yardstick regulation” o f service quality and efficiency in a situation \$here provision o f water and sanitation services i s fragmented since i t i s the responsibility o f individual municipalities. 15 Bank strategy o f promoting commercialization and privatization. Regulatory regimes and agencies have been created with the explicit task o f monitoring compliance by private operators with the contractual terms o f various types o f private sector participation. However, regulation i s arguably needed for both government and private operators-effective regulation becomes much more difficult when water supply and sanitation systems are managed by public utilities. The reason i s that sanctions to correct for breaches in service quality or efficiency do not function well for public operators. For instance, fines that are levied are often paid out o f higher tariffs or from government subsidies in the case o f publicly managed utilities rather than out o f the profits o f private operators. 3.13 Fourth, the need for regulation may be even greater when the government i s the service provider since this will usually be done without any contractual relationship and without any performance indicators. The corollary i s that the establishment o f quality regulation, as a minimum, should be a minimum requirement under Bank assistance. The Bank’s project to encourage the creation and application o f benchmarking o f sector performance indicators i s a good first step but does not seem to have been widely accepted. L i t t l e use would appear to have been made o f those data that have been collected. 3.14 Finally, the novelty o f the regulation concept makes it unrealistic to expect it to become an accepted part o f the sector in the course o f one Bank project with an implementation period o f as little as five years. I n reality it requires a decade or longer, which indicates a need for continued Bank assistance. This is particularly true whenever regulation i s promoted in parallel with private sector participation, possibly as one individual contract. The legal process in a country i s much slower than the process o f commercialization and privatization, which increases the risk that the regulatory framework will be without external assistance at the critical time when the first rate review comes up, typically within the first five years o f the private sector contract. I t then becomes necessary to plan Bank assistance for regulation to continue under subsequent lending operations or under parallel vehicles, such as structural adjustment operations. Such extended support to sector regulation would be particularly suitable under Adaptable Program Lending (APL). PROGRESS I N APPLYING PRICING POLICIES 3.1 5 The Bank’s pricing policies in the water and sanitation sector are laid down in two operational directives: Operational Manual Statement 3.72 o f September 1978 for energy, water supply and sanitation, and telecommunications (EWT), and Operational Directive 4.07 on water resources management (WRM), first issued in 1993 and updated in 2000. The two directives make i t clear that Bank’s lending in the water and sanitation sector should aim at objectives that help borrowers to: 0 provide basic infrastructure for other directly productive sectors; 0 build institutions to carry out national and regional programs in the sector, including mobilizing, through adequate pricing policies, the funds necessary for cost recovery and sectoral development; 0 implement pricing and incentive policies that promote water conservation and improve allocation o f water resources; and 0 provide adequate water and sanitation services to the poor. 3.16 The EWT directive recognizes the theoretical and practical difficulties in applying the principles underlying the policy objectives. The theoretical dfficulties o f tariff policies refer mainly to the application o f marginal cost pricing, particularly when there are conflicting social 16 goals or large indivisible investments. I t i s well known that the theoretically correct price i s short run marginal cost (SRMC). However, a tariff based on the theoretically correct price will lead to price instability since the SRMC will change with each incremental cubic meter o f water. To overcome the instability, the EWT proposes the use o f average incremental cost (AIC) as a way to smooth the variations in the SRMC. It should be noted that the AIC i s not universally accepted as a guideline for setting tariffs, for instance, in cities that have undertaken large investments financed by grants. I n such a situation, an AIC-based tariff may produce large amounts o f excess cash during the initial years. I n practice, the risk for large cash surpluses appears an acceptable price to pay in return for gaining the advantages that AIC-pricing bring: a forward-looking tariff calculation that i s particularly useful when the next addition to capacity w i l l be much more expensive than the present system. 3.17 The practical dffculties o f tariff policies, or “operational realities,” refer to the inadequacy o f incentives present in the service providers to effectively address the operational problems they face, such as physical losses, improper billing, poor bill collection, inability to meter or control consumption, and inefficient operation and maintenance o f the infrastructure. Generally, if the metering and the policies do not provide sufficient incentives for the service provider to attempt to convert a “lost” cubic meter into cash collections one cannot expect any tariff to have much o f an impact o f economic efficiency. Similarly, where consumers do not have sufficient incentives to try to reduce their level o f consumption tariff policies w i l l not improve economic efficiency. Effective metering o f both production and consumption i s a sine qua non for the incentives to work their intended effect. 3.18 Table 8 illustrates the complexities inherent in the application o f the Bank’s pricing policies and how these policies mesh with the operational realities o f the service providers. The table also indicates which approaches to operational realities are most likely to result in achieving the Bank’s policy goals. From the table it can be seen that: A l l consumption and production should be metered to make possible efficiency in both. 0 Tariffs, in most instances, should be guided by average incremental cost (AIC) o f each cubic meter consumed. As noted the AIC i s the average cost per cubic meter o f the next capacity increase for the system. As a rule, such capacity increases are designed for approximately 15 years, which i s the time horizon during which the incremental costs and incremental volumes consumed are analyzed. Both are discounted to allow the inter- temporal comparisons o f alternative schemes to increase capacity and consumption. 0 Direct government subsidies to the poor are required to meet social objectives. In particular, no consumer segment should be obliged to pay more than an agreed portion o f income for water supply and sanitation services. If direct government subsidies to consumers are not feasible, e.g., because o f fiscal weaknesses o f the government, some kind o f cross subsidy via the tariff i s one practical option to support the consumption o f the poor. Another option i s to explore different levels o f service and the use o f appropriate technologies to ensure access o f service to the poor. 3.19 Table 8 illustrates the expected impact from metering, from different types o f tariff structures, and from different vehicles o f subsidies. It can be seen that metering i s both an absolute requirement to meet the different aspects o f pricing policies as specified in OP 4.07. Metering will help the utility to produce as efficiently as possible and w i l l induce consumers to adjust their level o f consumption to an optimum. 3.20 As for the best practice o f tariff structures a two part tariff i s the optimum where the very basic consumption level, the lifeline, i s provided at a nominal cost and where all additional consumption i s charged at a uniform tariff set at the average incremental cost (AIC) to signal to 17 consumers what future costs o f service will be. The common tariff structure o f increasing block rates (IBT) where additional consumption costs increasingly more i s not optimal. Experience shows that it quickly degenerates into a structure where the majority o f domestic consumers are subsidized by increasingly fewer large domestic consumers and by commerce and industry. The resulting distortions grow and in the end the subsidizing large-scale consumers frequently cease being consumers (and subsidizers) o f the public system by building their own water supply and sanitation systems.15 Table 8. Choices in Pricing, Tariffs, and Subsidies Affect Achieving Bank Policy Objectives - -.=-ZxL2;- Pricing 8 Operational Economic Efficiemg ___ Finan cia1 Realities Utility Water Water Sustainability . . _- Production Allocation Conservation Pricing: ~ Metered + + + I + + - - Non-metered ______ - 7 7 Tariff structure: IBT -or? - or ? AIC + t + +or? +or? Customer 7 differentiation - __ - Subsidies to: 1 1 Utility (to supply) or ? Customers (to demand) Direct + + + + Cross -.-~--".". .,.-, _." ,-~-..*-", -4'*~".-..-.% 1 -or? -or? + denotes a positive effect on reaching Bank pricing objectives: - denotes a negative effect on reaching Bank pricing objectives; ? denotes uncertainty as to the effect on reaching Bank pricing objectives IBT: Increasing Block Tariffs AIC: Average Incremental Cost 3.21 Finally, Table 8 identifies direct income-based subsidies to consumers, Le., to demand, a s the best way o f supporting the consumption o f the poor. However, where the administrative ability o f borrowers i s insufficient to administer such direct subsidies because poor consumers cannot be reliably identified, a simple form o f cross-subsidy based on a two-tier, lifeline tariff may be the preferred alternative." To enable such lifeline tariffs, consumers must first be connected, which implies that the priority i s to subsidize access to the public system by connecting consumers at nominal cost. This will have a large beneficial impact since it will provide poor consumers with a safe source o f water supply at a cost that i s frequently 5 percent to 10 percent o f what they pay per cubic meter to water vendors. However, the lower tariff does not necessarily mean that consumer water bills will be 5-10 percent o f what they previously paid because their consumption levels will likely rise once they are connected. 3.22 Pricing Experience. A review o f the 304 project appraisal documents show there are major differences in the approach to pricing o f services in dedicated and non-dedicatedprojects. Dedicated projects appear to be more concerned with institutional and financial issues, including pricing, than the non-dedicated projects, as suggested by the number o f projects in which the stated objectives include institutional development and cost recovery, and by the number o f projects with covenants related to pricing and bill collection. (See Table 2.) Dedicated and non- 15. Increasing block rates i s critiqued in John J. Boland and Dale Whittington, "The Political Economy o f Increasing Block Tariffs in Developing Countries;" paper presented at a World Bank workshop, November 3-5, 1998. 16. The tentative conclusion i s corroborated by Lazlo Lovei, Eugene Gurenko, Michael Haney, Philip O'Keefe, Maria Shkaratan, "Maintainine IUtiIity Scr\ icch for the Poor--Policics and Practices in Central and Clastern Euroue and the Former Soviet Union." Washington DC: World Bank, May 12, 2000. 18 dedicated projects in rural areas share the objective o f assisting communities to establish adequate mechanisms to recover operational and maintenance costs from users. 3.23 Water Allocation. Water allocation issues are not discussed in the appraisal documents for any o f the evaluated projects in the selected sample. This omission i s understandable, however, considering that the water supply and sanitation sector uses, on average, less than 10 percent o f water resources, and that most countries give priority to water for drinking purposes. It i s also debatable whether projects in this sector are the best positioned to address water allocation issues. 3.24 Water Conservation. Concerns about marginal cost pricing are the exception rather than the rule, judging by the selected sample o f 15 projects out o f the 48 more detailed questionnaires mailed to World Bank task managers. Only 4 o f 10 project appraisals in urban areas, and one in rural areas include a calculation o f the marginal cost o f expanding output or services. N o project presents a plan or strategy to adjust tariffs to reach marginal cost. Moreover, operational and managerial shortcomings o f most service providers make i t impractical to implement marginal cost pricing, at least in the medium term. These operational realities include: lack o f metering, poor billing and collection, and inefficient operation and maintenance o f the infrastructure. In addition, all but one o f the projects in urban areas apply increasing block rates (IBT), a tariff structure commonly considered the best way to achieve water conservation and to ensure access by the poor to water and sanitation services. However, this tariff i s inconsistent with these goals, as discussed above. 3.25 Financial Sustainability. An important objective o f the Bank’s involvement in the sector is, as stated in the Operational Manual 3.72, to help borrowers “build up institutions capable o f effectively planning and implementing national or regional programs, including mobilizing through adequate pricing policies the funds necessary for sector development ” (italics added for emphasis). In accordance with that statement, rural water and sanitation projects are concerned with establishing the users’ contribution to financing part o f the investment. The projects also emphasize a demand-driven approach to ensure that communities take care o f the operation and maintenance o f the systems. 3.26 Pricing and financial sustainability issues o f service providers in urban areas are extensively discussed in the appraisals for dedicated projects and are at times the subject o f loan covenants and plans o f action. In contrast, this discussion i s perfunctory in non-dedicated projects. The review o f the appraisal documents in the sample o f projects suggests, however, that there i s substantial room for improvement in the analysis o f pricing and financial sustainability issues and in the emerging recommendations and plans o f action. 3.27 Serving the Poor. The pricing policies under Bank-financed rural water supply and sanitation projects are consistent with Bank‘s policy o f giving priority to the poor. They reflect the importance o f a small contribution by the beneficiaries to meet capital expenditures, based sometimes on willingness-to-pay surveys. The obligation o f project beneficiaries i s to implement charges that allow the community to generate revenues sufficient to cover adequate operation and maintenance expenditures. 3.28 Projects in urban areas are less effective than projects in rural areas in implementing pro- poor financing policies. This lack o f effectiveness i s the result o f prices that do not adequately cover the cost o f the services, coexisting with subsidy mechanisms that provide more support to the middle class than to the poor and fail to reach the very poor who lack direct access to services. 3.29 Factors in Project Performance. The Bank’s assistance has had limited success in inducing sector borrowers to implement pricing policies consistent with the objectives o f the Bank’s pricing policy. This seems to be the result o f strategic weaknesses in applying a gradual 19 approach to reform, as suggested in the Bank’s policy directives. It i s also the result o f an inadequate response to the often-conflicting policy objectives. This strategic weakness i s reflected in the design o f projects without adequate consideration o f the incentives that determine the “operational realities” o f the service providers, the continued use o f poorly targeted subsidies, and o f financial plans that do not adequately promote, from the outset, financial sustainability and resource mobilization. 3.30 Accommodation o f operational realities. A major weaknesses in project design i s the expectation that weak sector policies and poorly managed and accountable sector institutions will succeed in carrying out large investments in expansion and rehabilitation, while simultaneously creating the enabling conditions and incentives for the implementation o f the Bank’s pricing policies. An important effect o f that expectation i s that “quality at entry” i s compromised. Project designers and appraisers should strive for a more realistic assessment o f the challenge to improve performance o f poorly managed service providers and thus to create the required conditions for the effective implementation o f the Bank’s pricing policies. Actions and investments aiming at “putting the house in order” should then have greater priority over large investments for expansion in the early years o f project implementation. 3.3 1 Conflicts between policy objectives. Economic efficiency, water allocation and conservation, and serving the poor are all accepted objectives, but they do not necessarily lead to a consistent pricing policy. A detailed analysis o f pricing trade-offs to reconcile, to the extent possible, conflicting policy objectives i s necessary in all projects to arrive at a pricing strategy best suited to the particular project conditions. 3.32 Subsidies that do not serve the poor. A large proportion o f projects in the sample, particularly those in urban areas, have been ineffective in implementing pro-poor policies that foster access to services. This lack o f effectiveness i s the result o f prices that do not cover cost, coexisting with ineffective subsidy mechanisms that benefit better-off groups more than the poor. A primary example i s the use o f IBT, which often discriminates against poor consumers who consume much less water per capita than average consumers. Consumption levels o f 30 lcd are not uncommon for both unconnected and connected o f the lowest-income consumers. Under the circumstances, it i s an sub-optimal use o f scarce subsidies to subsidize those consumer segments who consume much more than this lifeline level of consumption. Any additional financial resources available for subsidies are best used to subsidize the connection cost o f the poorest to enable them to benefit from a safe and low-cost lifeline consumption. The application o f direct government subsidies to the poor might be beyond the institutional capabilities o f many governments. 3.33 Ability to achieve financial sustainability and resource mobilization. Financial sustainability o f the service providers and resource mobilization for sector development, although extensively discussed in the project appraisals, remain elusive goals. The financing plans underlying the projects are not fully adequate to ensure the sustainability o f the infrastructure, the reliability o f the services to a l l consumers, and the capitalization o f the service providers. All water and sanitation service providers in urban areas should be able to recover all recurrent and capital costs o f an efJicient operation in accordance with the principles o f economic pricing. In rural areas, recovering at least all operation and maintenance costs i s necessary to ensure sustainability. Often rural beneficiaries are also required to contribute a portion o f the capital investment costs. I t i s essential, therefore, to develop enforceable agreements with borrowers and implementing agencies on meaningful and monitorable plans for operation and maintenance properly priced and budgeted. I n addition, service providers in urban areas should progressively generate sufficient funds to finance a meaningful share o f investments for expansion and rehabilitation. The only country that has applied a coherent pricing policy i s Chile (Box 3). 20 Box 3. Chile Represents Best Practice in Pricing Policies A special Water Supply and Wastewater Law from 1989 obliges all service providers to charge tariffs that reflect long-run marginal cost o f an efficient operation. Tariffs should also enable service providers to cover their financial costs, a trade-off that signals the importance Chile assigns to having a financially sound utility. This criterion i s generally satisfied as long as consumers pay the average incremental cost for their consumption since future costs are invariably above historical costs that constitute financial costs. This i s true even though only the AIC for efficient producers i s allowed to be levied. Consumers who cost the same to serve must pay the same tariffs and cross-subsidies are eliminated. Subsidies are for low-income households, as defined by central government guidelines. Local governments are responsible for identifying these households. Subsidies are defined yearly and paid by the central government, via local governments, to the utility, which sends an invoice for the total subsidy to the municipality. The tariff structure i s kept simple to make payments administratively simple and to increase transparency. The two-part tariff has a fixed monthly component and a constant unit rate for each cubic meter consumed. Subsidies cover part o f the bill up to 20 cubic meters per month up to a maximum subsidy o f 85 percent o f the full cost-based bill. The bill to the consumer indicates the full cost o f the services provided, the amount o f the subsidy the consumer i s entitled to, and the difference to be paid by the consumer. PROGRESS I N PRIVATE SECTOR PARTICIPATION 3.34 Private provision o f water supply and sanitation services i s not a new concept. Private ’ concessionaires built a number o f early water supply systems in Europe in the late 1tithcentury and in Latin America, Africa, the Middle East, and Asia in the late 19‘h century. Subsequently, with the emergence o f ai1 economic model where the state was presumed to be more closely aligned with the social aspirations o f the populations, the role o f private operators shrank. In the French colonies, private service providers remained longer than they did elsewhere, influenced by the prevalence o f this model in France itself. For instance, in CGte d’Ivoire a private operator, la SocietC de Distribution d’Eau de CBte d’Ivoire (SODECI) had been established in 1959, before independence, to supply the capital, Abidjan, with water and sewerage services. In most other developing countries private sector participation (PSP) faded, discouraged by tariff legislation that did not permit a return commensurate with the risks. In many newly independent nations there was also a natural reaction against the previous models o f managing services. The PSP model was almost universally replaced by public service providers that were either central government agencies or municipal providers. 3.35 The paradigm o f government service providers proved a natural fit for World Bank lending to the sector, which began in the late 1960s and expanded rapidly in the 1970s. The Bank’s statutory requirement o f sovereign guarantees was easily satisfied as long as service providers were central government agencies. The same was initially thought to hold true for municipal service providers. However, gradually the moral hazard became apparent o f allowing municipal borrowers to perform poorly without suffering the full consequences since the central government was obliged to pay for World Bank debt service under i t s sovereign guarantee. 3.36 Disenchantment with the poor performance o f lending to state providers grew, however, and accelerated with the debt crisis o f the 1980s. Parallel to the growing disillusions with public service providers, the private sector gained prominence following the wholesale privatization o f water supply and wastewater services in England and Wales under the Thatcher government in the 1980s. What had until then been a phenomenon limited to France and Spain became a model that was suddenly thought to be worth considering everywhere. This paved the way for the Second Water Supply Project for GuinBe, which resulted in a 1989 IDA credit (Credit 1985-GUI) 21 that financed investments managed by a state holding company, SONEG. The operator was a private company, SEEG, that operated the publicly owned assets under a 10-year lease. 3.37 The next project with significant private sector participation was the concession o f Obras Sanitarias de la Nacion in Buenos Aires. The World Bank played a role in the preparation and assisted as an adviser to the Argentine government in the bidding and contracting process o f this landmark concession. Following the May 1993 takeover o f the Buenos Aires system, the concessionaire, Aguas Argentinas, could demonstrate rapid improvements in the quality o f service and i n the level o f investment. The success o f the Guinee lease and the Buenos Aires concession spurred widespread interest in PSP. Supported by research, dissemination, and technical assistance, PSP became de rigueur in most regions, starting with Latin America, but rapidly spreading to the Middle East, East Asia and the Pacific, Africa, and South Asia. The pendulum swung far, and came to favor PSP as a panacea to the perceived performance problems o f the Bank’s water supply and sanitation portfolio. 3.38 It took a number o f PSP contracts that did not live up to expectations to correct for this excessive faith in the private sector as operator. The first disappointment was a management contract in Trinidad and Tobago, where the Bank had financed investments managed by a management contractor with a British operator and a local contractor. Following a three-year interim contract, the board o f the national water supply and sewerage company, WASA, and the private operator were unable to agree on a deepened PSP contract and the private consortium left without any appreciable improvements o f service quality. The next disappointment-a concession bid and contracted for the city o f Cochabamba, Bolivia (Box 4)-proved more significant. As a consequence o f this failure, governments, international lending agencies (and among them the Bank), and the private sector have acquired a more nuanced view o f PSP. I t i s not a panacea to deep-seated problems and cannot be expected to substitute for decisions that only governments have the power and obligation to make. PSP i s better likened to a sharp tool. A capable government can use i t to great advantage to improve t h e water supply and sanitation situation but an inept government can make matters worse through an injudicious use o f PSP without providing clear quality and price regulation and lending strong and sustained support to PSP. Box 4, Bolivia Offers a Cautionary Lesson in Concessions Under an IDA credit the World Bank had promoted a concession o f the Cochabamba water supply and sewerage system as a cost effective way to improve city’s poor service, but local opposition arose against the Bank recommendation to use cheaper water from the existing Corani dam. The IDA credit closed without any concession being signed. Later, the municipal coalition that had opposed the Corani scheme negotiated directly (without any Bank involvement) a concession contract based on the more expensive Misicuni scheme. The central government regulator signed the contract. The concessionaire, Aguas del Tunari, took over in late 1999 and promptly raised tariffs, as authorized by the Bolivian sector regulator, but without improving service quality. Severe civic opposition and rioting followed, and prompted the government to cancel the contract in early 2000. The regulation law that had been adopted in late 1999 was also modified and the regulator was forced to resign. The experience shows that for PSP to be successful (1) investments should meet least-cost criteria or else the government needs to provide subsidies; (2) it is imprudent for private operators to raise tariffs without improving service quality; and (3) PSP contracts should be bid and negotiated in a transparent fashion. The political fallout from the failed concession will likely affect the credibility o f fbture PSP schemes not only in Latin America but in other parts o f the world as well. 22 Bank Analytical and Advisory Assistance to PSP 3.39 In addition to lending for investments under lease and management contracts, the Bank has extended considerable analytical and advisory assistance (AAA) to i t s client countries. In the early 1990s some o f the corresponding economic and sector work (ESW) was financed out o f the Bank’s own budget. Subsequently, after i t s establishment in 1999 with funding from a number o f donors under a Trust Fund, the Public-Private Infrastructure Advisory Facility (PPIAF) has financed an increasing share o f A A A activities in the water supply and sanitation sector’s effort to prepare for private sector participation. Table 9 provides details o f the PPIAF-funded activities over the first 30 months o f operations. 3.40 PPIAF has been a flexible and Table 9. PPIAF I s an Important Supporter valuable source o f financing to support the of PSP in Water Supply and Sanitation Bank’s effort to promote PSP in the sector. “ * -* --^ In many instances, PPIAF has enabled the Purpose of PPIAF Grant No. of Amount of Grants Bank to follow up quickly on the initial . . . . _______ ___________ Activities (US$) Studies and dissemination 7 0.75 million diagnostic mission, which would likely not PSP Options and Strategy 15 4.50 million have been possible given the inflexibility o f Preparation of specific deal 4 1.05 million the Bank’s own current budget. I n fact, the Preparation of regulation s 1.70 million Total -.l,l.ll*..”.l-, 32 8.00 million OED evaluation indicates that the necessary x, time and effort to properly prepare PSP contracts were underestimated. PSP contracts take longer to prepare than traditional investment projects for two reasons. First, there i s a need to draft a robust contract and, second, there i s large effort required to build political consensus. 3.41 Another source o f A A A for the water supply and sanitation sector, although not exclusively in the area o f PSP, has been the Water and Sanitation Program (WSP). It i s the oldest trust-funded source o f A A A and was established in 1978 to manage the development o f appropriate sanitation technologies and promote their adoption in the Bank’s member countries. The WSP has lately expanded i t s activities to consider the possibility o f small-scale service providers to cater to the needs o f the urban poor. Given its limited scope as a desk study the present evaluation did not evaluate the PPIAF-financed and WSB-driven activities, nor the impact o f the customary sector dialogue, workshops and seminars promoted and implemented by the Bank. Such activities can arguably have a considerable impact on sector performance and should be self-evaluated more systematically.” 3.42 The water supply and wastewater sector’s Table 10. The Riskiest of All political importance and the fact that it i s by far the Infrastructure Sectors most capital-intensive in relation to annual revenue makes it the riskiest o f all infrastructure sectors for Infrastructure Sector Fixed Assets/ Annual Revenue prospective private operators. Table 10 explains the - - w ~ s u p p l yand Sanitation 10-1 2 point by relating the required capital stock of the four TOII Roads 7 infrastructure sectors tothe annual gross revenue. The Electric Power 4 Telecommunications 3 water SUDUIV and wastewater sector has the highest ~ ~ ~ . ~ ~ I . I w ratio o f the four sectors. This is largely because water supply and sanitation services are under-priced Council on Public Works (Washington DC: compared to other infrastructure services. In light of Wade Miller Associates, h c . 7987; ~ ~ ~ ~ based $ On data ~ On ~ ~ ~ t e s i t s capital intensity it i s hardly surprising that the water supply and sanitation sector has attracted the least private financing-6 percent o f total foreign direct investment-over the past 10 years. The fact 17. PPIAF and the Water and Sanitation Program are among the programs covered in OED’s forthcoming evaluation o f the World Bank’s Global Programs. 23 that most water supply and sanitation services are managed locally also converts risk into sub- sovereign risk. On the other hand, Bank lending requires a statutory sovereign guarantee, which limits the Bank’s ability to support local private operators with financing unless the central government i s willing to guarantee local borrowers and their performance. OED Ratings o f Completed Projects with PSP 3.43 The relative novelty o f PSP under Bank projects explains why few projects closed during the 1990-2001 period. Consequently, there are few projects with OED ratings. However, seven projects did close during this period; their ratings are shown in Table 11. Table 11. The Few Closed Projects with PSP Have Generally Performed Well - Prokt __and PsP-ty~e -I i I *_1_x^. Y l_i- x _.*_- ~~ O_utcome IDI’ ___ Sustainability Argentina I, Concession Satisfactory Modest Likely Argentina 11, Concession Satisfactory Substantial Likely Bolivia, Concession and one cooperative Satisfactory Substantial Likely GuinBe, lease Satisfactory Substantial Likely Indonesia, concession Satisfactory Modest Unlikely ~- Romania, concession Satisfactory Substantial Likely I- ~ _ .Unsatisfactory I- IyxI Modest Unlikely a ID1 = Institutional Development Impact Source OED evaluabon summaries 3.44 The ratings are generally positive. OED usually rates projects within a year o f the closing o f Bank projects, which i s a short time. However, the risk tolerance o f private operators i s probably lower than for government operators. Risks that materialize w i l l likely affect the operating cash flow and the profits o f the private operator immediately and the private operator could be expected to react immediately to mitigate risks and protect the financial returns under the contract. I n contrast, a government operator may not even be aware o f the impact o f certain risks and may be slow in reacting, particularly when there are no clear performance criteria. The ratings for projects with PSP are higher than for those projects without PSP, as Table 12 shows. The OED evaluation methodology rates project outcome on the basis o f tracking the degree o f fulfillment o f project objectives. Further, OED employs an explicit methodology for rating institutional development impact (IDI) and sustainability. I t i s important to keep in mind that the OED ratings are for the most part qualitative and are not exclusively based on a comparison o f the values o f performance indicators before and after the Bank loan. 3.45 Although the OED ratings seem to indicate Table 12, And They Are Rated Higher that the initial experience from PSP has been Than Projects without PSP positive in comparison with the average project, caution i s in order, given the small number o f closed RatedProject Aspect Projects Projects with PSP without PSP projects. I t could also be argued that initially Outcome, satisfactory 86% 60% successful concessions such as those for Buenos IDI, Substantial 57% 28% Aires and Santa Fe in Argentina are battling serious Sustainability, Likely 71% 35% difficulties in the wake o f that country’s deep macroeconomic crisis. However, this i s likely to be true for all projects, with or without private operators. It i s even likely that public service providers w i l l be more vulnerable than private operators since their cash flows are likely to be weaker and more dependent on central government budgets. Nor i s a successful, long-term PSP contract assurance o f continuity: the Conakry lease contract lapsed after 10 years and after another 2 years o f ultimately unsuccessful negotiations operations reverted back from the private operator to the government o f GuinCe. 24 Performance Indicators Under Bank-Assisted Projects, With and Without PSP 3.46 Using standardized definitions to ensure comparability performance indicators were collected through a mail survey o f operators and in the context o f selected OED project assessments for projects without PSP and projects with PSP and that had been facilitated by Bank technical assistance.” Data were received from 1 1 private operators out o f the 20 that received the questionnaires. Those not responding were largely small Argentine private operators. Unfortunately. the limited number of responses does not allow for generalizations regarding the relative performance o f different types o f PSP-model: concession, lease, or management contract. The number o f surveyed projects without PSP numbered 28 out o f which 18 were from the Bank’s lending program in China. In order to avoid a bias because o f the large proportion o f projects in China, the statistical calculations used the simple average from the 18 individual Chinese projects. The values o f the performance indicators for projects with and without PSP are shown in Table 13, which also compares the situation before and after the Bank assistance. The analysis was conducted on the aggregate average ratings to respect the wishes for confidentiality from a number o f those private operators who agreed to participate in the survey. The original intent was to compare the data supplied by the private operators themselves with those supplied by the regulators. However, in almost all cases surveyed there i s no ready regulatory capacity to supply an independent set o f data. Table 13. Projects With and Without PSP Both Show Improvements -.” ”--~.-yII.-I^Illi- -- - --- - ..- - -- --__ -II-u. -“~“..-yls-y---. -*_ -- Performance Indicator_- With PSP ~ Without PSP ___I- Before Project After Project Before After Project Coverage Share of Connected 68% 94% Not available” Not available Households receiving -- - - connected --- _--- Financial WorkingRatio 0 77 0 66 Share of Wastewater 7Oh 13% 9% 31% I treated I II” m- II - _11.-^-.“.I . Legend Unaccounted Water = (Metered Water Production-Metered and Estimated Unmetered Consumption)/Metered Water Production Employees per Water Connections = Number of Regular Employees/Number of Households ConnectedllOOO Financial Working Ratio = Cash Operating Costs (excluding depreciati0n)lOperating Revenue Share of Produced Wastewater Treated = The percentage of total wastewater produced (not necessarily collected) that receives any level of treatment 18. Antalya, Turkey (Ieaseicontracted operations with collections risk); Barranquilla, Colombia (“empresa mixta”); Buenos Aires, Argentina (concession); Cartagena, Colombia (“empresa mixta”); Conakry, Guinke (lease); Jakarta, Indonesia (PALY JA3cooperation agreement); Jakarta, Indonesia (TPM, cooperation agreement); L a Paz, Bolivia (concession); Manila West, the Philippines (concession); Santa Fe, Argentina (concession); Trinidad and Tobago (management contract). 19. “Not available” denotes that too few data prevent meaningful comparisons o f “before” and “after” project. 25 Service-for-All 3.47 The performance indicators on service coverage and quality demonstrate solid progress, both for projects with PSP and for those without PSP. With due caution paid to the fact that the sample sizes are small i t can be said that the differentials between the situation before and after the PSP are similar except for “share o f the population receiving disinfected water” where the improvement was substantially larger than under projects without PSP. The better performance by the private operators may be because their contracts with the government spelled out in detail the water bacteriological standards. The greater sense o f accountability, backed up by contractual sanctions, would then provide the incentive for the private operators to improve the bacteriological quality. I n contrast, public agencies rarely work under performance contracts that specify water quality standards. I t i s important to note that effective PSP requires well-drafted contracts that align incentives for the private operators with the objectives o f the client. Thus, if higher coverage i s an objective, the contract should f i x targets accordingly and link remuneration to higher connection rates. The time required to reach these improvements varied for the different contracts from 3 years up to 9 years. Service coverage w i l l require more time and higher investment to achieve compared to improvements i n service quality, which often depend more on the operator’s know-how. At times, improvements were reached without substantial investments in water production, as Box 5 illustrates. B o x 5. Colombia Shows the W a y in Better U s e o f Existing Capacity Barranquilla, a city o f 1.5 million on the Caribbean coast, was under private operation from 1925 to 1960. Universal coverage and good service quality made the city’s water supply the best in Colombia. However, the municipal government took over operations in 1961 and in three decades operations and service quality deteriorated to rival the worst in the country. The crisis reached a climax in 1991 when the World Bank suspended disbursements under an existing loan. The central government and civic organizations in the city rallied around reform that resulted in the creation o f a new company. A Spanish private operator, AGBAR, assumed operational control in 1996 to be followed in 2002 by another Spanish operator, Canal Isabel 1 1.A capable manager was successful in boosting water coverage from 78 percent to 97 percent in five years and sewerage coverage from 68 percent to 88 percent. Service quality improved rapidly. The positive achievements were possible mainly through a better use o f existing facilities and firm management that sharply reduced undocumented connections and unmetered consumption. Annual water production actually dropped from 17 to 15 million cubic meters although annual consumption rose by a third from 7 to 10 million cubic meters and the number o f connections grew by 44 percent from 180,000 to 260,000. As a result, the percentage unaccounted water decreased from 5 5 percent to 38 percent in five years. Efficiency of Service-Operating Efficiency 3.48 Private operators have ready incentives to improve operating efficiencies under both lease and concession contracts since by doing so they can directly increase their profit. The thumbnail indicator for operating efficiency i s unaccounted water that measures the share o f produced water that i s “lost” and does not generate revenue. In the surveyed PSP contracts unaccounted water decreased from 53 percent to 46 percent. Projects without PSP also recorded lower unaccounted water, but the improvements were smaller-between 3 8 and 40 percent. These levels are s t i l l high compared to the best practice utilities with unaccounted water between 10 and 20 percent. The explanation could be poorly designed tariff structures that do not sufficiently reward operators for reducing unaccounted water. In the case o f the Buenos Aires concession the concessionaire was slow in raising the metering coverage and ipso facto in reducing unaccounted water. The explanation was the financial incentives for the concessionaire were to continue billing according to the historical, un-metered tariff structure. Greater investment efficiency i s likely only under 26 concession contracts where the private operator will attempt to optimize investment returns since the concessionaire i s responsible for financing and paying for all investments at i t s own risk. While meeting contractual goals o f service coverage and quality higher investment efficiency will directly increase the operator’s profits. This explains the keen interest by concessionaires in reducing investments through squeezing more production out o f existing capacity. The first major concession, Aguas Argentinas in Buenos Aires, was characterized by many small investments to remove system bottlenecks rather than major investments in additional production capacity that a government operator might have undertaken. Under the same contract, the concession managed to postpone indefinitely costly additions to the capacity o f the sewerage interceptors simply by accelerating maintenance o f the existing interceptors. The concessionaire was also successful in quickly boosting production capacity through a series o f small but well selected investments in the existing treatment plants. The cumulative incremental efficiency in this kind o f investment policy i s difficult to document given that concessionaires, for competitive reasons, may not be willing to disseminate their success in raising investment efficiency. 3.49 The other performance indicator for efficiency i s staff productivity as measured by the number o f regular employees in the water supply and sewerage company divided by the number o f households connected (expressed in thousands). For projects with PSP this indicator decreased on average from 8.2 to 4.2 employees per thousand households connected to the water supply system, which represents a doubling o f productivity while non-PSP projects recorded a reduction from 7.6 to 4.4. The halving o f regular staff i s explained by the strong incentive o f lease and concession operators to improve the cash flow through attrition and reduction in force. Some o f the displaced employees were likely shifted to sub-contractors in the private sector who continued performing their erstwhile functions under outsourcing. To the degree that outsourcing increased with PSP, the reported productivity increases might be exaggerated. Nevertheless, staff reductions are particularly difficult for private operators, which explains why some concessions (such as the recent one in Casablanca) stipulate an interim period during which the private operator cannot reduce the number o f employees. I n the case o f the Cartagena “empresa mixta” it was the mayor who made the decision to reduce the number o f employees in the municipal water supply company before bidding for a private operator. This demonstration o f political support was likely decisive for the subsequent successful bid and contract negotiations since it resolved the difficult issue o f staff reductions. Sustainability of Privately Operated Services-Financial Sustainability 3.50 A good indicator o f financial sustainability i s the working ratio that relates cash operating costs to operating income with depreciation excluded. A ratio below 100 percent indicates that operations are generating a surplus that can be used for debt service and for investment. The collected data show a slight improvement in the average working ratio from 0.77 to 0.70 percent for PSP-operators and a corresponding drop from 0.71 to 0.66 for operators without PSP. In the case o f PSP-operators working ratios dropped and as a result internal cash generation rose because: (i)tariff levels rose on average by 20 percent after the take-over by a private operator and (ii) unaccounted water dropped from 53 percent to 46 percent, which implies that previously unrecorded water consumed was converted into metered water consumption and into collections. On average, the share o f metered water connections rose from 54 percent to 64 percent, and in those PSP contracts where the operator had a clear incentive to meter, the effectively metered connections was almost 100 percent within a few years (Barranquilla and Cartagena). 27 Environmental Sustainability 3.5 1 The eighth and final performance indicator i s environmental sustainability and measures the share o f total wastewater generated (although not necessarily collected) that i s treated. This indicator has also risen under the PSP projects surveyed from 7 percent o f wastewater treated before PSP to 13 percent during the latest available year with PSP and for non-PSP-operators from 9 to 3 1 percent. The small sample sizes make it imprudent to rely heavily on the differences in the shares o f wastewater treated registered for with-PSP and without-PSP projects. Investments in sewage treatment are heavily dependent on the availability o f concessionary financing since such investments are generally not financially profitable. It could be expected that private operators will put a lower priority to sewage treatment as compared to investments in water supply that will be easier to make profitable. Comparing the Performance of Projects With and Without PSP 3.52 On average, Table 13 shows that Bank projects-with or without private sector participation-produced improvements in all the indicators. The changes are quite similar for the two groups o f projects with a slight edge for PSP projects in areas such as quality and efficiency indicators. The relatively small differences support the conclusion that PSP i s not a sine qua non for improving service coverage and quality, and the efficiency and sustainability o f services. The question o f how long the improvements will last i s difficult to answer since both PSP- and non- PSP-assisted projects were monitored during a relatively short period, before and after the take-over by a private operator and before and after the assistance provided by the Bank. Given that the sample sizes are relatively small (1 1 PSP projects and 28 non-PSP projects) it will be important to continue adding to the database by collecting the performance indicators for all Bank-assistedwater supply and sanitation projects. I t will be particularly important to require all project completion reports to furnish such data over the project implementation period, which in turn should provide an incentive to incorporate the performance indicators in project supervision reports. 3.53 The reported slight differences in evolution o f the performance indicators between projects with PSP and without PSP must be compared to the OED ratings for closed projects, which are clearly higher for projects with PSP than for those without PSP (Table 12). The apparent inconsistency i s explained by the fact that OED does not rate its projects on the basis o f performance indicators but against other aspects, such as relevance, efficacy, and efficiency (combined as project outcome), institutional development impact (IDI), and resilience against adverse changes in a number o f areas (sustainability). Somewhat lower ratings in these areas are possible even though the performance indicators show little difference between PSP and non-PSP projects, for instance, if gains in performance indicators were achieved with heavy investments, financed with government guarantees (rather than without them, as in the case o f concessions); where ID1was slight (because institutional change i s more difficult to achieve with public borrowers), and with financial sustainability made possible with tariff increases rather than improved operating efficiency (as might be necessary in the case o f concessions). The relative performance o f PSP and non-PSP projects will require deeper analysis using additional data in the future. 3.54 The review o f PSP projects highlights two important factors o f performance. First i s the way the different types o f risks are allocated. And second, as already noted, different results can often be explained by the incentive structures embedded in the operator contracts. Both risk allocation and lack o f incentives go a long way in explaining the evolution o f performance indicators in the case o f public operators. In these cases risks are almost exclusively borne by the government and operator management do not stand to benefit personally for superior performance. It i s not surprising then that the upshot will be lackluster and erratic performance. (The more positive evolution emerging from Table 13 cannot be termed typical for public operators since the 28 incremental changes took place within the scope o f a Bank project with more focused governance than i s typical for public operators without external financial and technical assistance.) The situation for private operators differs considerably. They will operate under contracts that - if well drafted - w i l l specify how risks are allocated between the private operator and the client government. The differences between the three main forms o f PSP - management contracts, lease contracts, and concession contracts - center around the risk allocation. Private management contractors assume l i t t l e if any risk and are often paid according to the their costs. Although management contracts with incentive-based remuneration have been increasingly used under Bank projects these continue to shift little financial risk to the private operator. Lease contracts represent a much greater risk for private operators since they are often based on how much they can collect and since their profits will drop if they are unable to control operating costs. Only in concession contracts i s the private operator assuming most risks, including the construction and financial risks associated with financing and implementing an investment program. 4. Findings and Conclusions 4.1 The starting point for evaluating the Bank’s assistance to the water supply and sanitation sector in the 1990s should be the previous OED evaluation o f Bank assistance during the 1967- 89 period. OED evaluated Bank performance against four perceived principal objectives: To help governments achieve least-cost solutions to infrastructure needs; To foster institution building; 0 To help institutions achieve financial viability; and 0 To ensure a minimum supply o f safe water to the poor. 4.2 A t the time, OED concluded that the achievement o f these four objectives was partial at best. The first objective was hardly met in the face o f endemic project completion delays, and with sanitation and environmental protection trailing water supply accomplishments. The second objective rarely succeeded. The evaluation singled out lack o f progress in improving operations and maintenance (O&M), in reducing unaccounted water, and in improving the quality o f utility management as three areas o f shortcoming. The third objective was also felt not to have been met, in part because o f rushed reforms and the need for more time than single Bank operations can afford. Finally, the fourth objective was poorly documented or not addressed at all. 4.3 The Bank response to the largely negative OED evaluation became the focus on promoting “efficient, sustainable service-for-all” where “efficiency” was a proxy for least-cost policies, “sustainability” o f financial and institutional autonomy, and “service-for-all” for service for the poor. The chosen instrumentality for reaching the general targets was private sector participation, at times underpinned by regulatory reforms and a willingness to involve users in the selection and administration o f the systems. HAS PROGRESS BEENUNEVEN 4.4 Based on the data and analysis contained in the present evaluation OED concludes that the Bank has made uneven progress in helping its borrowers and client countries to move toward the targets of “escient, sustainable service-for-all. ” The assessment i s based on OED ratings o f water supply and sanitation loans that closed in the 1990-2001 period; on the evolution o f the performance indicators applied by OED in its project performance assessments since 1999; and on the quality o f the active water supply and sanitation portfolio. These three sources do attest to progress made but, in contrast, progress to establish regulation and applying the pricing policies 29 o f OP 4.07 has been uneven or absent in many instances. I t should be noted that reforms based on the “new assistance paradigm” gained pace from the mid-1990s onwards. The fact that water supply and sanitation projects have an average implementation period o f 6.7 years implies that it i s not possible to capture the full effect o f the “new assistance paradigm” on OED-ratings since few o f the projects approved from the mid-1990s onwards are included in the 1990-2001 cohort o f projects analyzed in the present evaluation. For the same reason, the full effect o f the reforms can only be assessed for projects that closed from 2002 onwards. 4.5 OED ratings hint at improvements. Outcome ratings o f “satisfactory” or better rose from 58 percent for projects that closed in the 1980s to 64 percent in the 1990s. Ratings for institutional development impact as “substantial” or better held steady at 32 percent. The “sustainability” ratings o f likely or better were mixed, however. The “1ikely”ratings rose from 32 percent in the 1980s to 40 percent in the 1990s but so did the “unlikely” ratings from 19 percent to 30 percent. It i s clear that water supply and sanitation investments financed under non-dedicated projects performed less well than under dedicated water supply and sanitation projects. A systematic analysis o f completion reports for non-dedicated projects indicates that outcome was rated “satisfactory” in 46 percent o f the closed projects (versus 64 percent for dedicated projects); that ID1ratings were rated “substantial” in 7 percent of the closed projects (versus 32 percent for dedicated projects); and that sustainability was rated “likely” in 24 percent (versus 40 percent for dedicated projects). The latter ratings are tentative but confirm the intuitively obvious: water supply and sanitation investments are better prepared, implemented, and managed by specialized “dedicated” divisions rather than by divisions where water supply and sanitation are just one o f a number o f sub-sectors. 4.6 The evolution o f the performance indicators i s only available from 1999 onward for those projects that OED has assessed (about 25 percent o f closed projects on average) and for those projects with private sector participation. However, based on a sample o f 11 projects the trend i s encouraging. Water supply and sewerage connections rates rose substantially from 66 percent to 80 percent and from 38 percent to 48 percent, respectively, comparing the situation subsequent and prior to PSP; service continuity and bacteriological quality rose from 68 percent to 94 percent and from 82 percent to 97 percent, respectively; efficiency improved as measured by unaccounted water and higher doubled staff productivity; and financial and environmental sustainability o f service also strengthened. Similarly, the performance indicators for non-PSP water supply and sanitation projects also demonstrate similar positive changes from Bank financing (Table 13). 4.7 The positive trend from the evaluation o f the effectiveness o f private sector participation i s also mirrored in the quality o f the active water supply and sanitation project portfolio. Due to efforts to improve performance the Bank dedicated water supply and sanitation portfolio o f about 100 projects now has 15 percent o f projects at risk, in line with the Bank average o f 17 percent, and has realism and pro-activity indices in line with Bank averages. It can be concluded that with active portfolio management the water supply and sanitation portfolio i s not any riskier than the average Bank project and does not have any ratings substantially different from other sectors when adjusted for the effects o f the dual economic goodhocial good nature o f the sector. Much o f the improvement can be attributed to the conscious effort to manage the portfolio more actively by closing poorly performing projects and not accepting new projects in countries without the political will to create an enabling environment for projects to prosper. REMAINS BUT MORE TO BE DONE 4.8 Despite these improvements, the sector continues to suffer from significant shortcomings. At the sector level, quality and economic regulation remain a distant goal. The best sector regulation in developing countries was created without Bank assistance, and few countries that have received Bank assistance can point to a sustainable and coherent legislative and regulatory 30 framework. The scarce progress to effectively implement the Bank’s own pricing policies in its client countries i s particularly damaging to sector progress since it negatively affects the financial and institutional performance o f the agencies implementing Bank-financed projects and raises the risks for serious private investors and operators. 4.9 A t the project level, many o f the deficiencies noted in the previous OED evaluation persist. First, the fundamental flaw i s the lack o f appropriate incentives that could motivate utilities to convert unaccounted water into paid revenue and that could induce managers to reduce both operating and investment expenditure. In the absence o f PSP, improved incentives remain elusive. Second, project completion reports continue to highlight overestimated demand as a main culprit for failed projects. The reason might be more than projection error: it could be a matter o f pressure to make projects economically and financially feasible that provides an upward bias in favor o f higher-than-prudent demand growth. Third, the lack o f financial autonomy spills over into lack o f institutional autonomy, which can help explain the breaches o f financial covenants. The latter are inevitable in the absence o f clear criteria for setting tariffs. After all, i t i s unrealistic to expect utilities to comply with financial covenants without being liberated from political interference in the fixing and updating o f tariffs. THEW A Y FORWARD 4.10 The future direction o f the sector within the Bank i s intimately linked to the Bank’s strategy to reach its Millennium Development Goals (MDG). I t i s already clear that the water supply and sanitation sector must gain importance if the MDGs are to be met. After all, three o f the M D G targets depend on the success in expanding and improving water and sanitation services: 0 Reducing by two-thirds, between 1990 ad 2015, the under-five mortality rates (Target 5); 0 Halve by 201 5 the proportion o f people without sustainable access to an improved water source (Target 10); and 0 Halve by 2015 the proportion o f people without adequate sanitation (Target 11 as amended by the Johannesburg World Summit o f Sustainable Development in August 2002). 4.1 1 I t i s less clear what the Bank’s strategy will be to achieve these targets. The recent trend o f improvement i s welcome but must be seen for what it is: an improvement in performance indices o f projects in the active portfolio. Some o f these improvements in portfolio performance are due to simply closing poorly performing loans. The greater and more difficult challenge will be how to expand the portfolio from the present low level o f new commitments while maintaining an improved performance. There are signs that this will not be easy, nor will it be cheap. In some regions o f the world lower preparation and supervision budgets have slowed sector development and made it difficult for the Bank to build the critical mass that i s necessary to maintain excellence in staff to match the growing proficiency o f client staff. In other regions, notably South Asia, the Bank has slowed or stopped the preparation o f new projects in the sector, in frustration over the lack o f necessary reforms o f policies. Yet, it i s precisely in these countries that an expanded Bank sector role i s necessary in order to meet the MDGs for the simple reason that such a large share o f the world’s poor live there with their basic needs unmet. 4.12 The sector will benefit from sharpening i t s present message. The last sector development strategy was adopted in the mid 1970s and has not been updated formally. Instead, the water supply and sanitation sector has been lumped with the water resources management sector. This has been less fortunate since the focus on how to improve the management o f water resources i s different from the need to provide safe water supply and sanitation services. After all, the amount o f water required to sustain the needs o f both urban and rural populations i s less than 10 percent o f the amounts o f water used for irrigation and i s rarely an impediment. What i s more to the 31 point: the solutions to the water supply and sanitation sector challenges requires a strategy all o f its own for the sector’s special needs. This lends urgency to the preparation o f the sector’s business development strategy. A business development strategy i s under preparation within the Bank and i s expected to be finalized in 2003. The strategy could benefit from a number o f conclusions that are emerging from the present OED evaluationhecommendations. Finding One: Monitoring Performance Indicators and Evaluation Systems Will Be Necessary to Track the Bank’s Progress in Helping I t s Member Countries Achieve the MDGs 4.13 The broad MDGs are general and cannot serve as a primary basis for planning and implementing detailed country or sector strategies. Instead, the Bank should revisit and confirm the use o f the performance indicators that were adopted in 1999 for the water supply and sanitation sector. Without continuous estimates o f service coverage and quality, efficiency and likely sustainability, it will be difficult to formulate credible development strategies. Without discontinuing the production o f portfolio performance indicators that caters largely to the Bank’s internal needs, a shift in resources i s necessary to assist the Bank’s member countries in implementing analogous monitoring systems geared to the performance indicators. (See paragraphs 2.6-7, 2.10,2.13, 2.19,3.46, and 4.6.) Finding Two: Regulation Must Move from Prescription to Implementation 4.14 A decade after beginning the effort to create regulation, only a handful o f countries have put in place well functioning systems o f quality and economic regulation in water supply and sanitation. Africa, Asia, and the Middle East do not have a single regulator with a proven record o f applying regulatory guidelines consistently. A few countries in Latin America do have regulation, but the best regulatory systems were conceived and established without any Bank involvement. A n enhanced focus on indicators would provide further incentives for countries to strengthen their regulatory frameworks, since well-designed quality regulation would enable planners, politicians, and managers to benchmark their system or sector in terms o f coverage, quality o f service, and o f efficiency. A t times quality regulation may be provided in a more cost- efficient fashion by making use o f private audit firms to collect and monitor data rather than through the establishment o f dedicated public regulatory agencies. (See paragraphs 3.9, 3.1 1 and Table 7.) 4.15 A push toward economic regulation is equally overdue. The review o f pricing policies under Bank projects has revealed that predictable tariff regulation through the consistent application o f transparent rules does not exist in any country that is an active Bank borrower. In the few countries where dejure tariff regulation does exist, it can rarely withstand pressure to conform to short-term political expediency. This diminishes i t s value to investors and operators since the tariff can no longer be predicted with any measure o f confidence. The Bank would do well to return to a policy o f requiring either tariff regulation according to economic principles or explicit and covenanted tariffs as a condition o f lending. A minimum requirement should be to adjust the tariff annually to compensate for inflation. (See paragraphs 3.24 and 3.26.) Finding Three: Ensuring that the M D G s are Achieved Will Require Translating them into Implementable Sector Development Strategies 4.16 The development community expects to meet the MDGs by 2015, that is, a l i t t l e more than a decade from now. Despite this urgency, most countries lack sector strategies that describe the present situation, and analyze policy reforms, pinpoint priorities, and pre-identify priority projects. The Bank i s well placed to assist its countries in preparing country-specific sector strategies geared to achieving the MDGs. But this will likely require a shift o f resources toward economic and sector work. In the past decade 1990-2001 (see Annex I )about 90 pieces o f economic and sector work were prepared by the Bank, but only 10 percent were sector studies that were directly aligned with 32 sector development. Future ESW will have to be more closely linked to lending. (See paragraphs 4.11-12 and Annex I.) Finding Four: Private Sector Participation has SIaown Promising Results and Remains an Important Tool to Improve Coverage and Quality 4.17 The tentative conclusion from the present review i s that private sector participation has been effective in raising service coverage and quality. However, it has not been a panacea for the sector’s problems, which has prompted a recent reassessment o f its use. The wholesale questioning o f the justification for PSP i s ill advised, but the unrealistically high expectations from clients and Bank staff must be tempered. What i s needed i s a continued careful evaluation o f each PSP-led program with a view to identifying factors o f success and failure. Continued promotion o f private sector participation i s sensible as PSP offers the promise o f more transparent performance by operators; faster gains in coverage, quality, efficiency, and sustainability; and accelerated development o f regulation. (See paragraphs 3.12, 3.44,3.47, and 3.48.) Finding Five: Operators Require Special Incentives to Serve the Poor 4.18 Both private and public operators should be held accountable for their performance in improving service to the poor. I n order to enable operators to serve the poor better, the government will have to ensure effective regulation and, on occasions, provide transparent subsidies linked to service provision to the poor. Output-based aid (OBA) i s one promising model (the results o f which will need to be carefully evaluated), under which operators are remunerated partly on the basis o f their success in serving poor households. OBA w i l l entail particular attention to be paid to reforms o f pricing and subsidy policies. It may be difficult for all but the financially strongest and administratively sophisticated governments to be able to provide subsidies from public budgets to underpin OBA. (See paragraph 3.32.) The necessary incentives include direct subsidies to pay for part or all o f investments that will favor low-income segments; subsidies contingent on the use o f appropriate low-cost technologies; and modified tariff structures to ensure that operators will find it profitable to provide quality service to the poor once the capital investments have been made. I n addition, service to the poor will often require informal, small-scale solutions in order to reduce costs to levels that are affordable. 33 Annex A Annex A. Glossary Service Coverage The percentage of households connected to a public potable water system or sewerage system. Financial autonomy The situation where a water and wastewater company i s able to collect revenue sufficient to pay for the costs of operations, adequate maintenance, depreciation on assets, and for paying interest and amortization on its loans. Financial costs Monetary payments by a water and wastewater company for operations and maintenance, and for interest on loans. Economic costs The alternative value of resources used for operations, maintenance and investments if they were sold in a completely free market. Marginal cost The future economic costs o f producing one additional cubic meter of water. The marginal cost i s often approximated by the Average Incremental Cost (AIC), which i s the average unit cost water in an incremental project to capture, transport, treat, and distribute water and then collect, treat and safely dispose of the wastewater. Tariff The payment by consumers in return for a cubic meter of water and/or for a cubic meter of wastewater collected, removed from the house, and possibly treated before being returned to the environment. A tariff can be expressed in financial or economic costs that are not necessarily equal. Subsidy The difference between ‘the marginal cost of water and wastewater and the tariff. If the difference i s negative the customer pays a tax instead o f receiving a subsidy. Cross subsidy The difference between the average financial costs of service and the tariff paid by each customer category. Connection fee The contribution from customers to the investment cost o f a water or sewerage connection. Transparent subsidies Subsidies where the exact amount and recipient are known. Unaccounted water The ratio between the difference between metered water produced and (the metered consumption + estimated non-metered water consumption) divided by the metered water production. Regulation The monitoring and control of water and wastewater service providers to ensure that they comply with the laws and regulations o f the sector (“the rules of the sector”). Regulation consists o f quality regulation where the quality and efficiency of the service provided i s monitored and compared, and economic regulation that applies objective and predictable criteria for setting and monitoring tariffs. Annex A 34 Economic or tariff regulation The monitoring and authorization to charge tariffs given to water and wastewater service providers. Regulatory agency The institution responsible for regulating services and tariffs. Quality regulation Collecting and analyzing data on the service coverage, quality, efficiency, and tariffs of water and wastewater services in order to compare how well different service providers perform. This type of regulation i s also called yardstick regulation where each company’s performance i s compared to the sector benchmark. Efficiency The situation when water and wastewater companies use the minimum amount of economic resources to meet the demand o f consumers for water and wastewater services. Sustainability Sustainability has a number of meanings in connection with water supply and sanitation services: (i)sustainable services that provides uninterrupted service over the foreseeable future and i s made possible by competent operations and adequate maintenance; (ii) financially sustainable operations and utilities are those that manage to collect sufficient financial revenue to defray all their financial costs; and (iii) environmentally sustainable water supply and wastewater operations are those that do not upset the ecological balance, e.g. by treating wastewaters generated before returning them to the environment. Private Sector Participation PSP comprises contract forms where private companies perform different functions. In outsourcing the water and wastewater company sub-contracts for example meter installations and maintenance with private firms and pays a fixed amount per meter installed. In management contracts the private company operates and maintains the systems and receives an annual fixed amount for its services. In leases the private company operates and maintains the systems and i s paid out of the tariffs that it collects ( the private company assumes “collections risk”). In concessions the private company also has to finance and expand system capacity in addition to operating and maintaining the systems. Present value The discounted value of future benefits and costs that have been reduced by an annual percentage called the discount rate. Future benefits and costs are therefore worth less in present value if they are earned or incurred later rather than sooner. Risk The danger that future benefits and costs will be respectively lower or higher than projected. The party to whom risks are allocated suffers the consequences if the danger actually materializes. Guarantee The assurance of a party to compensate another party if risk materializes.Government guarantees are also called sovereign. 35 Annex B Annex B. De( iicated Water Supply and Sanitation Lending, by Country, 197 1-2001 (Constant US $ 2000 millions, -_l-*",--me-m-.* L1 Commitments net o f Cancellations) .L -1 As a share of Cumulative 1970-79 Country _. __-.1980-89 __ - 1990-2001 Total Total Lending Share Brazil 485 1582 1019 3086 13% 13% India 667 561 906 2135 9% 23% Algeria 139 997 116 1252 5% 28% Mexico 280 260 689 1230 5% 34% Colombia 54 1 355 238 1135 5% 38% Nigeria 152 47 1 452 1075 5% 43% Turkey 107 500 369 975 4% 47% China 106 675 78 1 3% 51% Philippines 227 159 297 683 3% 54% Morocco 208 247 221 676 3% 57% Tunisia 274 199 125 597 3y o 59% Indonesia 84 130 349 563 2% 62% Pakistan 52 266 244 562 2% 64% Korea, Republic of 358 186 545 2% 67% Argentina 48 349 397 2% 68% Yugoslavia, former 226 106 332 1% 70% Kenya 151 8 115 273 1% 71% Jordan 47 170 56 273 1% 72% Senegal 4 32 229 265 1% 73% Cote d'lvoire 90 66 87 243 1% 74% Syrian Arab Republic 188 42 230 1% 75% Bangladesh 34 62 117 213 1% 76% Iran, Islamic Republic of 205 205 1% 77% Peru 14 19 158 191 1% 78% Egypt, Arab Republic of 99 91 189 1% 79% Thailand 128 61 189 1% 80% Vietnam 188 188 1% 80% Yemen, Republic of 50 57 69 176 1% 81% Chile 127 36 162 1% 82% Uganda 51 110 161 1% 83% Paraguay 10 18 116 144 1% 83% Poland 143 143 1% 84% Sri Lanka 17 95 26 137 1% 84% Malaysia 75 55 130 1% 85% Ghana 37 43 48 128 1% 86% Malawi 13 29 83 125 1% 86% Nepal 46 43 34 122 1% 87% Russian Federation 121 121 1% 87% Ecuador 54 35 32 120 1% 88% Congo, Democratic Republic of 42 74 117 1% 88% Mozambique 115 115 0% 89% As a share of Cumulative Country 1970-79 1980-89 1990-2001 Total Total Lending Share ~ - _ _ . - . I .. .._ Bulgaria 103 103 0% 89% Zambia 21 73 93 0% 90% Madagascar 51 42 93 0% 90% Guinea 21 44 26 92 0% 90% Uzbekistan 84 84 0% 91% Bolivia 9 15 58 82 0% 91 % Israel 77 77 0% 91 % Tanzania 35 39 74 0% 92% Burkina Faso 74 74 0% 92% Ethiopia 31 38 69 0% 92% Portugal 69 69 0% 93% Cyprus 15 0 53 68 0% 93% Costa Rica 41 27 68 0% 93% Azerbaijan 64 64 0% 94% Uruguay 26 33 58 0% 94% Rwanda 37 20 57 0% 94% Niger 9 47 57 0% 94% Lebanon 56 56 0% 95% West Bank and Gaza 48 48 0% 95% Botswana 15 33 48 0% 95% Afghanistan 47 47 0Yo 95% Burundi 12 35 47 0% 95% Nicaragua 43 4 46 0% 96% Jamaica 32 12 43 0% 96% Venezuela 41 41 0% 96% Panama 24 16 40 0% 96% Sierra Leone 38 38 0% 96% Croatia 0 38 38 0% 96% Slovenia 35 35 0% 97% Cameroon 34 34 0% 97% Somalia 10 23 33 0% 97% Bosnia-Herzegovina 33 33 0% 97% Haiti 11 22 33 0% 97% Cambodia 32 32 0% 97% Hungary 32 32 0% 97% Turkmenistan 32 32 0% 98% Armenia 31 31 0% 98% Mauritius 18 13 31 0% 98% Bahamas, The 18 12 30 0% 98% Benin 19 11 29 0% 98% Macedonia, former Yugoslav Republic of 29 29 0% 98% Guyana 28 28 0% 98% Singapore 28 28 0% 99% Greece 28 28 0% 99% Romania 26 26 0% 99% 37 Annex B --s-s-------.-- As a share of Cumulative Country 1970-79 1980-89 Total Total Lending Share - I I_--1990-2001 _ _II .-”_ Honduras 26 26 0% 99% Trinidad and Tobago 25 25 0% 99% Kazakhstan ’ 25 25 0% 99% Gabon 25 25 0% 99% Ukraine 24 24 0% 99% Albania 23 23 0% 99% Lesotho a 12 20 0% 99% Liberia 13 7 20 0% 100% Mongolia la 18 0% 100% Togo 17 17 0% 100% Mali 14 14 0% 100% Comoros 11 11 0% 100% Mauritania 11 11 0% 100% Gambia, The 8 a 0% 100% Swaziland 6 6 0% 100% Dominican Republic 5 5 0% 100% Guatemala 4 4 0% 100% Caribbean 0 0 0% 100% Grand Total 51 03 8132 9733 22968 100% 100% Annex C 38 Annex C. Dedicated W a t e r Supply and Sanitation Lending, by Sector, 1970-2001 (Constant US $ 2000 millions1 Commitments Net of Cancellations Share of Lending - I - - Sub-sector 1970-79 1980-89 1990-2001 Total l__I _l_l_ - .I_- - - __ I I - 1970-79 1980-89 1990-2001 Other Water, Sanitation 14 191 1142 1394 0 3% 2 3% 11 7% Rural Water Sup/Sanitatron 13 279 1079 1371 0 3% 3 4% 11 1% Sewerage 593 452 1821 2865 11 6% 5 6% 18 7% Urban Water Supply 4464 6803 4063 15329 87 5% 83 6% 41 7% Water, Sanitation Adjustm 19 409 1628 2056 0 4% 5 0% 16 7% Total 5103 8132 9733 22968 100.0% 100.0% 100.0% __-_ - - -- - - I - - - __ - I___ Number o f Projects Percentage Share - ___ - - ____ Sub-sector __ 1970-79 1980-89 ___ 1990-2001 ~~ 1970-79 1980-89 1990-2001 Other Water, Sanitation 1 7 16 1% 5% 11% Rural Water Sup/Sanitation 2 8 24 2% 6% 17% Sewerage 14 10 26 14% 8% 18% Urban Water Supply 84 99 52 235 82% 77% 37% Water, Sanitation Adjustm. 1 5 24 1% 4% 17% Grand Total 102 129 142 373 100% 100% 100% - I - __ 39 Annex D Annex D. Dedicated Water Supply and Sanitation Lending, by Region, 1970-2001 (Constant U S $ 2000 millions) Commitments Net of Cancellations Share of Lending -___ Region 1970-79 198589 1990-2001 198589 19952001 I_ - AFR 652 1173 1678 13% 14% 17% EAP 542 869 1745 11% 11% 18% ECA 445 640 1231 2316 8% 13% LCR 1520 2621 2856 6997 30% 32% 29% MNA 1080 1803 897 3780 I 21% 22% 9% SAR 863 1027 1327 13% 14% Total 5103 8132 9733 22968 3217 100% 100% _- - _-- - - - ~~~~ I_ Number of Projects __ Sub-sector 1970-79 1980-89 19952001 _ _I __ _ _ I __ AFR 23 37 32 EAP 9 13 24 ECA 9 9 26 44 9% 7 Yo 18% LCR 26 31 30 87 25% 24% 21% MNA 22 26 16 64 22% 20% 11% SAR 13 13 14 40 13% 10% 10% Total 102 129 142 373 1 100% 100% 100% -_---lll_l_.I - -- -- 1 “I __- .-- - Annex E 40 Annex E. Non-dedicated Water Supply and Sanitation Lending, by Country, 1990-2001 (Net Water Commitments constant " -.-I-y--- U -- S $2000 m)x y__(_II Water Lending as a Share Cumulative Countrv 1990 I991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 Total of Total Lending . ._.... . . . China 89 112 108 61 383 104 156 171 182 1367 39% 39% Indonesia 63 77 48 57 118 41 39 36 479 14% 53% Brazil 22 88 37 11 45 29 74 307 9% 61% Vietnam 175 175 5% 66% Mexico 3 97 100 3% 69% Peru 26 70 96 3% 72% Argentina 49 32 4 9 94 3% 75% Pakistan 59 3 18 80 2% 77% Tanzania 1 62 6 69 2% 79% India 12 43 13 69 2% 81% Egypt, Arab Republic of 30 30 2 5 67 2% 83% Russian Federation 44 44 1% 84% Ethiopia 39 39 1Yo 85% Venezuela 18 14 32 1% 86% Croatia 19 11 30 1% 87% Honduras 3 2 7 5 2 9 28 1% 88% Tunisia 22 6 28 1% 88% Ghana 9 8 8 2 26 1% 89% Yemen, Republic of 11 4 8 22 1Yo 90% Nicaragua 4 5 12 20 1% 90% Colombia 11 8 19 1% 91 % Madagascar 7 11 18 1% 91 % Guatemala 6 11 18 1% 92% Malawi 10 7 17 h 0O 92% Swaziland 17 17 0% 93% Jordan 16 16 0% 93% Ecuador 5 6 4 15 0% 94% Bangladesh 14 14 0% 94% Burkina Faso 12 12 0% 94% Cote d'lvoire 5 7 12 0% 95% Zimbabwe 11 11 0Yo 95% Panama 10 10 0% 95% Bolivia 7 3 10 0% 96% Angola 4 5 9 0% 96% Togo 9 1 9 0% 96% Mali 8 8 0% 96% Alaeria 3 5 8 0% 97% 41 Annex E Water Lending as a Share Cumulative Country 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 Total of Total Lending - - __ - I_ ___ Bosnia- Herzegovina 1 6 8 0% 97% Armenia 3 2 2 8 0% 97% Benin 3 2 3 7 0% 97% Georgia 2 5 7 0% 98% Lithuania 5 2 7 0% 98% Belize 3 4 7 0% 98% Cambodia 2 5 7 0% 98% Albania 1 4 2 7 0% 98% Burundi 2 4 6 0% 98% Latvia 3 3 6 0% 99% Bhutan 5 5 0% 99% Rwanda 5 5 0% 99% Sri Lanka 4 4 0% 99% Sudan 4 4 0% 99% Mauritius 4 4 0% 99% Mozambique 4 4 0% 99% Morocco 3 3 0% 99% Guinea-Bissau 3 3 0% 100% Zambia 2 2 0% 100% Eritrea 2 2 0% 100% Lebanon 2 2 0% 100% Djibouti 2 2 0% 100% Haiti 2 2 0% 100% Philippines 2 2 0% 100% Moldova 1 1 0% 100% Lesotho 1 1 0% 100% Bulgaria 1 1 0% 100% Estonia 1 1 0% 100% Poland 0 25 025 001% 100% GrandTotal 27 - 117 123 166 508 252 694 307 255 309 243 ~ ~ L x y - y-III - * I 511 3512 100% Annex F 42 Annex F. Non-dedicated Water Supply and Sanitation Lending, by Sector and Year, 1990-2001 (Net Water Commitments constant U ---e S $2000 -- " -- m) -- -- Sector Group 1990 1991 1992 1993 1994 1995 1996 1997 1998 7999 2000 2001 Total ____I--I_ - - ___ Agriculture 11 8 2 1 8 37 87 154 Education 59 13 72 Environment 5 51 45 210 70 114 7 111 170 171 954 Finance 18 18 Hlth, Nutn & Popultn 43 43 Multisector 5 5 Public Sector Mgmt 35 35 Social Protection 37 2 19 32 7 157 120 18 54 23 64 533 Transportation 12 10 22 3 4 52 Urban Development 22 69 63 89 205 163 423 150 51 54 7 351 1646 Total 27 117 123 166 508 252 694 307 255 309 243 511 3512 __ - I . - - -- - --___I Sector Group I990 1991 1992 1993 1994 1995 1996 1997 1998 I999 2000 2001 Total -I_I I___- -- - Agriculture 1 2 1 1 1 4 7 17 Education 1 1 2 Environment 1 2 1 3 4 2 1 3 2 2 21 Finance 1 1 Hlth, Nutn & Popultn 2 2 Multisector 1 1 Public Sector Mgmt 1 1 Social Protection 4 1 4 2 2 11 6 4 11 4 6 55 Transportation 1 1 1 1 1 5 Urban Development 1 3 5 2 7 10 8 4 7 4 2 4 57 Total I_ -_-I_I _I_I __ __ - 2 8 10 8 __ 14 18 -- - 21 - -- 13 20 19 10 19 162 43 Annex G Annex G. OED Ratings for Dedicated Water Supply and Sanitation Projects, 1970-2001 Outcome _ _ Ratings . (99) _ ..- Sustainability Ratings (?A) Highly Country Satisfactory Substantial Modest .... .-.. Negligible- _II- Likely - Uncertain Unlikely Unlikely Albania 0% 0% 0% 100% 0% 0% 0% 100% Algeria 40% 0% 20% 60% 0% 40% 40% 0% Argentina 100% 50% 50% 0% 100% 0% 0% 0% Bahamas 50% 0% 50% 0% 0% 50% 0% 0% Bangladesh 80% 0% 40% 20% 0% 40% 20% 0% Benin 67% 33% 0% 33% 33% 33% 0% 0% Bolivia 75% 50% 25% 0% 50% 25% 0% 0% Bosnia-Herzegovina 100% 0% 100% 0% 0% 0% 50% 0% Botswana 100% 0% 0% 0% 0% 0% 0% 0% Brazil 57% 21% 36% 7% 29% 29% 7% 0% Burkina Faso 0% 0% 0% 100% 0% 0% 100% 0% Burundi 100% 33% 33% 0% 67% 0% 0% 0% Cameroon 100% 0% 50% 0% 0% 50% 0% 0% Caribbean 100% 100% 0% 0% 100% 0% 0% 0% Chile 75% 50% 50% 0% 100% 0% 0% 0% China 75% 50% 50% 0% 75% 25% 0% 0% Colombia 69% 0% 8% 46% 15% 38% 0% 0% Congo, DR 67% 33% 0% 33% 0% 33% 33% 0% Costa Rica 0% 0% 100% 0% 0% 100% 0% 0% Cote d'lvoire 100% 0% 40% 0% 0% 40% 0% 0% Croatia 0% 0% 100% 0% 100% 0% 0% 0% Cyprus 75% 50% 0% 0% 50% 0% 0% 0% Ecuador 50% 0% 0% 50% 0% 50% 0% 0% Egypt 60% 0% 20% 0% 0% 0% 20% 0% Ethiopia 100% 0% 0% 0% 0% 0% 0% 0% Gabon 100% 0% 0% 0% 0% 0% 0% 0% Gambia 0% 0% 0% 100% 0% 100% 0% 0% Ghana 80% 20% 20% 40% 40% 40% 0% 0% Greece 100% 0% 0% 0% 0% 0% 0% 0% Guatemala 0% 0% 0% 100% 0% 0% 100% 0% Guinea 50% 50% 0% 0% 50% 0% 0% 0% Haiti 0% 0% 0% 50% 0% 0% 50% 0% Honduras 0% 0% 0% 100% 0% 100% 0% 0% India 71% 36% 21% 14% 29% 36% 7% 0% Indonesia 88% 25% 63% 0% 25% 25% 38% 0% Iran 100% 0% 100% 0% 0% 0% 0% 0% Israel 100% 0% 0% 0% 0% 0% 0% 0% Jamaica 67% 0% 33% 0% 0% 0% 33% 0% Jordan 33% 50% 0% 17% 0% 67% 0% 0% Kenya 57% 0% 29% 43% 14% 14% 43% 0% Korea, Rep 100% 57% 43% 0% 86% 14% 0% 0% Lebanon 0% 0% 0% 100% 0% 0% 100% 0% Lesotho 100% 67% 0% 0% 67% 0% 0% 0% Liberia 0% 0% 0% 50% 0% 0% 50% 0% Madagascar 33% 33% 33% 33% 33% 0% 67% 0% Malawi 100% 67% 0% 0% 0% 67% 0% 0% Malaysia 100% 0% 25% 0% 25% 0% 0% 0% Annex G 44 Outcome Ratings (%) ID Ratings _. (%) Sustainability Ratings (%) Highly Country ------..I ._- Satisfactory Substantial Modest ._ - Negligible Likely Uncertain Unlikely Unlikely Mali 100% 0 Qio 100% 0% 0% 100% 0% 0% Mauritania 0% 0% 0% 100% 0% 0% 100% 0% Mauritius 100% 100% 0% 0% 100% 0% 0% 0% Mexico 29% 14% 29% 29% 14% 43% 14% 0% Morocco 100% 50% 17% 0% 67% 0% 0% 0% Nepal 0% 0% 25% 25% 0% 0% 50% 0% Nicaragua 100% 0% 0% 0% 0% 0% 0% 0% Niger 0% 0% 100% 0% 0% 100% 0% 0% Nigeria 25% 0% 75% 25% 0% 25% 75% 0% Pakistan 60% 0% 60% 0% 20% 20% 20% 0% Panama 50% 0% 0% 50% 0% 0% 50% 0% Paraguay 100% 33% 0% 33% 67% 0% 0% 0% Peru 0% 0% 0% 100% 0% 50% 50% 0% Philippines 63% 25% 38% 13% 25% 50% 0% 0% Portugal 100% 0% 0% 0% 0% 0% 0% 0% Rwanda 50% 50% 0% 50% 50% 0% 50% 0% Senegal 100% 0% 50% 0% 0% 50% 0% 0% Seychelles 100% 0% 100% 0% 100% 0% 0% 0% Singapore 100% 0% 0% 0% 0% 0% 0% 0% Slovenia 100% 100% 0% 0% 100% 0% 0% 0% Somalia 100% 50% 0% 0% 0% 50% 0% 0% Sri Lanka 75% 25% 0% 25% 0% 50% 0% 0% Swaziland 100% 0% 0% 0% 0% 0% 0% 0% Syria 50% 0% 0% 0% 0% 0% 0% 0% Tanzania 0% 0% 0% 50% 0% 0% 100% 0% Thailand 100% 50% 0% 0% 50% 0% 0% 0% Togo 100% 100% 0% 0% 0% 100% 0% 0% Trinidad & Tobago 0% 0% 100% 0% 0% 0% 100% 0% Tunisia 100% 36% 0% 0% 45% 0% 0% 0% Turkey 83% 17% 50% 17% 33% 33% 17% 0% Uganda 67% 33% 33% 33% 67% 0% 33% 0% Uruguay 100% 0% 100% 0% 100% 0% 0% 0% Uzbekistan 0% 0% 0% 0% 0% 0% 0% 0% Yemen Arab Republic 75% 0% 0% 0% 0% 0% 0% 0% Yemen, PDR 100% 0% 25% 0% 0% 25% 0% 0% Yemen, Rep 67% 0% 67% 33% 0% 100% 0% 0% Yugoslavia, former 83% 17% 0% 0% 0% 17% 0% 0% Zambia 0% 0% 0% 100% 0% 0% 100% 0% Grand Total 68% ~, "l*.. 20% ..*- I "_ 25% 17% 24% 23% 15% 0% ......... .. . . . . . . . . . . . . . . . . . . . _. i___ ......... ._ ........... ......... ................ ~- ................. . . ......... . -. . . . . . . . . ...... ~~ . .. .. .. .. .. - 1 ................. - I W 7 N W b .... a m w N m kcor. 0 0 0 0 m m u , b r . r w 0 0 0 W r.- v) N O - (D w m w v) v ) W b b W b r . 7 v) wlnm .r ava m Z Q Z - - aaa z 0 z z z N m aaa m z z z N gaa W 0 ) F Z Z b N gaa m Y Z Z 47 Annex I Annex I.Bank Economic and Sector Work in Water Supply and Sanitation Sector __ __ - I 1970-79 1980-89 - ____ 1990-2001 Total Policy Research Working Paper 1 16 17 Departmental Working Paper 19 6 13 38 PPIAF 13 13 Working Paper 1 2 10 13 Sector Report 10 9 9 28 World Bank Technical Paper 12 9 21 Water & Sanitation Discussion Paper 7 7 Working Paper (Numbered Series) 3 18 4 25 Infrastructure Action Program 3 3 UNDP-Water & Sanitation Program 3 3 Internal Discussion Paper 1 1 Staff Working Paper 1 1 World Bank Discussion Paper 1 1 Total I -"" ~ 1_ I I^I- I I 34 I-. .* "*- 50 a7 171 Annex J 48 Annex J. Lessons from Water Supply and Sanitation Project Cohort, FY98-2 00 1 Lessons From FY99 Water Supply and Sanitation Project Cohort Africa Burkina Faso Ouagadougou Water Supply Engineering Credit (Credit 2519) Long-term technical assistance does not guarantee long-term institutional strengthening o f a public water supply company. Lasting improvements require an enabling framework and contractual incentives for a private operator to improve efficiency o f financial performance. Ghana Water Sector Rehabilitation Project (Credit 2039) Stable and sustained management i s a necessary condition for project success. As it happened GWSC had none and project performance suffered: Operating efficiency did not improve, the cash flow was much worse than projected, and technical progress turned worse with components actually canceled for lack o f counterpart funding. Under the circumstances the twinning arrangement with Thames Water was ineffective. It i s likely that the Ghana water supply and sewerage sector performance will not improve until a private operator takes over with appropriate incentives to increase efficiency and produce operational profits. Kenya Nairobi 1 11Water Supply (Credit 2060) This Project was doomed to fail for lack o f an enabling legal and regulatory environment that could promote financial and institutional autonomy that could make the project sustainable. One result was that the service coverage and quality to the low-income, most needy population improved only marginally. The Project illustrates the thesis “ Old Lessons in New Projects”. The main lesson i s that the technical analysis and implementation were allowed to proceed with the Project’s socio-economic, financial, and institutional aspects receiving insufficient attention. The technical concept was also flawed since distribution investments and efforts to increase connection rates lagged production investments. Future Bank operations are likely to prove futile unless the political will appears to change the enabling framework, and the incentive structure for water supply and sewerage operators. Mauritania Water Supply Project (Credit 2389) The lessons from this project cover important issues (such as, Borrower commitment to policy reform, sound capacity assessment o f the responsible project entity, match between the objectives and components o f the project, and coordination o f efforts by various participating funding agencies), and are largely applicable to preparation o f any urban water and sanitation project. There are two lessons that should be especially highlighted as follows: (i) appropriate timing o f new investments in relation to institutional strengthening o f the executing agency; and (ii) tariff increases to achieve financial sustainability should be firmly linked with parallel efficiency gains in the O&M o f the system. Uganda Second Water Supply Project (Credit 2124) The most significant lesson from this project i s that well-designed specific components are necessary to address the many and complex issues related to major institutional reforms and improvements, as was the case in Uganda. Furthermore, to achieve meaningful results: (i) a realistic and gradual implementation program needs to be established; and (ii) TA and training, to be effective, need to be supported by appropriate incentives within the operating framework. Lessons from two previous projects were not adequately considered; as a result, “the quality at entry” o f this project was poor, not only with regard to the institutional objectives, but also as no 49 Annex J component was designed for dealing with an essential objective o f sanitation improvements. The above, and many other useful project specific lessons have already lead to a detailed analysis o f necessary reforms, privatization being one option under consideration. Guinea Second Water Supply Project ((21 985) The completion report gives several useful project-specific lessons learned. An additional aspect can be cited to highlight an important “Quality at Entry” lesson: The project i s a “good practice” example o f the Bank obtaining a firm commitment from the Borrower for crucial restructuringo f the sector prior to project appraisal and the credit effectiveness. I t provides sound evidence on the importance o f such early agreements on fundamental reform and changes for successful project implementation and sustained outcome. Nigeria Lagos State Water Supply Project (L2985) Expectations for project start up should take into account the implementing agency’s prior experience with Bank projects. Weak institutions should be assisted incrementally with projects that are modest in design and objectives. A more radical approach, such as private sector participation in operations, could help achieve sustained results. East Asia and Pacific Indonesia Second Jabotabek Urban Development Project (L3219) The completion report presents several important project-specific lessons, many o f which have wider application. The following i s particularly highlighted: “Complex institutional issues surrounding water supply should be addressed with equal priority to physical objectives and the design o f respective components should be adequately specific. Institutional performance cannot simply be improved by providing technical assistance and training”. Philippines Water Supply, Sewerage and Sanitation (L3242) Projects with multi-agencies should be avoided. I t i s recommended that a single agency should serve as the executing agency. The capability and experience o f Local Government Units i s important to allow decentralized aspects o f projects to function successfully. South Asia India Hyderabad Water Supply and Sanitation Project (Credit 2115) The project provides a strong lesson on the importance o f competent and committed senior management (including heads o f various operational departments) supported by effective and timely staff training in the laying o f a solid foundation for a newly established entity when at the same time a major investment program was being carried out. India Maharashtra Rural Water Supply and Environmental Sanitation Project (Credit 2234) The completion report addresses some key deficiencies in project preparationthat caused less than satisfactory outcome at the end. These are mainly in the areas of: (i) design o f components to appropriately match project objectives; (ii) community participation; and (iii) institutional and financial reforms and arrangements. Furthermore, alternative technical solutions should have been evaluated so as to make new services better affordable to users. The lessons from this project should be carefully reviewed when preparing follow-on projects in Maharashtra and elsewhere in India. Annex J 50 Europe and Central Asia Turkey Istanbul Water Supply and Sewerage Project (Loan 2888) The completion report presents several lessons, which have a wider application on the importance of: (i)a thorough and timely mid-term review, (ii) consultations with affected population at project design stage, and (iii)realism o f project’s design assumptions as well as project scope that i s commensurate to the proven capacity o f the implementing entity. One finding especially worth mentioning i s the fundamental role o f a careful institutional analysis at appraisal to ensure that project scope does not become unrealistic and overly ambitious. Latin America and the Caribbean Chile Second Valparaiso Water Supply and Sewerage Project (L3331) This project reinforces the case for importance o f sound preparation o f all components prior to project appraisal. In addition, an approach for tackling the risks in a complex project as this one should be agreed upon at the outset. When a project gets into serious problems early in i t s implementation, a comprehensive review (e.g. at mid-term) should be carried out to determine the most feasible course o f action, including radical curtailment o f the project scope or cancellation o f the loan as an ultimate solution. Bolivia Major Cities Water and Sewerage Rehabilitation Project (C2187) The completion report presents several useful project specific lessons. The following are emphasized for broader application: (i) High turnover o f Task Managers in the Bank does not foster sound supervision, does not support implementation efforts and should be avoided; and (ii) the successful privatization o f the sector operations in L a Paz shows the importance o f strong political will and commitment. Paraguay Third Rural Water Supply and Sanitation Project (Loan 3519) The project experience suggests three important lessons. First, it i s important that the project presented to the Board for approval contain objectives that are realistic and consistent with the project design and the resources provided. Second, given a set o f expected impacts (as indicated by the objectives) the project needs to make adequate provisions to monitor the degree o f their achievement. Third, the provision o f water and sanitation to rural areas lends itself to the use o f the Bank’s “Adaptable Program Lending” instrument. An “APL” would have been relevant for Paraguay where the Bank has recently approved a fourth rural water supply and sanitation project. Caribbean The Wider Caribbean Initiative on Ship-Generated Waste project There are three broad lessons. First, a thorough preparation or, preferably, a flexible project implementation schedule need to be allowed for regional projects involving several countries, languages and legal systems. All countries need to be involved in project preparation from the beginning. Second, in regional projects the responsibilities between the implementing and the executing (‘the borrower’) agencies must be spelled out clearly in the project preparation phase, and not left for ‘ad hoc’ decision making during the project. This only causes procedural disputes and issues o f turf between two competent agencies. And third, for regional projects, a serious consideration should be given to create a regional organization either execute or supervise the project on behalf o f the beneficiaries. 51 Annex J Lessons F r o m F Y 9 9 W a t e r Supply and Sanitation Project Cohort Africa Nigeria Lagos Drainage and Sanitation (Credit 251 7 ) Projects need to be designed, supervised, and monitored to reach stated objectives. In particular, sustainable benefits require autonomy and incentives for stakeholders to strive to achieve the stated objectives. Drainage projects need to adopt integrated solutions ensuring well-functioning tertiary, secondary and primary drainage over the long-term; which requires adequate and dependable financing, both o f investment and o f operations and maintenance., Kenya Second Mombassa and Coastal Water Supply Engineering and Rehabilitation Project (Credit 2333) (a) The water utility implementing the Project should be financially viable and autonomous and, without addressing this issue, the attainment o f physical objectives ultimately proves pointless; (b) Cross-subsidies are counter-productive if they imperil the financial viability o f the water utility by postponing the necessary tariff adjustments in the subsidized areas; (c) Simple technical assistance i s no substitute for private sector participation to reach improved efficiency and financial viability; (d) The preparation o f quality technical studies by itself cannot attract financing if the fundamental issues o f insufficient institutional and financial viability are left pending; (e) Improved financial viability should stand on two legs: regular tariff adjustments and improved commercial procedures; (f) Detailed project preparation, and in particular confirming the availability o f ground water, should be undertaken before appraisal. Rwanda Second Water Supply & Sanitation (Credit 1783) (a) Early operations with inexperienced executing agencies should be simple and offer hope o f tangible progress in the form o f better and extended service. (b) Cofinancing should be designed in such a way that the withdrawal o f one or more co-financiers (as happened following the 1994 civil war) should not stop entirely components but at the most reduce their scope. (c) Project ownership and government commitment are key factors to make success possible and this i s particularly true o f innovative project components that represent a sharp break with the practice o f the past. East Asia and Pacific China Rural Water Supply and Sanitation Project (Credit 2336) (a) Demand for improved latrines i s correlated with higher income levels and, in this case, the income levels o f the rural poor were too low to support the latrine improvement strategy; (b) Investing in model villages with external financing did not yield the desired demonstration effect in non-project villages where the same external financing was not available; and (c) Training o f water plant operators and accountants i s critical to ensure sustainability o f the systems and financing must be provided to ensure that it will occur. China Changchun Water Supply and Environment (Credit 2457) (1) Design and project management need to be handled by the best experts available. The reluctance o f the Borrower to use “expensive” I D A funding for technical assistance proved “penny-wise but pound-foolish.’’ (2) The financial and institutional arrangements and policies should have been analyzed and corrected before the project was approved and implemented; (3) More resources need to be devoted for supervision o f large projects with first-time borrowers and particularly in the area o f procurement; and (4) it i s better to declare a project to have positive Annex J 52 demonstration effects once the experience has been rated as positive. In the event, most o f the project lessons turned out to be negative from which it i s also possible to learn. South Korea Seoul And Kwangiu Sewerage (Loan 3590) (1) A comprehensive wastewater strategy should include both water conservationand wastewater collection and treatment. Where water supply and wastewater services are administratively separate it i s more difficult to achieve such an integrated strategy. In particular, the water agencies’ commitment to water conservation may be difficult to achieve since they do not perceive the large, down-stream costs in wastewater collection and treatment; (2) Financial covenants requiring financial rates-of-return o f 2 or 3 percent are unrealistic to achieve in the short term when the investment programs are very large. I t i s simply inappropriate to expect assets with a very long l i f e to be recovered in a relatively short time. A covenant with a minimum percentage contribution to investments from the internal cash generation would likely have been more appropriate; (3) Though not included in the project design, Kwangju successfully privatized the operations and maintenance o f its wastewater treatment plant with cost savings in the order o f 30 percent. This example i s o f wide interest in Korea and elsewhere. South Asia Nepal Urban Water & Sanitation Rehabilitation (Credit 2239) Most o f the lessons are trite and have been known since the mid-1970s. Once again it should be recalled that: (1) I n the face o f general agreement o f what had gone wrong in the past (under three previous Bank-financed water supply projects) it was a fallacy to expect that half-measures that failed to radically change the legalhegulatory framework and the incentives would produce any better results than in the past; (2) Politicization o f tariff increases, staff recruitment, and o f other administrative and financial matters will destroy the best-intentionedproject and the costs will ultimately be borne by the consumers in the form o f poor and unsafe service; Pakistan Second Karachi Water & Sanitation (Credit 1987) (1) Without a fundamental legislative and regulatory reform, including changed incentives and contracting o f a private operator, the project was doomed from the start; (2) Financial covenants are ineffective if the water supply and sewerage authority i s not given the authority and means to comply with them. In the particular case o f KWSB, excessive politicization o f the tariff settings and o f the management added to the difficulties; (3) The project design should have incorporated more o f community participation and especially so under the sanitation component where Karachi had gained valuable experience from the Orangi sanitation works. Sri Lanka Community Water Supply and Sanitation Project (Credit 2442) The completion report presents several important findings and lessons applicable to the project and useful for many future projects in Sri Lanka and elsewhere. One that should be further highlighted i s the need for rigorous M&E during project implementation, the key for achieving 53 Annex J sustained results. Similarly, the continued funding o f TA activities i s the key to critical institutional improvements. Europe and Central Asia Slovenia Slovene Coast Water Supply and Sewerage Project (L3070) First, demand forecasting needs to be conservative in the face o f major macro-economic and structural reforms that can often reduce industrial and commercial demand for water. Under the original project design demand was probably over-estimated. Second, successful programs to reduce unaccounted-for-waterrequire strong Borrower commitment and a well-designed and strongly supported action program. The Borrower’s success in reducing unaccounted-for-wateri s exemplary. Third, where the Bank chooses to include financial covenants it should pay sufficient attention to ensure effective financial supervision. Turkey Ankara Sewerage Project (Loan 3151) (1) I t i s preferable to have a few, simple and consistent objectives that can be achieved when lending to a novice institution for the first time rather than overreach with a multitude o f objectives that ultimately remain unfulfilled; (2) I t i s essential to reach full agreement on procurement procedures before going ahead with project implementation. The procurement problems in the early years o f the Ankara project are largely due to procurement disagreements. (3) The Project calls into question the merits of financial covenants when the implementing agency, ASKI, does not have de facto full authority to take all actions (such as tariff adjustments) to assure full compliance. Cyprus Southeast region sewerage and drainage (Loan 3398) (1) The difficulties o f the joint venture approach for the Paralimni and Ayia Napa Sewerage Board to collaborate on the implementation o f a joint sewage treatment plant are instructive for all projects that envisage close collaboration between h l l y independent legal bodies, such as municipalities or autonomous boards; (2) The requirements in terms o f time and leading-edge advice on how to procure and implement a turnkey contract for sewage treatment plants are easily underestimated and resulted in substantial delays; (3) The assessment o f willingness-to-pay was inadequate. This resulted in resistance from hoteliers to pay since they were asked to cross-subsidize domestic consumers and since they were reluctant to write-off their own recent package treatment investments; (4) The project scope should have been defined in a more participatory manner to gain the support o f the general public, which would likely have facilitated the collection o f wastewater charges. Bosnia-Herzegovina Water Sanitation and Urgent Works (1) Emergency projects necessarily focus on meeting urgent needs but should also be designed to gradually meet objectives that are relevant for the long-term sustainability o f sector operations; (2) The multiplicity o f donors and their differing financing terms can create tensions between municipalities that receive widely differing financing terms for similar investments. The necessary harmonization o f financing terms could be helped by creating a central financing facility that would finance all emergency works under the same terms, and through which all external aid would flow; and Annex J 54 (3) Implementationshould utilize locally available expertise to a maximum, and (more expensive) expatriate technical staff should be limited to areas where they clearly add value. Middle East and North Africa Yemen A1 Mukalla Water Supply (Credit 1944) (a) The project demonstrates how close supervision and a flexible response to adverse external events can rescue a project with serious implementation problems; (b) Experienced contractors and consultants are also critical to provide continuity and support in the face o f rapidly changing circumstances and in the end can compensate for prolonged turmoil and weakness in the public sector agencies; and (c) Delays in local counterpart funds causes severe disruptions in project implementation. Republic o f Yemen Taiz Flood Disaster Prevention and Municipal Development Project (Credit 21 60) I D A made a deliberate decision to continue supporting the Project despite implementation difficulties due to the 1994 civil war. The more intensive and effective supervision through a strengthened IDA Resident Mission proved the key to rescue the Project and i t s associated flood control benefits. Yemen Tarim Water Supply (Credit 21 60) i)There i s a need for broad-based and comprehensive sector reform that emphasizes autonomy and private sector participation. This was indeed done in the follow-up project, the Sana’a Water Supply and Sanitation Project. (ii) IDA should have been quicker to take pro-active measures to restructure the project once unsatisfactory ratings were maintained for more than one year. The restructuring came rather late in the project Lebanon Coastal Pollution Control and Water Supply L4136 (i)I t i s essential to confirm the Government’s support o f a project as early as possible in the project cycle to minimize the risk o f expending substantial amounts to prepare a project that i s subsequently aborted. In this particular case, the political situation was fluid and in spite o f the Government’s assurances, the political support vanished. However, it would have been preferable to have the necessary parliamentary legislation o f sector reform approved before Board approval, which might have served as a bellwether o f the true political backing o f the project; (ii)Projects with parallel financing should be designed so that components could be self- contained with separate sources o f financing and separate execution. In this case, the EIB financing o f US$48 million was canceled since it was tied to the aborted Bank loan; (iii) The site for the wastewater treatment works at Kesrouan turned out to be controversial, although studies confirmed it to be the most feasible one. I t would have been preferable to advance the involvement o f all stakeholders in this controversial decision before including the component in the project. Latin America and the Caribbean Uruguay Water Supply Rehabilitation (Loan 2921) (a) The effectiveness o f twinning arrangements i s limited as long as the incentive framework continues to hamper efficiency improvements and institutional change; (b) Project monitoring should dovetail with the objectives and procedures o f the utility itself and not only serve the 55 Annex J Bank’s reporting requirements and they should focus not only on the borrower’s performance but also on the project itself; (c) I t i s preferable to set more modest objectives that match the country’s appetite for reform and institutional history and succeed in contrast to force too rapid reforms that stand little chance o f success. Haiti Port-au-Prince Water Supply (Credit 2052) (1) With the wisdom of hindsight the project should have been conditioned on private sector management with the private operator obliged to take on the risk for collecting for water distributed; (2) The water kiosks demonstrate the financial and operational success that community-selected water committees assisted by NGOs can produce in the distribution o f minimal water quantities in low-income areas; (3) Close coordination between co-financiers i s essential in order to share and achieve objectives. Trinidad and Tobago Water sector institutional strengthening (Loan 3 784) 1. The Bank should include legal and privatization specialists early in the process of privatization contract preparation to ensure the contract i s clear and enforceable; 2. When the main objectives o f a project are institutional reform and policy implementation an adjustable program loan may be more effective for managing conditionalities; 3. The Bank’s procurement rules need to evolve to deal with successive privatization contracts (for instance determining the eligibility of the incumbent winner of an Interim Operational Agreement to bid on a subsequent Long Term Agreement); 4. Financial audits should include monitoring o f compliance with specific covenants; 5. When projects are linked (in this case a Flood Control and Drainage Project was promised) this should be formalized to avoid misunderstanding; 6. Water Resources Management and Water Supply Sector Reform should be more closely coordinated yet the sectoral specializations of most task teams has meant these are followed independently of each other, with limited interaction. Lessons From FYOO Water Supply and Sanitation Project Cohort Africa Madagascar Antananarivo Plain Development (Credit 211 7) (1) One project should not try to address a large number of diverse issues such as agricultural development, flood control, institutional strengthening, taxation reform, urban development, and wastewater collection. This type o f “Christmas tree project “ i s bound to fall well short of expectations. (2) Co-financing should be targeted at discrete and physically unlinked project components to minimize disruptions of the type that followed AFD’s suspension o f disbursements from 1994- 97; Annex J 56 (3) The institutional and financial resource mobilization arrangements should be defined before implementationto increase the chances o f sustainability, rather than late during implementation as was the case for the project. Lesotho Highlands Water Phase 1A (L3393) a The Bank can leverage significant value, particularly in fostering regional cooperation and providing technical advice, even when it finances only a small proportion of costs (less than 3 percent.); 0 A sophisticated trust mechanism, like that devised by the Bank for this project, reduces perceived risk by saving potential financiers the embarrassment o f being seen to receive guarantees from a country under international sanctions (RSA in 1990.) Clear and equitable sharing o f benefits between international parties should be established at the design stage; 0 Vesting responsibility for rural development, water supply and sanitation, and other social facets, in an organization like LHDA, with a primarily engineering function, runs the risk they will devote insufficient attention to them; a Independent panels o f environmental and engineering experts provide valuable advice which impacts designs and reduces costs; a Participation o f agencies and communities should be promoted as early as the project conceptualization stage; a Technology transfer from expatriates to locals should be through structured training programs coupled with professional institution guidelines. East Asia and Pacific - Indonesia Sulawesi Irian Jaya Urban Development (L3340) 1, Physical achievements cannot be sustained unless they are supported by increased institutional capacity and changed attitudes towards service delivery and maintenance. 2. In the absence o f on-the-ground supervision, the preparation o f environmental assessment documents did little to improve environmental management during implementation. 3 . The Bank should give more attention to the impacts o f urban services and ways to measure them. 4. Local governments need to move away from a “project focus” towards more of an overall urban management approach. 5. Much more emphasis should be placed on community involvement in service delivery. Indonesia Water Supply and Sanitation for Low Income Communities (Loan 3629) (1) Rather than rely on line ministries projects o f this nature should try to maximize the community development in order to make the investments more relevant, stretch available project funding and increase the chances for sustainability; 57 Annex J (2) Project selection criteria need to provide for some flexibility to move the program forward and capitalize on strong community support o f the program; (3) The communities readily see the benefits o f water supply investments.In contrast, sanitation programs need additional support, both o f private latrine construction and of sanitation education in order to change the population’s sanitation habits. South Asia Pakistan Rural Water (C2228) (1) A demand-driven rural water supply and sanitation strategy based on strong community participation requires a longer time to take root than what i s usually offered by one single project. The AJK component performed better than the Balochistan and Sindh components precisely because it enjoyed a centuries old tradition o f community participation whereas the other two did not. (2) Changing the habits of excreta disposal implies a much greater effort than providing water supply. Symptomatically, the water supply investments performed much better than the latrine components. (3) Project objectives should be stated in terms that would allow quantitative monitoring o f progress towards reaching them. Europe and Central Asia Croatia Istria Water Supply (L3069) (1) Turnkey contracts are complicated to design and supervise. In this case, the contract went seriously wrong; (2) Demand projections and project design always need to incorporate flexibility, and particularly for transition economies; and (3) For the Bank to be able to influence the implementation o f a given component, it should finance a share o f costs, sufficient to provide leverage. In this case, the Bank financed nil o f the sewerage and treatment works, yet these had been included in the project. Latin America and the Caribbean Argentina Water Supply 1 1(L3281) on sub-project identification and selection: Detailed preparatory work should ensure that a project i s front-loaded with a significant pipeline of sub-projects at loan effectiveness. Grant financed technical assistance to prepare the engineering projects helps accelerate this process. on sub-loan allocation: Mechanisms should not be too rigid, allowing investment choices to be driven by demand. If a sub-loan remains unused for a period of time, it should be possible to readily reallocate unused amounts. Risks need to be spread (not allocating one half o f a loan to a single sub-project that could be cancelled, as happened in this case). Annex J 58 0 on monitoring indicators: A change o f strategy-toward privatization during this project-represents a significant challenge for monitoring and requires supervision missions and Borrower executing agencies to track agreed monitoring indicators closely on project executing agencies: Staffing them with a qualified team o f external consultants helps project execution and continuity as management changes, but fails to transfer project knowledge to local staff. 0 on institutional strengthening and reform: Major institutional reform i s needed for improved efficiency o f public utilities that lack incentives and are set in a culture o f ‘expand and spend’. 0 on private sector participation: Strong political commitment at the top to reform and a clear government strategy are necessary. Necessary conditions for success include: a sound regulatory framework, capable regulators, credible dispute resolution mechanisms, standard clauses on arbitration in concession contracts, and advice on all these matters to local governments involved. Brazil Water Sector Modernization Project (L3442) (1) Political support i s critical for agreement on sector policies. In spite o f a large number o f studies prepared under the project, o f which many o f high quality, a clear regulatory sector framework i s s t i l l absent in Brazil. The important question i s s t i l l not defined who has the power to concede water supply and wastewater services in urban areas. ((2) Sector investment loans should be demand-driven and allocated according to the performance of potential sub-borrowers in order to accelerate disbursements and provide incentives for reform at the local level. Brazil Minas Gerais water quality and pollution control (Loan 3554) 1. To facilitate creation o f a river basin agency, introduction o f water use and pollution fees, and drafting and enactment o f the necessary legal framework, projects should: (i) make budget provision (in the form o f seed money) for the agency, until i t can demonstrate i t s value, credibility, and capacity to collect water use and pollution fees; (ii) promote creation o f a transitional institution to take the lead in creating the agency; (iii)ensure that users and communities are well informed; and (iv) ensure that the legal covenants are realistic and that they do not affect the consensus building process adversely. 2. The Adaptable Program Lending approach i s highly suitable for introducing the comprehensive river basin approach. 3. Monitoring indicators defined at appraisal should capture incremental and partial achievements o f the project objectives. A baseline survey should be conducted to make it possible to compare the situation before and after project intervention. 4. Environmental education programs should be designed to be implemented before and during physical interventions, so as to ensure beneficiary communities’ ownership, support and commitment. Ensuring that other public and private agencies, neighborhood associations, business associations, are involved guarantees the continuity o f the program. 59 Annex J Lessons From FYOl Water Supply and Sanitation Project Cohort Africa Ghana Community Water & Sanitation (C2604) (a) The soundness o f demand-driven implementation o f community water supply and sanitation programs was confirmed under the project. This confirms the validity o f the model developed in Latin America and elsewhere in the 1960s and 1970s. (b) There i s the need to think o f potable water supply, sanitary excreta disposal, and hygiene education as a package, “ a three-legged stool”, on which the health improvements depend. Hygiene education o f school children needs to start as early as possible. Benin Rural Water Supply & Sanitation Project (C2622) (1) The private sector can respond dynamically to economic opportunities when the public sector i s phased out o f the preparation and implementation o f rural water supply and sanitation works. (2) The satisfactory outcome o f the program was possible because o f a relatively significant investment in capacity building, successful partnerships between national agencies and private f i r m s and the communities, and the support o f a coalition o f external assistance agencies, such as IDA. DANIDA, and UNICEF. Nigeria State Water I (C23 72) (1) Without administrative autonomy for water supply and wastewater agencies, it i s impossible to reach commercial autonomy; (2) Demand was overestimated and the dropping utilization o f the installed water production capacity can most o f all be explained by the insufficient investments in distribution works, coupled with insufficient possibilities for the populations to connect. The same experience with overestimated demand has been derived from projects in all regions since the Bank started lending for water supply and sewerage investments; (3) Many o f the urban towns and cities in Africa have more in common with rural villages and they could best be served through a “small towns” approach with greater reliance on community participation and participation, and health education campaigns than i s usually the case in the Bank’s work in urban areas. South Asia India Karnataka Water Supply and Environment Project (C2483) 1. A sustained improvement in environmental health requires three ingredients: safe, potable water; sanitary excreta disposal; and health education; 2. Sustainable water supply and sanitation systems can best be assured by delegating the administrationto the lowest, appropriate level which i s to the village level through the establishment o f village-based committees. 3. Financial sustainability requires user fees to be levied to pay for the costs o f operations and maintenance. If, in addition, users are obliged to contribute something-in cash or in kind-towards the construction o f the system they can be expected to care more for efficiency o f investment and for the continued operations and upkeep o f the facilities. Annex J 60 Europe and Central Asia Albania Durres Water Supply (C2610, CP749) (1) The project experience confirms that demand projections in restructuring formerly planned economies, need to anticipate dropping demand, and especially dropping industrial demand; (2) The lack of sector analysis preceding project preparationproduced a failure to identify key weaknesses of the sector and failure to design a project that could have addressed them; and (3) The planned utility improvements proved impossible to achieve for lack o f institutional autonomy that in turn was impossible to attain in the face o f complete lack o f financial autonomy. Uzbekistan Pilot Water Supply (L4090) The PCN suggests that a Project Preparation Facility or other instrument with lower transaction costs for the Bank would have been equally effective in facilitating the preparation of the full- scale project. Boznia-Herzegovina Republika Srpska Reconstruction Assistance Project (C3028; CQ055) 0 Initial assistance to war-affected farmers should not be on a strictly commercial basis, since market conditions and production support may not exist in a post-conflict situation. The poorest among them cannot be expected to contribute to full cost recovery. 0 Despite their political sensitivity, project attention should focus more upon occupancy and tenure issues in the housing sector in a post-conflict situation. Institutional capacity can be strengthened through on-the-job training o f local counterparts in the management and implementationo f a project. 0 Important assistance-especially concerning procurement matters-can be best provided through specialists based in Bank Resident Missions. Management of broad-based procurement though ICB and LCB can be too time- consuming in an emergency situation.