Document of The World Bank ReportNo.: 47950 PROJECTPERFORMANCE ASSESSMENTREPORT INDONESIA DAMSAFETY PROJECT (CPL-37420; SCL-3742A; SCPD-3742s) April 1,2009 Sector Evaluation Division Independent Evaluation Group (World Bank) Currency Equivalents (annual averages) Currency Unit =Indonesian Rupiah 1994 US$l.OO Rp2144 2001 US$l.OO Rp 11350 1995 US$l.OO Rp2220 2002 US$l.OO Rp 8750 1996 US$l.OO Rp2330 2003 US$l.OO Rp 8300 1997 US$l.OO Rp2398 2004 US$l.OO Rp 8800 1998 US$l.OO Rp 11700 2005 US$l.OO Rp9400 1999 US$l.OO Rp 7800 2006 US$l.OO Rp9200 2000 US$l.OO Rp 8236 2007 US$l.OO Rp9800 Abbreviations and Acronyms BalaiPSDA BalaiPengelolaan Sumber Daya Air or Provincial Water Resources River Basin Management Unit Balai WS BalaiWilayah Sungai or Central River Territory Organizations BDSF Basic Dam Safety Facilities CDMU Central DamMonitoringUnit DGWR Directorate General o f Water Resources (former DGWRD) DGWRD Directorate General o f Water Resources Development DMU DamMonitoringUnit DOISP DamOperation andImprovement o f Safety Project DSC Dam Safety Commission DSO Dam Safety Organization DSP Dam Safety Project DSU Dam Safety Unit EAP Emergency Action Plan FY FinancialYear GO1 Government o f Indonesia ICOLD International Commission on Large Dams ICR Implementation Completion Report IEG IndependentEvaluationGroup MPW Ministry o f Public Works O&M Operation and Maintenance OMS Operation, Maintenance and Surveillance PJT Perum Jasa Tirta or RiverBasin Organization PLN State Electricity Corporation PMF Probable Maximum Flood PMP Probable Maximum Precipitation PPAR Project Performance Assessment Report PWRS Provincial Water Resources Services SAR Staff Appraisal Report WISMP Water Resources and Irrigation Sector Management Project Fiscal Year April 1-March 31 Director-General, IndependentEvaluation : Mr.Vinod Thomas Director, IndependentEvaluation Group (World Bank) : Ms.Cheryl Gray Manager, Sector Evaluation Division : Ms.MonikaHuppi Task Manager : Mr.Ramachandra Jammi i IEGWB Mission: Enhancingdevelopment effectiveness through excellence and independence in evaluation. About this Report The IndependentEvaluationGroupassessesthe programs and activitiesof the World Bankfor two purposes: first, to ensure the integrityof the Bank'sself-evaluationprocessand to verifythat the Bank'swork is producingthe expectedresults,and second, to helpdevelopimproveddirections,policies,and proceduresthrough the disseminationof lessonsdrawn from experience.As part of this work, IEGWB annuallyassessesabout 25 percent of the Bank`s lendingoperationsthroughfield work. Inselectingoperationsfor assessment,preferenceis givento those that are innovative,large,or complex; those that are relevantto upcoming studies or countryevaluations;those for which ExecutiveDirectorsor Bank management have requestedassessments;and those that are likelyto generate importantlessons. To preparea Project PerformanceAssessment Report (PPAR), IEGWBstaff examine projectfiles and other documents, interviewoperationalstaff, visit the borrowing country to discuss the operationwith the government, and other in-countrystakeholders,and interview Bank staff and other donor agency staff both at headquartersand in localoffices as appropriate. Each PPAR is subject to internal IEGWB peer review, Panel review,and managementapproval. Once cleared internally,the PPAR is commented on by the responsibleBank department. IEGWB incorporatesthe comments as relevant.The completedPPAR is then sent to the borrowerfor review;the borrowers'comments are attached to the document that is sent to the Bank's Board of ExecutiveDirectors.After an assessmentreport has been sent to the Board, it is disclosedto the public. About the IEGWB RatingSystem IEGWB's use of multipleevaluationmethods offers both rigor and a necessary level of flexibility to adapt to lending instrument,projectdesign, or sectoral approach. IEGWB evaluators all apply the same basic methodto arrive at their project ratings. Followingis the definitionand rating scale used for each evaluation criterion (additionalinformationis availableon the IEGWBwebsite: http://worldbank.org/ieg). Outcome: The extent to which the operation's major relevantobjectiveswere achieved,or are expectedto be achieved, efficiently.The rating has three dimensions:relevance, efficacy,and efficiency. Relevance includes relevance of objectivesand relevanceof design. Relevanceof objectives is the extent to which the project's objectivesare consistentwith the country's current development priorities and with current Bank country and sectoral assistancestrategies and corporate goals (expressedin Poverty ReductionStrategy Papers, Country Assistance Strategies,Sector Strategy Papers,Operational Policies). Relevanceof design is the extent to which the project'sdesign is consistentwith the stated objectives.Efficacy is the extent to which the project`s objectives were achieved,or are expectedto be achieved, taking into account their relative importance.Efficiency is the extent to which the project achieved, or is expectedto achieve, a return higher than the opportunity cost of capital and benefitsat least cost comparedto alternatives.The efficiencydimension generally is not applied to adjustment operations. Possible ratings for Outcome: Highly Satisfactory,Satisfactory,ModeratelySatisfactory, Moderately Unsatisfactory,Unsatisfactory,Highly Unsatisfactory. Risk to DevelopmentOutcome: The risk, at the time of evaluation, that developmentoutcomes (or expected outcomes) will not be maintained(or realized). Possibleratings for Risk to Development Outcome: High Significant,Moderate, Negligibleto Low, Not Evaluable. Bank Performance: The extent to which services provided by the Bank ensured quality at entry of the operationand supportedeffective implementationthrough appropriatesupervision (includingensuring adequate transition arrangementsfor regular operationof supported activitiesafter loadcredit closing, toward the achievement of development outcomes.The rating has two dimensions:quality at entry and quality of supervision. Possible ratings for Bank Performance: Highly Satisfactory,Satisfactory,ModeratelySatisfactory, Moderately Unsatisfactory,Unsatisfactory,Highly Unsatisfactory. BorrowerPerformance: The extent to which the borrower (includingthe governmentand implementing agency or agencies) ensured quality of preparationand implementation,and compliedwith covenants and agreements, toward the achievementof developmentoutcomes. The rating has two dimensions:government performanceand implementingagency(ies) performance.Possibleratings for Borrower Performance: Highly Satisfactory,Satisfactory,ModeratelySatisfactory, Moderately Unsatisfactory,Unsatisfactory,Highly Unsatisfactory. ... 111 Contents PRINCIPAL RATINGS ..................................................................................................................... V KEY STAFF RESPONSIBLE .......................................................................................................... V PREFACE ...................................................................................................................................... vi1 SUMMARY ...................................................................................................................................... IX BACKGROUND ............................................................................................................................... 1 THE PROJECT ................................................................................................................................ 3 Objectives .......................................................................................................................... 3 Design and Quality at Entry ............................................................................................. 3 Implementation .................................................................................................................. 6 Monitoring and Evaluation ............................................................................................... 6 PROJECT EVALUATION ................................................................................................................ 7 Relevance .......................................................................................................................... 7 Efficacy .............................................................................................................................. 8 Efficiency ......................................................................................................................... 12 Outcome ........................................................................................................................... 12 Risk to Development Outcomes .................................................................................... 13 Safeguard Compliance ................................................................................................... 14 Strengthening Country Systems for Safety of Dams .................................................. 14 Bank Performance .......................................................................................................... 14 Borrower Performance ................................................................................................... 15 LESSONS ...................................................................................................................................... 16 ANNEX A .BASIC DATA SHEET .................................................................................................. 19 ANNEX B OF DAMS SITES VISITED AND ORGANlZATlONSlDEPARTMENTS CONSULTED .LIST................................................................................................................................. 23 ANNEX C .DAM SAFETY INSTITUTIONSAND DATA REPORTING RELATIONSHIPS ...........25 ANNEX D BORROWER COMMENTS . ........................................................................................ 27 This report was prepared byRamachandraJammi(Task Manager). who assessedthe project inJanuary 2008. Soon-Won Pak urovided administrative suuuort. V ICR* ICR Review* PPAR Outcome Satisfactory Moderately Moderately Unsatisfactory Unsatisfactory Institutional Modest Modest Development Impact** Risk to Moderate Development Outcome Sustainability*** Likely Unlikely Bank Satisfactory Satisfactory Moderately Satisfactory Performance Borrower Satisfactory Satisfactory Moderately Satisfactory Performance * The ImplementationCompletion Report (ICR) is a self-evaluation by the responsible Bank department.The ICR Review is an intermediate IEGWB productthat seeks to independently verify the findings of the ICR. **ASof July 1,2006, Institutional Development Impact is assessed as part of the Outcome rating. ***ASof July 1,2006, Sustainability has been replaced by Risk to Development Outcome. As the scales are different, the ratings are not directly comparable. Project Task ManagedLeader Division Chief/ Country Director Sector Director Appraisal H. V. Voothuizen Anthony Cole Marianne Haug Completion llham Abla Mark D.Wilson Andrew Steer vii Preface This is a Project Performance Assessment Report (PPAR) for the Indonesia: DamSafety Project. The Dam Safety Project was approvedon M a y 31, 1994, for a Loan o fUS$55 million (CPL-37420; SCL-3742A; SCPD-3742s). Following slow implementationand the economic crisis o f 1998-2000, two loan cancellations were made -US$8.0 million in August 1998, and US$11.8 million inDecember 1999. By the end o f the project, US$35.2 million (64% o f the original loan) was disbursed. After three extensions to allow greater progress inboththe institutional and physical works components, the project closed on March 31,2003, two-and-a-half years later thanplanned. The report presents the findings o f (i)an IEGmission to Indonesia during January 21-31, 2008, including visits to 9 project-assisted dam sites and discussions with Government officials and agencies, project directors and staff, beneficiaries, NGOs, and academia; (ii) discussion with Bank task managers and other staff inWashington and Jakarta; and (iii) review o fthe project's implementation completion report, appraisal report, loan agreement, and other relevant material. The cooperation and courtesies extended to the IEGmissionby all is gratehlly acknowledged. The project was selected because it was one of the first two projects (along with the India DamSafety Project) to focus exclusively onDam Safety, which is also the object ofthe Bank's OP/BP 4.37 on "Safety o f Dams" and one of the ten safeguard policies o f the Bank. It also provides input to the ongoing IEGStudy on the Bank's support for Water Resources Management. Following standard IEGprocedures, the draft PPAR was sent to the borrower for comments before beingfinalized. Inaccordance with the Bank's disclosure policy, the final report will be available to the public following submissionto the World Bank's Board o f Directors. i x Summary This Project PerformanceAssessment Report reviews the IndonesiaDamSafety Project, which was approvedin 1994 andclosed in2003. This was the first Bank project in Indonesia that was devoted entirely to dam safety and only the second worldwide after the IndiaDam Safety Project (1991-2000). Prior to the project, there had been long-standing concerns on the part o f the Government andother stakeholders about sustaining the existing damstructures inthe country and increasingthe efficiency o fwater use, while makingprovisions to protect society against potential disasters due to dam failures. At that time, there were no mandatory government regulations or procedures that required on-going monitoring, analysis and evaluation o fthe safety o f dams. The development objective - reduce the risk of damfailures in Indonesia -was challenging because the relevant Government ministries and agencies lacked the institutional, technical, and financial capacity to deal with the immediate as well as long term needs for improving dam safety. Project implementation was delayed at the beginningdue to a late start o f training and capacity buildingactivities, and later due to the economic crisis during 1998-2000, as well as organizational changes. The project succeeded inestablishing abasic framework for dam safety -where none existedbefore - at the central andprovincial levels. This was sustained, but with several shortcomings incapacity, staff and budgetresources. The mainregulatory entities at the center -the Dam Safety Commission andthe Dam Safety Unit-need to strengthen their independence andtechnical capacity. They need to emerge as champions for mainstreaming dam safety concepts, inparticular through highlightingthe costs and benefits o fbetter dam safety. Remedial works were carried out at the major Jatiluhur dam, and for 72 other smaller dams andwere reportedto have produced broadly positive results. Basic dam safety facilities including dam monitoring instruments were installed as plannedbut their quality andmaintenance were not up to expectations, andthe equipment has deteriorated since project completion. Overall project Outcome i s ratedModerately Unsatisfactory. Taking into account recentpositive developments inrespect o f staffing andbudget resources, andprogress on over-arching regulation, Risk to Development Outcome i s ratedModerate. The project design appropriately balanced the immediate need for rehabilitation o f dams with the longterm needto create a viable institutional framework for dam safety. The Bank demonstrated a good understanding o f technical issues and supplemented it with international expertise increating a sound design for remedial works as well as for institutions and processes requiredto promote dam safety. However, the Bank A underestimated the time neededto buildcapacity from a relatively low base. During implementation, it could have done more to impress upon the borrower the need to speed up training and capacity-buildingactivities. Onthe whole, Bank performance is rated Moderately Satisfacto y. The Government showed steady commitment to the project by passing enabling legislation ahead o fthe project, andpersistingwith the project inspite o f the economic crisis. Five years after project completion, the basic institutional framework for dam safety has been maintained, though with shortcomings relating to staffing, capacity andbudgetaryresources. However, there have beenrecent positive developments inimproving staffing andbudget resources, and infinalizing over-arching regulation. The implementing agency performedwell interms o f carrying out remedial works, giventhe reduction inresources anduncertainties faced duringthe economic crisis. However, the implementingagency could have done more inspeeding up the training component and procurement o f consultants. Overall Borrower performance i s ratedModerately Satisfacto y. Keylessons from the project experience are: To sustain outcomes ina situation where new concepts andprocesses are introduced, the focus on institutional development needs to be maintained during the project andbeyond. The risk to dam safety inthis project remains moderate or substantiaVconsiderable due to relatively lower attentionto institutional development vis-a-vis physical activities duringimplementation and inadequate follow-up thereafter. New regulatory institutions require technical expertise and adequate operating budgets to function in a capable and independent manner. Inthis project, shortage of experiencedqualified staff andinadequate operatingbudgets contributed to under-performance o f the dam safety regulatoryunit. Buildingdam safety institutions can be a longprocess when it requiresmajor changes inorganizational attitudes, establishing new work methods, and upgrading technical expertise. Insuch cases, the Bank should consider an extended commitment through a sequence o fprojects or through adaptable program lending (APL). Whennew monitoring andreporting procedures are instituted,their purpose should be made clear to all those involved, and appropriate and regular feedback should be given to those originating the data. Inthis project, there was little systematic analysis or feedback from higher levels on dam level reporting, which progressively affectedquality and compliance. New techniques and equipment should be tailored to the existing level o f capacity; facilities and funds for ongoing maintenance; and site security. Most o f the relatively sophisticated instrumentationinstalled at dams under this project has deteriorated for lack of maintenance. xi Project benefits should be quantified to the extent possible inorder to provide clear economic justification for an investment and improve stakeholder support. The mainbenefit from this project i s the prevention o f losses from dam failure, which can be difficult to quantify. However, other benefits such as increased water supply are more amenable to being quantified and may well justify the investment. Vinod Thomas Director-General Evaluation 1 Background 1. Dams are structures for water storage and use inwater supply, irrigation, hydropower, and flood control. They play a crucial role inmajor sectors of economic activity and can contribute to poverty reduction. At the same time, dams can be associated with negative environmental and social impacts including those caused by natural disasters such as floods and earthquakes. Dam Safety concerns itselfwith the adequacy o f the operations andmaintenance o f dams, and aims to limit their possible adverse impacts on human life, health, property, and the environment, as well as plans for dealing with emergencies'. Ensuring dam safety can also improve the efficiency o fwater resource use, while keeping the dams environmentally and socially sustainable. 2. According to the World Commission on Dams2estimates, there are around 45,000 large dams3inthe world, with the largest numbers inChina (22,000), U.S.A (6575), India (4,291), Japan (2675) and Spain (1196). 3. The World Bankhasbeen involvedinfinancing dams andancillary facilities since the 1960s, though the rate o f involvement slowed sharply inthe mid-l980s, with the focus shiftingtowards ancillary facilities. Inrecent years, there has been greater emphasis on promoting dam rehabilitation and safety. At the beginning o f 2008, there were around 70 active projects that contained components relating to damrehabilitation or dam safety assessment. Ofthese, three projects4(in Sri Lanka, Armenia and Albania) are predominantly concerned with dam safety. 4. In1977, the Bankissuedits first formal policyon "Safety ofDams'" that underscored the importance o f dam safety arising from inadequate designor natural phenomena. After revisions that incorporatednew thinking on dam safety issues, the current version o f the Bank's policy was issued in October 2001 (Operational Policy [OP] 4.37, together with Bank Procedure [BPI 4.37). The standards set by this policy are at least as stringent as those recommended by the WCD. OP 4.37 i s also one o f the ten World Bank "Safeguard Policies" which require that potentially adverse environmental and social impacts o f Bank-financedprojects involved with dams be identified, mitigated, and monitored. 1.2002.Bradlow, D.D.,PalmieriA., Salman S.L.A., RegulatoryFrameworks for DamSafety. World Bank. 2. The World Commission on Dams (WCD) was established in 1988 with a mandate to review the development effectiveness o f large dams and develop internationally acceptable criteria, guidelines and standards for large dams. 3. Large dams are defined as those having a height o f 15 meters or more andor a reservoir capacity o f at least 500,000 cubic meters. 4. Sri Lanka: D a m Safety And Water Resources Planning Project (DSWRPP); Armenia: Irrigation D a m Safety 2 Project; Albania: A P L 5 For Albania D a m Safety 5. Operational Manual Statement (OMS) 3.80 (1977) 2 5. Indonesiai s a vast equatorial archipelago o f 17,000 islands extending 5,150 kilometers (3,200 miles) betweenthe Indian and Pacific Oceans. The largest islands are Sumatra, Java, Kalimantan, Sulawesi, and IrianJaya. The country has a per capita Gross National Income (Atlas method) o f US$1420, and a population o f around 220 million o f which 60% live inJava alone. Lyingnear the intersection o f shiftingtectonic plates, Indonesia i s prone to the effect of earthquakes and volcanic eruptions. 6. InIndonesia, damshavebeenconstructed since the early 1900s, initiallyfor irrigation purposes, but increasingly as multipurpose dams for irrigation, power generation, and flood control, as well as providingwater for major urban centers. Prior to 1965, all dams inIndonesiawere planned, constructed and operated by the Ministryo f Public Works (MPW). Subsequently, the hydropower facilities were separated from MPW andplacedunder the State Electricity Corporation (PLN). In1994, there were about 120 large dams inIndonesia. By 2008, there were 240 dams inthe country, 132 o f which were "large dams", o fwhich 106 were owned by MPW, 18 by P L N and 8 by miningcorporations. Together they serve about one millionhectares o firrigatedlands, and generate 6000 GwH (billion watt-hours) o felectricity. The majority o fthe dams are located inJava, which contains the largest proportion o f arable landinIndonesia. 7. The IndonesiaDam Safety Project -the project reviewed inthis PPAR-was formulated inresponse to long-standing concerns on the part o f the Government about sustaining the existing dam structures and increasingthe efficiency o fwater use, while makingprovisions to protect society against potential disasters due to dam failures. In particular, at the time o fproject preparation, it was estimated that the condition o f more than 95 large dams owned by the MPW was declining andposed a public safety hazard largelybecause o finsufficient maintenance. 8. At the time o fproject preparation, there were no mandatorygovernment regulations or procedures that requiredon-going monitoring, analysis and evaluation o f the safety o f dams. The core activities o f the Directorate General o f Water Resources and Development (DGWRD6)- the main agency under M P W with the responsibility for managing dams -were focused more on irrigation infrastructure. DGWRDhadmuch lower priority for operational management o f the dams. More generally, there was an unevenunderstanding o fthe economic, environmental andsocial dimensions o f dam operation. Therefore, the project plannedto establish a national regulatory institution for large dam safety assurance under MPW as well as for undertaking remedial works for the 95 large dams owned by MPW and operated by DGWRDor PJTs (River Basin organizations7). 9. The IndonesiaDam Safety Project (DSP) benefited from the experience o f the IndiaDam Safety Project (1991-2000), which was the first Bank project focused exclusively on dam safety. At the end o f FY2008, there was no active Bankproject with damcomponents inIndonesia. However, a Dam Operation and Improvement o f Safety Project (DOISP) i s under preparation, and will essentially be a follow-up to the DSP. 6. Currently the Directorate General for Water Resources (DGWR) 7. Citarum and Brantas River Basins 3 The ongoing WISMP (Water Resources and Irrigation Sector Management Program) includes a component to ensure continuity for some dam safety related activities inthe interveningperiodbetween the DSP and DOISP. The Project OBJECTIVES 10. The Dam Safety Project's development objective was to reduce the risk of dam failures in Indonesia through the followingproject components, o f which the first three were mainly institutional innature, while the last component related to physical works. a) Establishmento f dam safety institutions; b) Periodic inspections and safety evaluations o f all large dams; c) Provision o f basic safety facilities at existing dams where suchprovisions are now lacking; and d) Implementation o fremedial works at dams with safety deficiencies. 11. Duringproject implementation, two additional institutional components were added: e) A "Participatory Dam Safety Program" or "Model Dam" pilot to raise local public awareness and community participation for dam safety; and f ) The Prototype Basin Management Program to strengthen OMS (operation, maintenance and surveillance) inthree selected river basins The two additional components were added through amendments to the LA. The project components are listed inTable 1along with the planned and actual costs. DESIGNAND QUALITY AT ENTRY 12. The development objective-reduce the risk of damfailures in Indonesia -was challenging for several reasons. First, the majority o f dams inthe country were inneed o frehabilitation to different extents. Second, the institutional, technical and financial capacity for dealing with dam safety issues was weak at all levels o f the Government. Third, the ministryresponsible for dam safety didnot have enough separation betweenits "operator" and "regulator" roles as needed for effective management o f dam safety. Finally, dam safety was viewed by practitioners ina narrow technical manner. Less attentionwas paid to the larger economic, environmental and social costs andbenefits o f securing the safety of dams. 13. Inthis context the project was designed to balance the immediateneed for rehabilitation o f dams with the longer term needto create a viable institutional framework, buildcapacity, and to help all stakeholders understand the wider benefits o f securing dam safety. 4 Table 1.ProjectObjective,Components.PlannedandActual Costs OVERALL OBJECTIVE: Reduce the risk of dam failures in Indonesia Components Sub-components Costs (US$ million) Planned Actual - % (a) establishingand Establisha Central Dam Unit in DirectorateGeneral of strengtheningdam Water Resourcesand Development(DGWRD); safety institutions; EstablishDam MonitoringUnits (DMUs) in 8 provincial water resourcesagencies; Establish DMUs in 3 MPW- managedproject agencies (JatiluhurAuthority; Brantas River Basin Corporationand the Bengawan- Solo Project) 3.8 3.2 84 Institutionalcapacity building and operational support to the Dam Safety Commission(DSC), Dam Safety Units (DSU),Central Dam Monitoring Unit (CDMU) and DMUs. Provisionof TA (training,seminars,workshops)for DSC and MPW staff. (b) periodic TA for carryingout remedialworks for 93 MPW dams inspections and where detailed inspectionshad identifiedthe needfor safety evaluationsof such works and for upgradingtheir safety condition included in (9) all large dams; (includingthe Jatiluhur Damwith separate TA for supervisionsupport). (c)provision of basic Facilities,services and relatedTA for: carrying out safety facilities at dam surveys and site clearingto obtain informationfor existing dams where detailed safety inspections; such provisionsare nowlacking; and Provisionof communicationsequipment and facilities: 7.9 Provisionof instrumentationto record data relevantto the structuralconditionand operational safety of dam structures;and carrying out emergency actionsto ensure the safety of a dam. (d) implement Remedialworks and supervisionfor Jatiluhur Dam (30 remedial works at percentof the total component base cost) and 92 small dams with safety dams. 65.0 deficiencies. 41.1 I63 (e) "ModelDam" Incorporatelocal public awareness on dam safety and community participationin dam maintenancefor four pilot dams (Dawuhan, Cenglik, Sermo and Dharma in East, includedin (9) Central,Yogyakarta and West Java, respectively) (f) PrototypeBasin Strengtheningoperation,maintenanceand surveillance Management (OMS) proceduresto promotesustainabledam safety includedin (9) Program programsfor provincial DMUs in three selected basins. (9) Project This component providedfunds to cover: (i) the Management establishment and operation of a central Project Administrative ManagementUnit (PMU) with the necessaryfacilities and Management equipment; and (ii) supporting TA (4.7 million)for Financial institutionalcapacity buildingassistanceto the DSC, DSU, 5.0 5.9 120 Management and the DMUs, and technical reviews of activities under Participatory the Basic Dam Safety Facilitiesand RemedialWorks program components - 14. A new institutional framework for dam safety was designedconsisting of a Dam Safety Commission (DSC), DamSafety Unit(DSU) anda Central Dam Monitoring Unit (CDMU) at the center; ProvincialDamMonitoring Units(PDMU) at the provincial level 5 with a DamMonitoringUnits(DMUs) ineachriver basinmanagedby a Provincial Water Resources Management Unit (Balai PSDA) or by the River Basin Management Office (Balai or Balai Besar Wilayah Sungai - BBWS). The proposed institutional structure i s given inAppendix C. 15. The DSC would be a standing committee o fdiverse experts reporting directlyto the Minister o fPublic Works, andensuring that all dam relatedworks conformedto regulatory guidelines. The D S Uwould be the technical agency for overseeing the conduct o f dam safety functions all over the country. The "operator" function would be performedby the Central DamMonitoringUnit (CDMU), which would organize and execute dam safety-related activities under the DSCDSU's regulatory oversight. Some separation was to be achieved between the "operator" and "regulator" by locating the D S U and the C D M Uunder different directorates o f MPW, though ideally, D S Uas the regulatory unit may have reported to an even higher level. DamMonitoringUnits (DMUs) were to be created in 8 provinces and intwo RiverBasin Organizations. 16. Interms ofcapacity-building, staffofMPW andits Directorate General ofWater Resources (DGWR: the implementing agency) would undergo classroom-style and on- the-job training on technical and managerial aspects o f dam safety. 17. The framework for remedial w o r h complied with the Bank's OP 4.37 (Safety o f Dams), which was commendable since this policy was formalized several years after project effectiveness. The project also provided for installing various basic dam safety facilities (BDSF) including dammonitoring equipment, and computer software for data analysis. 18. Two small components were addedduringthe extension period of the project in 2001. These were a Participatory Dam Safety Program or "Model Dam" pilot covering four dams to improve dam security and upkeepincooperation with surrounding communities', and a Prototype Basin Management Program to strengthenOMS (operation, maintenance and surveillance) and address the special needs o f three selected river basinsg. 19. Prior to project approval, the Government issuedan important enabling Ministerial Decree on Dam Safety (1994). Inorder to have the remedial works begin within six months o fproject effectiveness, preparatoryactions were taken including drafting o f tender documents and pre-qualification o f contractors. Incontrast, such early preparatory steps do not appear to have been taken for training and capacity building. 20. There were no major risks identified at the time o f project preparation with the significant exception o f the ability o fthe borrower and implementingagency to buildthe required capacity ina relatively short period o f two years before the physical works 8. The "Model Dam" pilot was created inresponse to a rise invandalism and looting inthe wake of the economic crisis that set induring 1998. The four participating dams were: Dawuhan, Cenglik, Sermo and D a m dams inEast and Central Java, Yogyakarta, and West Java, respectively 9. Bengawan Solo inCentral and East Java; Citarum basin inWest Java; and Lombok island. 6 components picked up speed. This risk was underestimated, and might have been better recognized ifthe project preparation team had included an institutional expert with an understandingofthe Indonesiancontext. IMPLEMENTATION 21. The project was planned for implementation insix years during 1994-2000. However implementation was slow due to several reasons: (i) there were delays initially on the part o fthe Government infinalizing the training needs assessment, and later in appointing consultants for technical assistance (TA) activities; (ii) economic crisis o f the 1998-2000greatly reducedthe Government's ability to provide counterpart funds, and generally created an atmosphere o f uncertainty; and (iii) there were major organizational changes after a new decentralizationpolicy was implemented in 1999, which particularly impacted the new institutions, the DSUand the Central Dam MonitoringUnit,through relocation and reductions instaff. 22. Four years into the project, remedialworks hadbeen completed for about 30 out of the targeted 93 dams, leading the Bank and borrower to agree upon canceling remedial works for 25 low priority dams. Institutional development and capacity buildingalso continued to be slow. To allow for greater progress inthe physical works and institutional components, the project was extended three times and the project closed on March 31,2003, two and a halfyears later thanplanned. At this time, remedial works were completed for Jatiluhur and 72 other smaller dams; the physical targets for basic dam safety facilities were substantially met; and institutional development and training goals were nominally reached. The Prototype Basin Management Program (Citarum:Jatiluhur; Bengawan Solo; and Nusa Tenggara Barat: Lombok) was designedto address good dam management & safety practices (see para 1S), but activity was negligible duringthe two years that the program was operative due to low budgets and staffing constraints. 23. The total project cost at appraisal was estimated at US$97.4 million, o fwhich US$55.0 million was to be financed bythe IBRD's loan andUS$42.4 by the Government. Duringthe economic crisis o f 1998-2000, the U S dollar appreciated more than four-fold relative to the IndonesianRupiah, and the Government was not able to keep up its counterpart funding. As a result, portions o f the loan were cancelled- US$S.O million inAugust 1998, andUS$11.8 millioninDecember 1999. By the end o f the project, US$35.2 million (63% o fthe original loan) hadbeendisbursed. MONITORINGANDEVALUATION 24. Design. The Staff Appraisal Report (SAR) viewedmonitoring and evaluation in terms o f outputs tracked by implementation schedules for all activities under the project. The DSUhadthe responsibility for summarizing the progress against these implementation schedules. Itwas also expected to evaluate the effectiveness and impact o f the dam safety program, but no specific indicators were provided. It is recognizedthat 7 the Bank's operational guidelines at the time o fproject preparation (1994) did not require a logframe to be prepared. There was thus no M&E framework showing a causal chain from activities to outputs to outcomes. 25. Implementation. The DSUprepared progress reportsregularly. The endresults o f implementation interms o f outputs from all activities are comprehensively reported in the project implementation completion report. Mandatory technical audits were carried out by the MPW's Inspectorate General which was a financial rather than a technical entity and therefore didnot have the expertise to assess the quality and impact from the outputs. 26. Utilization. The feedback from the periodic DSUreportshelpedto make decisions on revising targets duringthe project. The findings from the technical audits carried out by the Inspectorate General were o f limiteduse intechnical terms for assessingthe outcome andimpacts from the project. Overall, M&Ei s rated modest. Project Evaluation RELEVANCE 27. The DSP was highly relevant for addressing the long-standing concerns expressed bythe Government about the state o f existing dam structures inthe country andthe efficiency o fwater use, as well as the inadequate state o f institutional, regulatory, technical and financial capacity to deal with these issues. There was a clear need to broadenthe focus o f the M P W beyond irrigation infrastructure, andto view the importance o f dam safety in a larger framework o f economic, environmental and social costs and benefits. 28. The project development objective reflected the concerns o fmost o f the Country Assistance Strategies duringthe 1994-2004 period. These concerns included strengthening public sector management to protect against risk o f dam failures; balancing central andprovincial responsibilities for development o fpolicy and regulatory systems andstandards; more integratedwater resources management on a river basinscale; and integrating environmental concerns into water resource management. The relevance o f objectives i s rated high. 29. Onthe whole, the project design was based on a sounddiagnosis o fthe institutional and rehabilitation needs for dam safety inthe country and reflected Governmentcommitment. The project designwas based on a good understandingo f technical issues and supplemented with international expertise increating a sound framework for remedial works as well as for institutions andprocesses requiredto promote dam safety. Its significant drawback lay inoverestimating the ability o fthe borrower to buildthe required institutional capacity ina relatively short period o ftwo years before the physical works components picked up speed. Relevance o f design is ratedsubstantial, and overall relevance is ratedsubstantial. 8 EFFICACY 30. Institutional development under the project was served by the first three componentso f the project viz. establishment o f dam safety institutions; periodic inspections and safety evaluations o f all large dams; provision o fbasic safety facilities; andthe additional component on the Participatory DamSafety Program. Establishing and strengthening dam safety institutions 3 1. Basicinstitutionalframework.Thebasic institutional framework for dam safety was put inplace largely as planned. The DamSafety Commission was constituted"; and the D S Uandthe Central DamMonitoringUnitwere created under different directorates" o f the DGWR. The status o fthese units, nearly five years after project completion, i s described briefly below: DamSafetyCommission. The DSC meets periodically to make recommendations directly to the Minister o f Public Works on adheringto safety guidelines incarrying out new and existing dam works, based on DSUadvice. DamSafetyUnit. FollowingDSP closure, the DSUsuffered from staff attrition to the extent that, except for review o f new dam designs, it became increasingly unable to carry out its regulatory function o f dam safety inspections (paras 35 & 49). The D S Upresentlyconsists o fthree sections12each headed by a qualified civil engineer and collectively supported by 4 junior technical staff and 13 part- timeretiredqualifiedexperienced engineers. Two o fthe originally planned sections for training and data no longer exist and their functions are notionally distributed acrossthe other units. CentralDamMonitoringUnit.The CDMUceasedto exist after the project and its functions are since beinghandledby the Sub-Directorate for Operations & Maintenance (O&M) which has a broader mandate. This reduces the focus on O&M needs for dam safety. ProvincialDamMonitoringUnits. Provincial DamMonitoringUnits(PDMUs) were created ineight provinces with distinct technical andadministrative sections. By2007, all largeriver basins on Java were redefined as "national River Basin Territories" reporting to the central government. Thus the large dams inthese basins were placedunder new nationally managed and financed River Territory 10. The DSC is chaired by the Director General o f Water Resources (DGWR) with the Head o fthe D a m Safety Unit(DSU) as the Secretary and members consisting ofrepresentatives o f the Ministries o f Home Affairs (Civil Defense Agency), Miningand Energy, Industry, and Environment; representatives o f professional engineering associations, the ResearchInstitute o f Water Resources Development, legal advisor, and the law enforcement agency. 11. The DSU was placedunder the Directorate o fRivers, Lakes andReservoirs and CDMUunder the Directorate o f Water Resources Management. 12.The existing sections are: (i) Program and Evaluation, (ii)a mMonitoring and Inspection, and (iii) D Review, Data and Information. (see Annex C). 9 Management Units (Balai Wilayah Sungai or Balai WS: see para 33). Thus, the role o f the provincial PDMUand River Territory Management Units became unclear. Over the years, the PDMUs have ceasedto be structural units and their functions have beennotionally distributedamong other staff, leading to a sharp deterioration o f capacity for dam safety issues at the provincial level. Two other DMUsthat were created intwo PJTs or RiverBasinCorporations-for O&M of MPW's large infrastructure inthe Citarum andBrantas basin- are still functioning as intended. Data collection, reportingand analysis. A standardized dam level data reporting fonnatl3 was introducedby the project, andi s broadly followed in practice, subject to the state o fmonitoring andrecording instruments. Generally, each dam reports to the provincial DamMonitoringUnit every 7 to 10 days. After takingnote o fdata that is immediately relevant for dam operations, the provincial DamMonitoringUnits submit consolidated reports to the D S Uand the Sub-Directorate (O&M) at the center (see Annex C). Butpractically no further analysis i s done at any level for longer term prioritization or planning, due to lack o f capacity. Dammonitoring software (DAMOSY) introduced during the project has not been usedsince andi s now outdated. There has beenno significant follow-up on compiling the national inventory o f dams andtechnical drawings/documents, which was started duringthe project. 32. The dam safety function at the provincial level is affectedby an overlap o f responsibilities betweenprovincial and central river basin units14(Balai PSDA andBalai Wilayah Sungai or WS respectively). This has implications for efficiency o f reporting - individual dams inthe river basins have to send reports to both Balais; and budget responsibilities -Balai PSDA provides for staff and inspections, but major O&M may or may not come from Balai WS. One possible solution is to have day-to-day activities under Balai PSDA control, while the larger andmore complex works are assigned to Balai WS. MPW officials reportedthat these issues are being sorted out inthe larger context o fwater resource management. 33. Training and Capacity-Building: Training beganinearnest only inthe year 2000 after the delayedpreparation o f a training needs assessment, and gained focus and scale only by 2001-2002. Ultimately, about 41 training courses covering 1,535 officials were heldwithin the country, and 25 officials attended courses/technical tours abroad. D S U staffreported that the impact o f the training has dissipated over time as trained staff have retired or rotatedto other positions. The DSUdoes not have the expertise or budgetary resources to carry out its training function to any extent. New staff have to learn on the job andthe units depend on the assistance o f retired or outside part-time experts for continuity. 13. The data covers climatological, hydrological and structuralparameters 14.Following the decentralizationpolicy adopted in 1999, central Government Balai WS were established to co-manage substantially the same issues as Balsi PSDA inrespect o f large dams inimportant river basins. 10 Periodic Inspections and Safety Evaluations 34. The D S Uperformed quality assurance functions duringthe project under the guidance o f international consultants for the Jatiluhur Dam and to a lesser extent for the other project-assisted dams. Presently it is far behindinperforming its crucial hnction o fmandatory five-yearly inspections o f all large dams to certify their safety or to establish rehabilitation needs. Against a yearly target o f at least 30 per year, DSUhas performed only 33 such inspections duringthe five years since project completion, including 7 emergency inspections inthe wake o f an earthquake and 8 following floods in2007. Thus, only 18 ofthese inspections havebeenscheduled andpreventativein nature. 35. A regulatory framework consistingo f23 guidelines for ensuringdam safety was formulated during the project. Five years after project completion 2 out o f 23 are still in the draft stage while 6 others need updating. All guidelines are generally available for use as technical references, but few o f them have been translated into Bahasa, limitingtheir utility,especiallyat the provincial level. The DSU's other important function o f certifying compliance with dam safety guidelines for all new dam construction and rehabilitation o f existing dams, i s occurring at a slow paceI5. Ingeneral, the DSUis constrained by a shortage o f staff, inadequate capacity and low budgets incarrying out its functions. Basic Safety Fa~ilities'~ 36. A variety o fmonitoring instruments were installedin74 out o f 88 targeteddams. Butinmost o fthe sites, the quality ofinstallation, supervision andoperationwas below expectations because o f inexperienced domestic contractors and consultants that carried out the work. This was exacerbated bypoor management support and control, inadequate understanding o f the purpose o fthe instruments, lack o f maintenance, and poor site security. Duringsite visits to 9 dams (see list inAnnex B), provincial officials at almost all sites reported instances o f instrumentation invarious stages o f disrepair. Sermo dam was a significant exception with most instrumentsreported to be inworking order. 37. Emergency Actions Plans (EAPs) - as a component o fbasic dam safety facilities - werepreparedduringtheprojectfor27damsonthebasisofDamBreakAnaly~es'~ No hrther EAPshave beenprepared since project completion. MPW officials reported that all dams have arrangements for communicating with the local government, civil 15. At project completion, the DSCDSUhad issued eighteen certifications for new dam designs (public and private), ten certifications for dam impounding, and four certificates for dam operation (including the Jatiluhur Dam). Since then, they have approved 6 new approvals for dam design, 5 approvals for filling and operation, and 18 technical reviews and certifications inprocess. 16. Basic D a mSafety Facilities are broadly defined as items that assist the operation and maintenance personnel at the dam to assess emergency situations and to respond with timely actions. These include Surveys (key drawings, reservoir capacity surveys); Communications (primaryand secondary communication links for operational and flood warning purposes); Instrumentation (Seepage: V-notch weirs; seepage flow recorders; reservoir level gauges and recorders; raingauges); Site preparation (clearance of dam and surrounding area); Emergency actions (Emergency actions plans). 17.An analysis that provides an estimation o f downstream flooding effects following dam failure. 11 defense and law & order officials incase o f an emergency. But these arrangements lack the comprehensive and analysis-based nature o f EAPs. 38. There is a shortage o f qualified andexperienced permanent staff for on-site conduct ofbasic dam safety facilities and overall management o f dam sites. InJatiluhur Dam, for instance, the 6 permanent technical staff depend on a similar number o fpart- time retired engineers to assist intheir work The smaller dams have one or two staff assigned on a full-time basis but better qualifiedtrained persons are needed for this task. Participatoiy Dam Safety Program or "Model Dam"pilot 39. An assessment o fthe "Model Dam" pilots infour dam sites by a local university- based team18does not yield any conclusive evidence o fpositive impacts on the dams and the local communities. However, anecdotal evidence from dam staff, beneficiaries and a local NGO19point to some favorable impacts on damupkeep and security, and on beneficiary incomes through agricultural, recreational and micro-business activities. Discussionswith university researchers, provincial officials and the local NGOs indicate that this program may require greater technical support, better coordination between Government agencies, and a better incentive framework for beneficiaries. 40. Based on the preceding discussion, each o f the components relating to establishing and strengthening safety institutions, periodic inspections, safety evaluations, and the "Model Dam" displayed moderate shortcomings. The component for basic safety facilities showed major shortcomings. Taking these findings into account, the overall efficacy o f the institutional development objective served by these components i s rated Modest. Remedial Worksand Supervisionfor Jatiluhur and other Dams 41. The remedial works carried out for the Jatiluhur dam2' (which accounted for 30% o f the cost for this component) were fully certified for operation in2003 by the DSU/DSC on the recommendation o f an independent panel o f international experts. According to M P W and dam officials, subsequent monitoring indicates that the gains from the project have beenlargely sustained. 42. The remedialworks that were carried out for 72 smaller dams were not subjected to the same level o f quality assurance as the Jatiluhur dam. The project ICR reports that the quality o f remedialworks could be considered `satisfactory' in90% o f these dams. Butthis assessmentis notbased on formal certification o fthe quality o fremedialworks, partlybecause the DSUdidnot have sufficient resources, and also because the agency that conducted periodic technical audits (Inspectorate-General o fMPW) was mainly gearedto perform financial audits. At the 9 dams that were covered insite visits, 18. Fakultas Pertania Universitas Sebelas Maret. 19. Mulyarejo Foundation 20. The remedial works at Jatiluhur repairs to the main dam structure, protective works to the Pasir Gombong West and Ubrug dikes, and a study o f the movement behavior and stability of its "Morning Glory" intake tower cum spillway. 12 provincial andMPW officials reported several positive results from remedial works including reduction in seepage and leaks, repaved dam crests, improved slopes of the downstream dam faces, and sediment removal. 43. The efficacy o fremedial works and supervision for dams is ratedSubstantial,but barely so, given the relatively less firm evidence on the results for dams other than Jatiluhur. Taken together with the Modest efficacy o f institutional development under the project, overall efficacy i s ratedModest. EFFICIENCY 44. The project was delayedby two and a half years due to reasons explained inthe section onImplementation. Finalproject costs appear commensurate with the revised targets for remedial works. The project S A R did not include any cost-benefit analysis. The project ICR notedthat the project's mainbenefits would be a reductioninthe risk o f damfailures, but evaluation was not done as "these would bedifficult to estimate." 45. At the same time there are anumber o fdirect andindirect benefits from a dam safety program that can possibly be estimated, such as water supply for domestic and industrial use, irrigation, power generation; fisheries andrecreation. For instance, the ICR notes that after rehabilitation o f the Jatiluhur dam, the water level inthe reservoir was first drawn to a minimumlevel (76 meters) duringthe 1997 dry season, which could not have been done earlier due to safety considerations. This helped ensure a continuous supply o firrigation water to 200,000 hao frice fields downstream from the dam, and the value o fthe crop that was saved is likely to have more than offset the cost of rehabilitation. 46. Taking into account the time overrun and the lack o f sufficient evidence for establishing benefits, the project's efficiency i s ratedModest. OUTCOME 47. The project documents do not specify any outcome indicators to assess the overall objective -to reduce the risk of damfailures in Indonesia. One approach to assessingthe risk o f dam failures inthe country i s to review the safety status o fthe dams as determined bythe latest inspections, but such information is incomplete or outdated. Inthe assessment o f the Dam Safety Commission (DSC) as conveyed orally to the IEGmission, most o f Indonesia's dams are operating nominally ina safe manner, though manyo f them need attention. Inthe absence o f any supporting evidence, the value o f this assessment i s limited. 48. Fiveyears after project completion, the dam safety institutions established bythe project (Dam Safety Commission and D S Uat the center; andthe provincial DMUs)have been sustained to a large extent. However, they have a longway to go infulfilling their intendedroles. The DamSafety Commission i s performing its regulatory role to a significant extent, but needs to do muchmore infostering awareness o f dam safety issues 13 within the Government and among other stakeholders. The DSUneeds to improve its credibility as a regulating agency that can provide unbiased and timely assessments o f the true status of dams; and guide and train provincial and dam level officials. The impact o f provincial DMUs has reduced sharply due to their marginalization interms o f staff and resources. The ability to monitor the health o f dams through new instrumentation has improved only marginally due to the poor quality of installation andmaintenance. While the remedialworks were completed substantially as plannedduringthe project, there has been relatively little remedial andrehabilitation activity since project completion despite pressingneeds. 49. Onthe basis o fSubstantial Relevance, Modest Efficacy, andModest Efficiency, the Overall Outcome for the project is ratedModerately Unsatisfactory. R I S K TO DEVELOPMENT OUTCOMES 50. At project completion in2003, Indonesiastill experienced the lingeringeffects o f the economic crisis. It appeared likely at the time that the gains made by the project mightbeeroded due to the Government's inability to provide sufficient resources for dam safety hnctions. A follow-up Bank project was contemplated, but inthe interim,the Bankhas included abridging component inthe Water Resources and Irrigation Sector Management Project (WISMP) for financing dam related O&M. While there i s general awareness of the importance o f dam safety at different levels o f the Government, in practice, dam safety issues get lower priority incomparison with irrigation and other water resource management issues. This i s reflected inthe relative lack o f focus on O&M for dam safety at the center and deterioration incapacity at the provincial level, as DamMonitoringUnitshavebeentransfonned from structural to functional units with few resources. 51. Five years after project completion, the basic institutional framework for dam safety i s broadly inplace though it i s functioning far below its potential due to shortage of staff, capacity andbudget resources. More recently, the Government has abolished its zero-growth policy on civil service staffing which was inplace for the last two decades, and has recruitedhundreds o f new engineers. DGWRhas increased budgetsboth for dam safety and for staff working on dam structural issues. The budgetfor 2007 has grown to about two-thirds o f the requiredlevel (para. 49), from less than 20% during 2000-2004. DGWRhas finally prepared the pivotal Government Regulation on Dam Safety, which is expected to be issued inearly 2009. Though the role o f the PDMUshas beenpartly taken over by the newly established River Basin Management Unitso f the central government, the monitoring capability at the provincial and local level needs to be re-established. While the current situation i s inmany respects similar to that which prevailed prior to preparation o f the current project (see para 7), taking into account recent positive developments inrespect o f staffing andbudget resources, andprogress on over-arching regulation, Risk to Development Outcome i s ratedModerate. 4 14 SAFEGUARD COMPLIANCE 52. The project was appropriately classified inCategory B for compliance with the Bank's safeguard polices, mainly to cover the possibility o f land acquisition inthe course ofremedialworks as applicableunder OP 4.30 (Resettlement). Inpractice, no such acquisition was needed, and the issue o f resettlement o fpeople didnot arise. STRENGTHENINGCOUNTRY SYSTEMSFOR SAFETY OF DAMS 53. An important objective o fthe project was to strengthen the institutional and regulatory frameworks for dam safety inIndonesia. The project also sought to strengthen dam safety considerations inthe process o fbidtendering, construction, O&M and associated works. These efforts are inline with the principal objectives o f the Bank's Safeguard OP 4.37 for "Safety o f Dams"21. 54. The project also introduced a comprehensive set o f dam safety regulatory guidelines which brought the country's standards inline with those o f the Bank. This can facilitate the Bank's new approach for using Country Systems22for environmental and social safeguards infuture projects that involve dam safety and rehabilitation. 55. Thus the project has helpedto advance the framework o f safeguards for safety of dams inthe country and to mainstream it to some extent at the central andprovincial levels. However, as discussed under the section on Outcome, Indonesiahas a long way to go inconsolidatingits dam safety institutions, especially inproviding them with adequate resources to function effectively. BANKPERFORMANCE 56. Project design was appropriate to the country's needs. It balanced the immediate needfor improving safety o fdams inthe country with the need to institutionalize dam safety concepts andprocesses inthe long term. The Bank demonstrated a good understanding o f technical issues and supplemented it with international expertise in creating a sound design for remedial works as well as for institutions and processes required to promote dam safety. But, inretrospect, the Bank was optimistic about the 21. Two important objectives o f the Bank's Safeguardpolicy for "Safety o fDams" (OP 4.37) are to: (i) work with the borrowing countries to strengthentheir institutional, legislative, and regulatory frameworks for dam safety programs; and (ii) ensure that the borrower adopts and implements dam safety measures for the design, bid tendering, construction, O&M o f the dam and associated works, and to make sure that experienced and competent professionals design and supervise construction 22. In2005, the Bank launched a Country Systems pilot to explore using a country's own environmental and social safeguard systems, where they are assessed as being equivalent to the Bank's systems, inBank- supported operations. K e y to the approach would be an increased emphasis by the Bank on capacity- building 15 ability o f the Government and implementing agency to respond to training and capacity- buildingefforts ina short period o ftwo years.23 The project ICRacknowledges that project preparation should have included an institutional expert with knowledge o f the Indonesianlegal and regulatory systemright inthe beginning. On the whole, quality at entry is ratedModerately Satisfactory. 57. Duringthe economic crisis, the Bankworked to easethe Government's requirement for counterpart funds throughraising the disbursement percentage for civil works from 55 to 80 percent from 1999 onwards. The Bank also showed flexibility in extending the project to enable completion o f on-going activities. 58. TheBank's supervision missions appropriately advised the implementing agency to urgently strengthenthe capacity o f the DSU, ifnecessary through co-opting experienced engineers from PLN. But the Bank could have done more duringthe mid- term review to ensure that satisfactory training and quality assurance arrangements were inplacebefore continuing with remedialworks andinstallationofbasic damsafety facilities. The Bank's supervision i s ratedModerately Satisfactory. Overall, Bank performance i s ratedModerately Satisfactory. BORROWER PERFORMANCE 59. Government commitment to the project was broadly consistent. An important enabling Ministerial Decree for Dam Safety (1994) was passed ahead o fproject effectiveness, and the Government persisted with the project duringthe economic crisis. The current Ministerial Decree i s inthe process o fbeing replaced by a more comprehensive PresidentialDecree that carries greater coverage and authority. The Government has maintainedthe basic structure o f dam safety institutions created during the project, which continue to carry out several important functions, though considerably below their potential. 60. To some extent, the Government could have insulatedthe new dam safety institutions from the effects o f at least two rounds o f organizational changes24,which resulted inrelocation o f the units, loss o f trained staff, and diluted focus on dam safety issues. It i s recognizedthat the Government has beenconstrained inproviding more staff to the dam safety institutions due to the overall zero-growth policy towards civil service staffing duringthe last two decades. More recently, the Government has abolished its 23. The S A R (paragraph 6.3) states that'I...there i s a relatively small riskthat it may take longer than at present anticipated for the new dam safety institutions to become operational and for their staff to be trained to the level requiredto manage nationwide program ...'I. 24 In 1999, MPW was split into the Ministry o f Settlements and Regional Development (KimBangWil) and a State Ministry for Public Works ((MeNegPu). DSC and the Project Management Unitremained with KimBangWil, while D S U was places with MeNegPu and several o f its staff were transferred out. In2000, the two ministries were combined into the Ministryo f Settlements andRegionalInfrastructure (KimPrasWil) within which the present DGWRwas located, and the D S U was transferred back to the new ministry,before the current structure as inAnnex C emerged. 16 zero-growth policy, and hundreds o fnew engineers have been recruitedrecently. On balance, government performance i s ratedModerately Satisfactory." The Government still needs to provide greater budget resources to the dam safety institutions, and increase allocations for dam O&M based on prioritized needs. Onbalance, government performance is ratedModerately Satisfactory. 61. The implementingagency, DGWR, initially tended to focus more on administrative and financial matters than on effective implementation andadherence to its own quality assurance requirements. But this improved significantly towards the end o f the project. The `training needs assessment' was delayed, as was the appointment o f consultants for technical assistance. Duringthe project, DGWR could have done more to ensure that the DSUwas staffed adequately, even ifit meant co-opting experienced staff from PLNas suggested by the Bank. Financial management, progress reporting, accounts, audits andprocurement were adequate. Implementing Agency performance i s ratedModerately Satisfactory, but barely so due to the initial delays intraining and remedial works. Overall, Borrower performance i s also ratedModerately Satisfactory, albeit marginally so. Lessons Keylessons from the project experience are: To sustain outcomes ina situation where new concepts andprocesses are introduced, the focus on institutional development needs to be maintained during the project and beyond. The risk to dam safety inthis project remains moderate due to relatively lower attention to institutional development vis-a-vis physical activities duringimplementation and inadequate follow-up thereafter. New regulatory institutions require technical expertise and adequate operating budgetsto function ina capable and independent manner. Inthis project, shortage o f experiencedqualified staff and inadequate operating budgets contributed to under-performance o fthe dam safety regulatory unit. Buildingdam safety institutions canbe a longprocess when it requires major changes inorganizational attitudes, establishing new work methods, and upgrading technical expertise. Insuch cases, the Bank should consider an extended commitment through a sequence o f projects or through adaptable program lending (APL). When new monitoring andreporting procedures are instituted, their purpose should be made clear to all those involved, and appropriate and regular feedback should be given to those originating the data. Inthis project, there was little systematic analysis or feedback from higher levels on dam level reporting, which progressively affected quality and compliance. New techniques and equipment should be tailored to the existing level o f capacity; facilities andfunds for ongoing maintenance; and site security. Most o f the relatively sophisticated instrumentation installed at dams underthis project has deteriorated for lack o fmaintenance. 17 Project benefits should be quantified to the extent possible inorder to provide clear economic justification for an investment and improve stakeholder support. The main benefit from this project i s the prevention o f losses fiom dam failure, which can be difficult to quantify. However, other benefits such as increased water supply are more amenable to beingquantified and may welljustify the investment. 19 Annex A AnnexA. BasicDataSheet INDONESIA (L3642-IND) DAMSAFETYPROJECT Appraisal Actual or Actual as % of estimate current estimate appraisal estimate Total project costs 97.4 54.9 56 Loan amount 55.0 35.2 64 Co-financing -- __ _ _ Cancellation _- 19.8 -- CumulativeEstimatedandActual Disbursements FY95 FY96 FY97 FY98 FY99 FYOO F Y O I FY02 FY03 FY04 Appraisal 4.6 13.8 29.4 41.3 48.6 53.5 55.0 55.0 55.0 55.0 estimate (US$M) Actual (US$M) 3.0 7.8 14.8 19.9 23.8 28.7 31.2 33.4 35.2 35.2 Actual as % of 65 56 50 48 49 54 57 61 64 64 appraisal Date of final disbursement: 10/06/04 Original Actual PCD 04/07/1993 Appraisal 09/27/1993 Board approval 05/31/1994 Effectiveness 08/23/1994 08/23/1994 MTR 09/22/1997 09/22/1997 Closing date 09/30/2000 0313112003 ANNEX A 20 Stage of Project Cycle Staff weeks US$ (`000) ~ Identification/Preparation 7.0 24.0 AppraisaVNegotiation 49.2 190.8 Supervision 153.6 599.2 ICR 12.0* 32.5 Total 221.8 846.5 Mission Data Date No. of Specializations lmplementation Development (month/year) persons represented status objectives Identification/ Preparation 4 Team Leader; 1993 Procurement./Operations Specialist; Dam Safety Specialist; Senior Engineer Appraisal/ Negotiation 5 Team Leader; 1994 Procurement./Operations Specialist; Dam Safety Specialist; InstitutionallOrganization s Specialist; Senior Engineer Supervision 11/04/1994 3 Mission Leader; S S Organization Expert; Dam Engineer 05/24/1995 3 Agricultural Economist; Dam S S Engineer; Dam Safety Eng. 01/30/1996 Dam Engineer; Dam Safety S Eng. 08/13/1996 Agricultural Economist; Dam S Engineer; Dam Safety Eng. 02/28/1997 Agricultural Economist; Dam S Engineer 10/03/1997 4 Agricultural Economist; Dam S Engineer;Civil Engineer; Dam Safety Engineer 12/10/1998 3 Civil Engineer;Water U S Resources Engineer 05/13/1999 3 Team Leader; Civil Engineer; S S Water Resources Engineer 11/I9/1999 Team Leader; Civil Engineer; Water Resources Engineer 11/I9/1999 Team Leader; Civil Engineer 12/08/2000 Team Leader; Civil Engineer 12/08/2000 Dam Safety Expert 04/21/2001 Task Team Leader; Civil Engineer; Dam Safety 21 Annex A Date No. of Specializations Implementation Development (month/year) persons represented status objectives InstitutionSpecialist 09112/2001 3 Task Team Leader; Civil S S Engineer; Team Assistant 09/12/2001 3 Mission Leader;Civil S S Engineer; Dam Safety InstitutionSpecialist 12/12/2001 3 Team Leader; Civil Engineer; S S Dam Safety Institution Specialist 05/31/2002 3 Team Leader; Dam Safety S S InstitutionSpecialist; Dam Safety Expert ICR 09113/2002 5 Team Leader; S S Dam Safety Institution Specialist; Dam Safety Expert; Procurement Specialist;Community DevelopmentSpecialist Other ProjectData Borrower/ExecutingAgency: FOLLOW-ONOPERATIONS ............. " " .I . .I ...........I" ............ .. " ......". .... ...-..".I.....I.........I .... ......"....."......_.."."..I_. .ll...l ............I. ..,,,.. .. . ,. ................................. ..........I. ... Operation Credit no. Amount Board date (US$ million) Indonesia Dam Operational Improvementand -_ _ _ 12/16/08 (R) Safety Project (DOISP) 23 Annex B Annex B. List of Dams Sites visited and Organizations/Departments consulted DamSites visited OrganizationsandOfficesconsulted Indonesia Directorate-General o f Water Resources, Ministryo f Public Works Director, River Lakes and Reservoirs, DGWR Dam Safety Unit Dam Safety Commission Balai PSDA and Balai WS and Dam Site Office Staff o f the 9 Dams listed above Study Team for "Model Dam" from Universitas Sebelas Maret Mulyarejo Foundation World Bank Project Task Team Leader and Water Sector Team, World Bank Country Office, Jakarta East Asia Rural Development, Natural Resources & Environment Division, World Bank, Washington D.C. 25 Annex C Annex C.DamSafety Institutionsand Data Reporting Relationships ~ MPW (Ministryof Ministryof Public PublicWorks) Corporations Directorateof Directorateof Water Rivers, Lakes Resource and Reservoirs Management I I : I I I I 1 Monitoring Field Dam !;-------- . - ~I .----_ _---- I I Dataand 1 [ Training i & Monitoring Design 1I ! Evaluation and and ii II Records ; i I and Section* 1 Section Review Review I 1------_-__-.Monitoring ,. j_ I Sections Section ! !I ; _Section*_ _ _ _ _[I _ _ _ _ _ 1 :I I I II Dinas PUP(Provincia1Water 1.L ResourcesServices) l i PDMU (Provincial Dam Monitoring Balai Wilayah Unit) Sungah (WS) (CentralRiver Balai PSDA I - I (River Basin /./ Management /./ Unitsof Bali Basin /' Dinas PUP) / / \. / \.'. /./ /' \ / . . Dam Units 1, 2, 3.....N *Unitsdo notpresentlyexist -SinceJan 2008, the Sub-Directorateof O&M is under the Directorateof Rivers, Lakes and Reservoirs -,-,- Dab ReporfingRelationship 27 Annex D Annex D. Borrower Comments FROM :DIT. SDW FRX NO. :0217261292 Mar. 30 2809 16:30 P I D E P A R T E M E N P E K E R J A A N U M U M D I R E K T O R A T J E N D E R A L S U M B E R D A Y A A I R 31. Pattimura No.20 Kebayoran BaruJakarta Selahn 121iOTelp. 7396616 Fax. 7208285 RyF: HL-ol-02-D&w Jakarta, March27,2009 To : MonikaHuppi Manager sector EvaluationDivhion Independent EvaluationGroup The World Bank Re :INDONESIA DamSafety Project (CPL-37420;SCL3742A; SCPD-3742s) Draft Projectperformance Assessment Report Dear Madame, Concerningthe World Bank's letter dated March5,2009, for the above mention subject, we hereby send you our explanation to the achievement of the importantcomponent of the DamSafety Project, . 1. RemedialWorks, Duringthe DamSafety Project, 73damshadbeenrehabilitated, consistingof onebig of JatiIuhur dam and72smaller dams. Inour opinion, the completionof remedialworks and supervisionof Jatiluhur damcontribute the significant positive impact to the national program After remedial works of Jatiluhw dam which is has very high hazard classification, the physical risk of failure of this dam was significantly reduced. As additionalbenefit, the services ofJatiluhur damafter remedialworks to continuously supply of 224,000 ha irrigationarea, supply of rawwater of 16m3/second to Jakarta city, andpower supply of 110MW,concluded thehighachievementto thewholeproject. 2. Performance d DamSafety Commiaaion(DSC) and DamSafety Unit (DSU) Uponthe effectiveness of DamSafety Project, the DarnSafety Commissionand Dam Safety Unit, were supported to meet the main goal as independent body incharge with makingrecommendations to the Minister of Public Worksondamsafety issues. We agree to the Bank that lack of engineer appointed as staff to DSU makingless performing its five-yearly inspections of all large darns to certify their safety ox to establish rehabilitation needs. However until the end of the year 2008, the DSU have already finished eighteen certifications for new designs, thirteen certifications for dam impounding, fifteen certifications for damoperation, andnowseventeen certifications of damunder process. We thereforeconcludedthat, eventhoughwiththe limitation of technicalstaff assignedto mu,the performance of DSUto supportDSChasbeenratedsatisfactory, Andtocatch upthetarget, wewidgiveanattention tostrengthenthe DSU. FROM :DIT. SDW FRX NO. t0217261292 Mar. 30 2009 16:31 P2 28 3. DamMonitoringUnitandThe Government's Effortsto Strengthenthe Operation andMaintenanceof Dams The establishment of Central dam MonitoringUnit (CDMU) and DamMonitoring Unit (DMU) were meet the target as stated inthe Staff AppraisalReport of DamSafety Project. Eventhough the performance is slightly below the expectation, two other DMUs that were created intwo PJTsfor O&M of M P W ' s large infrastructure inthe Citarumand Brantasbasinas mentioninBank's letter are still functioning as intended+ The government is fully committed to providing ongoing support to strengthening the CDMU/DMU, andpreparationof adequateO&Mbudgetfor darns. We also informyou the govemmenfs effort to clearing the role of the provincial DMU and River Territory Management Units through provision of Director General Water Resources Letter no UM 01 03-Da/08 dated January 9, 2009 subjected the application of Memorandum of Understanding (MOU) between BBWS/BWS (National River Basins Organization, under the control government) and Provincial Water ResourcesOffice, Basedon this MOW, continuationof O&M of dams wouldbe gradudy become better, Basedonthe abovecoment, we therefore concluded that the overallDamM e t y Projectisratedsatisfactory, andwe hopethat this explanationwillbereflectedas pos3ive impacttothefinalreporttoBoardof ExecutiveDirectorsof theWorldBank. Yours sincerely, cc: 1. MinisterofPublicWorks 2 Secretary General of PublicWorks 3. Deputychirman,InfrastructureBappenas 4. Directorof Water Resources andIrrigationBappenas 5. Director of ProgrammingGuidance DirectorateGeneral of Water Resources 6. Directorof Rivers, Lakes andreservoirs DirectorateGeneralof Water Resources 7, Headof Damsafety UnitMinistry of PublicWorks