ICRR 13915 Report Number : ICRR13915 IEG ICR Review Independent Evaluation Group 1. Project Data : Date Posted : 04/29/2014 Country : Indonesia Is this Review for a Programmatic Series? Yes No How many operations were planned for the 4 series? How many were approved? 4 Series ID : S118531 First Project ID : P107163 Appraisal Actual Project Name : Id-infrastructure Dpl US$M ): Project Costs (US$M): 200 200 (idpl) L/C Number : L7494 Loan/ US$M ): Loan /Credit (US$M): 200 200 Sector Board : Public Sector US$M ): Cofinancing (US$M): Governance Cofinanciers : Asian Development Board Approval Date : 12/04/2007 Bank, Japan Closing Date : 03/31/2008 06/30/2008 International Cooperation Agency Sector (s): Water supply (30%); Power (30%); Roads and highways (25%); General transportation sector (10%); Telecommunications (5%) Theme (s): Urban services and housing for the poor (29% - P); Infrastructure services for private sector development (29% - P); Environmental policies and institutions (14% - S); Land administration and management (14% - S); Public expenditure; financial management and procurement (14% - S) Second Project ID :P111905 Appraisal Actual Project Name : Second Infrastructure Project Costs ( US$M): US$M ): 200 200 Development Policy Loan L/C Number : L7607 Loan/ Loan US$M): /Credit (US$M ): 200 200 Sector Board : Urban Development US$M): Cofinancing (US$M ): Board Approval Date : 12/04/2007 Cofinancers :Asian Development Closing Date : 03/31/2008 06/30/2008 Bank, Japan International Cooperation Agency Sector (s): General transportation sector (32%), General water sanitation and flood protection sector (32%), Power (32%), Telecommunications (4%) Theme (s): Infrastructure services for private sector development (29% - P), Decentralization (29% - P), Environmental policies and institutions (14% - S), Municipal governance and institution building (14% - S), Urban services and housing for the poor (14% - S) Third Project ID :P115102 Appraisal Actual Project Name : Third Infrastructure Project Costs (US$M): US$M ): 250 250 Development Policy Loan L/C Number : L7785 Loan/ Loan US$M): /Credit (US$M ): 250 250 Sector Board : Urban Development US$M ): Cofinancing (US$M): Board Approval Date : 12/09/2009 Cofinancers : Closing Date : 03/31/2009 03/31/2009 Sector (s): General transportation sector (32%), General water sanitation and flood protection sector (32%), Power (32%), Telecommunications (4%) Theme (s): Rural non-farm income generation (29% - P), Infrastructure services for private sector development (29% - P), Environmental policies and institutions (14% - S), Municipal governance and institution building (14% - S), Urban services and housing for the poor (14% - S) Fourth Project ID :P118531 Appraisal Actual Project Name : Fourth Infrastructure Project Costs ( US$M): US$M ): 200 200 Development Policy Loan L/C Number : Loan/ Loan US$M): /Credit (US$M ): 200 200 Sector Board : Urban Development US$M ): Cofinancing (US$M): Board Approval Date : 11/18/2010 Cofinancers :Asian Development Closing Date : 03/31/2011 03/31/2011 Bank, Japan International Cooperation Agency Sector (s): General water sanitation and flood protection sector (32%), General transportation sector (32%), Power (32%), Telecommunications (4%) Theme (s): Rural non-farm income generation (29%), Infrastructure services for private sector development (29%), Environmental policies and institutions (14%), Municipal governance and institution building (14%), Urban services and housing for the poor (14%) Evaluator : Panel Reviewer : ICR Review Group : Coordinator : S. Ramachandran Roy Gilbert Christopher David IEGPS1 Nelson 2. Project Objectives and Components: a. Objectives: The series of four Infrastructure Development Policy Loans (IDPLs) were to fill the widening gap in infrastructure investment [presumably between actual and what a growing economy normally spends ]. IDPL-1's Loan Agreement describes the “program� being supported; the Project Document (PAD, page 37 to 55) describes the program and the objectives are explicitly listed only in Annex 3’s Policy Matrix (pp 88-89). Subsequent IDPLs alter the wording somewhat (for reasons explained in section 3b on the relevance of design ) without changing the thrust of the objectives which the ICR accurately summarizes (page vii) melding two of them (i.e. three objectives, instead of four ). The original four objectives listed in IDPL -1’s policy matrix are: 1. Increased quantity and efficiency of Central Government spending on infrastructure through improved subsidy/PSO [public service obligation]policy, and expenditure planning and budgeting 2. Improved sub-national infrastructure services through increased sub -national government spending and an improved incentives framework 3. Increased private investment in infrastructure through the establishment of a fiscally sound credible and transparent PPP [Public-Private Partnership] framework 4. Enhanced governance for infrastructure through improved land acquisition, environmental protection, and procurement, and audit processes within the Ministry of Public Works . These objectives were to be monitored through several indicators that the project documents list (and the ICR accurately summarized); but several of these “indicators� were sub-objectives that were dropped when subsequent IDPLs “streamlined� the indicators. These issues are discussed further under the M&E section . b. If this is a single DPL operation (not part of a series), were the project objectives/ key associated outcome targets revised during implementation? No c. Policy Areas: The four original policy areas listed in the PAD (pages 40 to 55) were: I National Spending on Infrastructure (the national budget or its Indonesian acronym, ABPN, allocation for power and roads) II Sub-national spending on infrastructure III Private sector financing through Public -Private partnerships IV Cross-cutting areas: land acquisition, environmental protection and fiduciary responsibility in public procurement and other spending. Each of the four loans in the series supported reforms in all four policy areas, although the emphasis on each shifted over time. For example, IDPL-1's rural electrification goals were dropped during IDPL -2 which emphasized the better targeting of subsidies instead, partly because the goals could not be met until the power companies increased cash inflows. Raising these cash inflows from higher revenue required a shift from general to targeted subsidies . So the series of IDPLs retained its policy goals and objectives, and adapted the measures needed to achieve them as country situation changed . d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: The first IDPL's PAD envisioned three loans (not four) to correspond with the 18 month lending cycle of the Asian Development Bank. Beginning with the IDPL-2, this schedule was altered to correspond to the government's annual budget cycle, and a fourth IDPL was added : so IDPL-2 through 4 were approved at the end of 2008, 2009 and 2010. The four IDPLs totaled $850 million. Parallel multi-year loans from the Asian Development Bank ($400 million and $280 million) and the Japan International Cooperation Agency ($100 million in March 2007 and another $100 million in February 2009) augmented the amounts (IDPL-3's PAD, page 32, para 93). These loans combined totaled $ 1,730 million, but still funded only a small fraction of Indonesia's large infrastructure needs (6 percent of 2012 GDP $936 billion is $56 billion a year). Substantial (though unspecified) private investments and PPPs were expected to follow when policy shortcomings were corrected . 3. Relevance of Objectives & Design: a. Relevance of Objectives: IDPL-1's PAD states (page 30) the program objectives were consistent with the Country Assistance Strategy (CAS) of 2003 and the CAS Progress Report of 2006. Investment in infrastructure had drastically fallen from over 7 percent of GDP before the 1997 East Asian economic crisis to roughly 2 percent and hovered around that level for a decade : it was almost 3 percent in 2008 when IDPL-1 was approved. There was concern that electricity shortages, congested roads etc. would hamper development and growth; so the four objectives were highly relevant . The government's Letter of Development Policy (Annexe 1) confirms that they were also the government's objectives . The issues in each of the different infrastructure sectors were diverse, and the IDPLs had a wide scope . With electricity , for example, installed capacity had barely increased between 2000 and 2006 and rising peak demands were causing black outs. Furthermore, almost half the rural population had no access to power, and many in urban areas sought illegal connections to the grid . Electricity tariffs were low and had not kept pace with inflation, let alone fuel costs that were rising even faster . Raising average electricity tariffs and devising a system of targeted subsidies were relevant objectives because without them, greater access would only exacerbate losses in the utilities, and ultimately the fiscal budget. The state owned utility's 29,705 megawatts of installed capacity was supplemented by an estimated 15,000 mw of private captive capacity, and augmenting this by harnessing private investment through PPP was therefore important. Poor construction and maintenance hampered the 34,628 km road network : 10 percent was in in poor condition and an additional 9 percent was in bad condition . Sub-national governments were responsible for road maintenance, and they were struggling to raise revenues and control spending; so better public finance management (PFM, supported by the earlier DPLs at the national level ) was needed also at the sub -national level. Corruption was known to be widespread in road construction; so better procurement of goods and services, more transparent land acquisition and more effective monitoring were needed . ater and treated sewage reduce disease and greatly improve the quality of life, but almost a fifth of the Piped water population lacked access . Investments in this sector had declined to a tenth of their 1990s level following the 2001 devolution to sub-national governments. Furthermore, water utilities (PDAMs in the Indonesian acronym) and/or their sub-national government owners were in arrears, and so precluded from any borrowing . Clearing these arrears and the set of related measures were also required . The program sought to increase overall government spending on infrastructure at both the national and sub -national levels, improve allocation across and within sectors, and to change policies that would increase private infrastructure investment. The numerous indicators/sub-objectives could have been usefully put into hierarchies thereby ensuring greater attention to the more important measures, as discussed in the section on Monitoring and Evaluation . b. Relevance of Design: A results framework linking chosen policy actions to the possible achievement of the DPL objectives was logically constructed for this operation . TheIDPL-1’s policy matrix shows how the first objective (increased infrastructure spending) would be achieved by increased budget allocations and publishing them more widely across Indonesia . The second objective (improved subnational infrastructure services ) would be achieved—although less convincingly so than is the case for the other objectives —by actions such as restructuring of subnational government debts and implementing (unspecified) performances incentives into the inter -governmental financial framework. The third objective (increased private investment in infrastructure ) would be achieved through private -public participation PPP tenders. The fourth objective (enhanced governance for infrastructure ) would be achieved principally through improved land acquisition policies identified by an inter -ministerial working group, and another working group to address improvements to procurement of infrastructure works in Indonesia (PAD pp. 88-90) The broad scope of the IDPL’s objectives required the Bank's engagement with many Ministries and levels of government. The series of loans allowed each sector to benefit from the lessons being learnt in other sectors and, because each IDPL was prepared while the previous one was being supervised, the necessary changes and adjustments could be incorporated into the subsequent IDPLs . The program being supported was coherent and the appropriate measures were being contemplated . The design provided ample opportunity to undertake additional or corrective measures that were subsequently found to be necessary. So the relevance of design was substantial . 4. Achievement of Objectives (Efficacy): The achievement of objective 1, namely, increased quantity and efficiency of Central Government spending on infrastructure through improved subsidy /PSO [public service obligation] policy, and expenditure planning and budgeting has been modest . Aggregate infrastructure spending has risen from 2 to 3 percent of GDP in the decade before IDPL-1 was approved to about 4 percent of GDP (Chart 3.1 and Table 3.1 in IDPL-4) by 2009, well short of the 6 percent of GDP that was sought . Examining developments by sector, Table 3.2 in IDPL-4's project document shows no increase in available capacity in electricity . Electricity tariffs remained unchanged since 2003 and only changed in 2010 but not to the level needed for cost recovery . Consequently, subsidies remain large, and the ICR reports an audit that found their incidence to be slightly more regressive than before . Clearly a better system to recover costs and target subsidies is not in place . The road network has not been extended but the ICR reports a slight improvement in their condition: 86 percent of national roads are in good /fair condition in 2010, up from 81.6 percent in 2006 and 84 percent targeted in IDPL-2. The achievement of objective 2, namely, improved sub-national infrastructure services through increased sub-national government spending and an improved incentives framework has been negligible . Access to piped water and sewage treatment have barely changed. There has been little improvement in an intermediate "output" that the IDPLs sought to improve, namely arrears in the PDAMs . The government had sought to (i) restructure the debts of sub-national governments and 165 eligible PDAMs (ii) institute a sanctions to deter a recurrence, and (iii) provide incentives to improve the effectiveness of sub -national spending. The Ministry of Finance has approved 68 of the 106 restructuring plans but no PDAM has yet borrowed to invest . Water projects undertaken have been mostly donor funded. The ICR states "little was achieved in rationalizing energy subsidies, PDAM restructuring has not taken place, and progress in developing well -structured, compliant PPP transactions has been painfully slow " (page 10). The achievement of objective 3, namely, increased private investment in infrastructure through the establishment of a fiscally sound credible and transparent PPP [Public-Private Partnership] framework has been modest . While such a framework has been set up, there has been only one compliant PPP transaction to date . Private commitments overall to infrastructure have fallen from 0.5 percent of GDP in 2005 to 0.3 percent in 2010. PPP requires private sector confidence in "the rules of the game" and while some inputs have improved (e.g. Perpres 67 issued), the dearth of substantial PPP investments suggests that outcomes were not achieved . The achievement of objective 4, namely,enhanced governance for infrastructure through improved land acquisition, environmental protection, and procurement, and audit processes within the Ministry of Public Works has been modest . The ICR reports improvements in several inputs : electronic procurement that increases transparency, better laws for land acquisition have been promulgated, internal audits are being conducted and on -line public procurement is being used. These improvements would likely reduce waste and corruption as these practices spread and take root. However, these are inputs to more and better infrastructure, not outcomes . With greater devolution to the sub-national level of government, better budgeting and expenditure controls are important, but these improvements are not easy in a vast country . Overall, some things have improved but these were insufficient to increase infrastructure investments in the aggregate to the 6 percent of GDP level that the IDPLs had sought . The lack of achievement despite relevant objectives, and design may be because the multiplicity of indicators made it easy to imagine more progress than was the case . Disbursement is unlikely to be delayed over seemingly "minor" issues or delays, particularly when there were also some advances : for example, when it was clear after the first IDPL that the system of targeted subsidies in electricity would take longer to put in place, the rural electrification requirement was dropped. When agreement on "prior actions" is urgent, excuses are found delays in other measures that were agreed to. With so many indicators, it is easy to lose the strategic perspective . Adequate supervision with attention to the strategic objectives of the program would have helped . Putting indicators into to hierarchical categories or periodically focusing on outcome indicators (infrastructure investment to GDP is available with a lag of several months ) would have alerted staff and management to meagre progress before the next loan was approved. 5. Efficiency (not applicable to DPLs): 6. Outcome: Although the objective of increasing infrastructure investment was relevant, it has not increased to the 6 percent of GDP level that the program sought (not an explicit target, but the main rationale for the loans ): the slight increase from 3 to 4 percent of 2010 GDP was in line with the earlier trend (from 2 percent in 2000 to around 3 in 2007). Major policies (e.g. electricity pricing, ensuring financing for piped water ) have not changed despite five years of effort under this program (and earlier efforts as well) and an aggregate (Bank, ADB etc) of $1,730 million in disbursed loans. There has, however, been some progress (qualitative) in several inputs needed for better outcomes : Public Financial Management (PFM begun under the earlier DPLs) has improved at the national level and in some sub -national levels of government; procurement practices and land acquisition are being improved, but whether these will translate into better outcomes in infrastructure remain to be seen . a. Outcome Rating : Moderately Unsatisfactory 7. Rationale for Risk to Development Outcome Rating: Adequate infrastructure requires continual investment in technology as well as in physical and human capital . Considering the financial shortfalls (for electricity) and insufficient tax revenues to finance roads and piped water (priced below full cost recovery ), it is unlikely that future infrastructure investments would be adequate . But although the increase in quantity and allocation of infrastructure investment at the national and sub -national levels was not attained, it is unlikely that they would fall in either absolute terms or as a fraction of GDP, and the risk to development outcome of this series of loans is therefore moderate . a. Risk to Development Outcome Rating : Moderate 8. Assessment of Bank Performance: a. Quality at entry: The ICR notes that the "design was based on good analytical work, Government demand for reform and focused on the principal areas impacting Indonesia's infrastructure deficit ." If this were entirely true, the reforms would have been easier to achieve; so support for these changes was not widespread and Parliament did not approve many needed measures, particularly raising electricity prices . The Bank could have anticipated these difficulties because they are not unique to Indonesia . The Bank framed the issue as the need to target subsidies better . Even if prices had been raised, changing prices later to reflect changing costs and technology would be equally difficult . A lasting solution would be to remove Parliament's role in setting prices, allowing the electric utility to alter prices as and when necessary . Each loan was prepared while the previous one was being supervised; so it would have been possible for the necessary devolution of price setting authority to have been made a prior action for the next loan . Such a change in course is not limited to electricity : piped water and road maintenance were devolved to sub-national governments without adequate revenues . If cost recovery from users was the governing principle, there should have been a mechanism to ensure sufficiently higher prices . Instead, the IDPLs put an unrealistic reliance on better PFMs in the hundreds of sub -national entities. Again, if this had not been initially considered, there were ample opportunities to have changed this in the subsequent IDPLs . Whether such omission/neglect reflects quality at entry or of supervision is difficult to say; but deferring to the ICR statement quoted, this review holds supervision responsible (discussed below). Nevertheless, the IDPLs lost their focus by including several types of infrastructure with very different problems . Had it limited itself to electricity and focused on getting Parliament out of approving prices, the program may have been more successful. Alternatively, it could have focused on roads (where land acquisition and procurement without corruption were to be improved). Clubbing reforms in disparate sectors into large loans that get the central government attention at approval does not always translate into continued attention over the diverse details . at -Entry Rating : Quality -at- Moderately Unsatisfactory b. Quality of supervision: Supervision in a series of loans should not lose sight of the objectives, and the opportunities to ensure greater progress towards the objectives appear to have been lost . It was not until IDPL-4 that electricity prices were raised (in 2010, the first change since 2003), and even with this increase there was no full cost recovery . The short period between the loans, and the urgency to agree on prior actions for the next may have lost the focus on the larger objective of raising infrastructure spending (without waste). A single tranche loan may require little supervision; but a series of quick disbursing loans requires greater attention to the objectives . The ICR's Annex 1 Table b shows total staff weeks spent on the loans were halved between FY08 and FY09, and halved again in FY10, and remained at this level in FY11. While this decline could reflect greater efficiency, it may also reflect diminished attention when implementation challenges seem undiminished. A PPAR assessment would therefore be advisable . Quality of Supervision Rating : Unsatisfactory Overall Bank Performance Rating : Moderately Unsatisfactory 9. Assessment of Borrower Performance: a. Government Performance: The program engaged multiple Ministries, agencies and departments within them, and their commitment and performance varied. The ICR notes that performance at more senior and central levels was greater than elsewhere. One reason the program was broad in scope was for more reform oriented parts of the government to improve other parts of government (e.g. national versus sub-national etc.). Government Performance Rating : Moderately Satisfactory b. Implementing Agency Performance: The Coordinating Ministry of Economic Affairs (CMEA) was the central coordinating agency but the respective Ministries were responsible for implementation . The CMEA performed its coordinating function well : it ensured compliance with Bank procedures and helped staff access the relevant agencies and Ministries during supervision and preparation of subsequent loans . The CMEA lacked the political authority over other Ministries responsible for the sectors; so it could not do much to hasten reforms . The ICR notes that the CMEA could have monitored the M&E indicators more actively instead of relying on Bank missions to spur data collection . Implementing Agency Performance Rating : Moderately Satisfactory Overall Borrower Performance Rating : Moderately Satisfactory 10. M&E Design, Implementation, & Utilization: a. M&E Design: The program used a variety of indicators to monitor progress towards the objectives : some of these were input indicators, others were output or outcome related . Several indicators were dropped even before the second IDPL was approved just 6 months after IDPL-1, underscoring the likelihood that they were chosen without sufficient thought . IDPL-1 linked infrastructure investment with poverty alleviation (PAD's Annex 5 described the links in detail), but full cost recovery in electricity may hurt some of the poor (those with access) if a system of targeted subsidies were not introduced . Excessive attention to the regressive incidence of tariff changes would only maintain the status quo . Low cost recovery and neglected maintenance were serious issues for many years, and under these conditions, poverty alleviation could become a distraction. The Bank's experience in many countries where services were long provided without adequate cost recovery should have underscored the importance of monitoring cost recovery . Considering the extent of sector work before the program was approved, better indicators could have been chosen . For example, the indicator of spending on infrastructure to increase by 25 percent over the 2006 level did not even specify if this would be in nominal or real terms . The indicator could also have been scaled by a measure of economic activity. It would have also been useful to distinguish input from output and outcome indicators : doing so may have alerted senior officials in the Bank and government to the very meagre increase and improvements in infrastructure. b. M&E Implementation: Some of the data needed for the M&E indicators chosen were neither available in a timely manner nor sufficiently reliable. The ICR reports that the implementing agency did not actively seek the data nor monitor the selected indicators; so the government did not use them . As mentioned earlier, the Bank did not use them well either since it did not distinguish input indicators from outcomes . c. M&E Utilization: Limited, partly because several indicators were input related and insufficient attention was given to the minimal improvements in output and outcomes . M&E Quality Rating : Modest 11. Other Issues a. Safeguards: No issues raised in the ICR. b. Fiduciary Compliance: No issues raised in the ICR. c. Unintended Impacts (positive or negative): The ICR reports that one unintended benefit of these IDPLs was the land acquisition law that resulted from the deliberations of the Land Working Group constituted under the project . Similarly, the success of the pilot projects for output based fiscal support in the water sector may benefit road maintenance and construction that the government is considering for this government program . The difficulties in tracking infrastructure expenditures at the national and sub-national level have also triggered efforts to improve data collection at the Ministries concerned . d. Other: 12. 12. Ratings : ICR IEG Review Reason for Disagreement /Comments Outcome : Moderately Moderately Unsatisfactory Unsatisfactory Risk to Development Moderate Moderate Outcome : Bank Performance : Moderately Moderately The Bank over-estimated public and Satisfactory Unsatisfactory Parliamentary support for the reforms, and did not take corrective measures or alter lending volume when it should have been clear that progress towards the objectives was inadequate . Infrastructure investment has not increased and no major policy impediment has been removed despite five years of work, and $1,730 million in disbursed loans from the Bank and co-financers. Borrower Performance : Moderately Moderately Satisfactory Satisfactory Quality of ICR : Satisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 13. Lessons: The ICR lists several lessons including the need to link performance indicators to development objectives through a results framework. In addition, there are lessons on how to retaining the focus on the overall objectives : with so many indicators, not all equally important, progress on some inputs and the urgent need to specify "prior actions" for the next loan give an illusion of progress . One additional lesson is the need for a thorough a mid -term review that takes stock of any progress, and alerts senior officials in government on what remains for the objectives to be attained . If an important issue was overlooked initially, or framed poorly (e.g. regressive subsidies instead of how to de -politicize electricity pricing), a mid-term review could have introduced it as a prior action for the loan that followed . A series of programmatic loans require mid-course corrections, and subsequent loans should not proceed on autopilot with perfunctory approvals. 14. Assessment Recommended? Yes No Why? These were four large loans, the first of their type dealing with multiple sectors and levels of government . There may be other lessons that could be gleaned by reviewing them in detail, perhaps in conjunction with conventional infrastructure projects undertaken around the same time . Such an assessment would help determine if more is gained or lost when different sectors are tackled in a combined policy loan . 15. Comments on Quality of ICR: The ICR clearly describes what was achieved, what was not, and why . The ICR is commendably frank and the ratings are fair and well-justified, except for Bank performance . The ICR could have extended self -evaluation further if it had: 1. discussed the risks to development outcome as they are now, instead of only reviewing whether macro risk was appropriately described in the original project documents 2. used the adjective "nominal" or "real" when describing spending (changes or amounts) and scaled them by GDP or other relevant measures to make comparisons with other countries easier . While the ICR discusses issues and progress by sectors and rates them fairly, the guidelines call for the objectives as stated to be rated. a.Quality of ICR Rating : Satisfactory