EXECUTIVE SUMMARY OF EVALUATION FOR PUBLIC DISCLOSURE FROM: C3P Evaluation of the Infrastructure Development Collaboration Partnership Fund (DevCo), a Private Infrastructure Development Group (PIDG) Facility Redacted Version 1. The Executive Summary of the DevCo Fund Evaluation has now been redacted for public disclosure in accordance with IFC’s 2012 Access to Information Policy, following the Procedure for Development, Management and Disclosure of IFC Evaluations effective on 04/01/2014. 2. The attached redacted version reflects the following adjustments:  Redaction of sensitive or confidential information related to confidential information provided to Castalia (Redacted Executive Summary, page 4).  Minor typographical corrections. 3. Questions on this document should be addressed to Dilyan Donchev (DDonchev@ifc.org) Page 1 of 6 Executive Summary of Evaluation Evaluation of the Infrastructure Development Collaboration Name of Evaluation Partnership Fund (DevCo), a Private Infrastructure Development Group Facility (PIDG) Date of Final Report of Evaluation January 2014 Author of Final Report of Evaluation Castalia Limited Date of this Executive Summary June 30, 2014 Author of this Executive Summary Castalia Limited; Dilyan Donchev, IFC, DDonchev@ifc.org Number of pages of this Executive Summary 5 Modifications from original Executive Summary With Modification (Please refer to cover letter) Executive Summary Approved for public PIDG PMU; Anna Hidalgo, IFC-ECRIO, disclosure by (name) on (date) AHidalgo@worldbankgroup.org Background DevCo was created in 2003 to support governments in poorer developing countries in attracting private sector investment to infrastructure. Given a recommendation from PIDG, DevCo was established by the IFC and the United Kingdom’s Department for International Development (DFID) with an initial capital fund of US$12.13 million1. Over time, DevCo gained more donors. Through PIDG, DevCo now receives funds from five donors, including DFID, IFC, Austrian Development Agency (ADA-BMF), Dutch Ministry of Foreign Affairs (DGIS) and the Swedish International Development Cooperation Agency (SIDA). DevCo provides funds to support transactions that IFC implements for developing country governments. IFC charges governments for its transaction advisory services, but many governments in poorer developing countries cannot afford the costs. DevCo helps these governments by covering the costs of consultants, engineers, lawyers and other specialists IFC engages to work on the transactions. DevCo can cover some of IFC’s costs associated with developing and implementing transactions. DevCo also funds monitoring and evaluation, knowledge products, and, on a selective basis, technical support for governments restructuring projects2 Objectives The Private Infrastructure Development Group (PIDG) engaged Castalia and Dr. Dieter Falk to evaluate one of its facilities: the Infrastructure Development Collaboration Partnership Fund (DevCo). DevCo’s performance was assessed against the five Development Assistance Committee (DAC) evaluation criteria: relevance, effectiveness, efficiency, impact, and sustainability. 1 “Memorandum of Understanding between the Government of the United Kingdom of Great Britain and Northern Ireland and the International Finance Corporation”. Infrastructure Development Collaboration Partnership Fund (DevCo Advisory). 26 June 2003 IFC committed US$750,000. DFID committed GBP6.8 million. Using 2003 conversion rate 1.674 (OANDA), this is equivalent to US$11,383,200. 2 DevCo Operating Principles Page 2 of 6 Overall, the conclusion is that DevCo has been successful—it is achieving most of its targeted results, and delivering the impacts that DevCo donors expect. Also, DevCo’s results justify its costs, especially when compared to other alternatives. However, DevCo could do more—it needs to increase the number of successfully bid-out projects, currently 60 percent below target. DevCo also needs to increase the amount of private sector investment raised in DAC 1, 2 countries3 (currently 82 percent4 below target5). Furthermore, DevCo could better address risks to sustainability by monitoring all closed mandates until they are delivering expected service levels. Finally, DevCo could have more impact by helping governments identify more privately financed infrastructure opportunities. Analysis Relevance DevCo is relevant to PIDG. DevCo objectives are well aligned to PIDG objectives. DevCo’s model is also still relevant and valid. DevCo’s model is to support the IFC’s Transaction Advisory Department on transactions that bring the private sector into the delivery of infrastructure services in developing countries. There is still a large infrastructure gap that private investors and operators can help to fill. Governments still need transaction advice. They regard IFC as the right institution to provide it, but are financially constrained and cannot pay for IFC’s services. These conclusions are drawn from evidence gathered during field visits to nine DevCo projects. In all nine cases, we concluded that the public authorities needed assistance in structuring and implementing transactions, and that they would not have been able to pay for IFC’s transaction advisory services without the funds DevCo provided. DevCo is supporting projects that governments prioritize. Results monitoring sheets for the nine sample cases explained how each project aligns with the country’s national development plan. During the field visits, we validated the information by interviewing government officials and reviewing policy documents. All nine cases were clearly infrastructure priorities for the governments concerned. DevCo is shifting IFC’s Transaction Advisory Department into frontier markets. From 2008-June 30, 2013, 71 percent of DevCo-supported projects are in DAC 1 and 2 countries. Among IFC’s non-DevCo projects, only 27 percent are in these countries. Similarly, 24 percent of DevCo-supported projects are in fragile countries. 3 DAC 1, 2, 3, 4 classifications are based on the OECD List of ODA Recipients. DAC 1 countries are “least developed countries” as listed in the first column of the DAC List of ODA Recipients. DAC 2 countries are “other low income countries” as listed in the second column of the DAC List. DAC 3 countries are the “lower middle income countries and territories” as listed in the third column of the DAC List. DAC 4 countries are the “upper middle income countries and territories as listed in the fourth column of the DAC List. Definitions taken from the PIDG 2012 Annual Report 4 Note, this was calculated by first calculating the target in dollar terms (the percent target multiplied by total dollars of PSI to be raised in poor countries), and calculating the total PSI predicted in the PIDG DevCo Database for projects closed 2009 to 2012. The difference is smaller by 82 percent. 5 This result takes into account the Central Java IPP project in the Philippines, a DAC 3 country. Predicted PSI raised from this project is US$3,500 million, which is five times higher than the average PSI raised for the other DevCo projects that closed from 2009 to 2012 (period evaluating target). Including the Central Java IPP in this PSI achievement calculation makes a large difference in the percent PSI achieved in DAC 1,2 countries. Page 3 of 6 Effectiveness and Impact DevCo is effective, and is making an impact. It has achieved most of its expected results. This conclusion was drawn from evaluating DevCo’s performance against targets since DevCo’s last mid-term review (that is, from 2009 to 2012). The case studies were used to verify the expected outputs and outcomes—that is, to compare the results reported in project results monitoring sheets to what is actually on the ground. Another level of analysis involved comparing actual to predicted results on DevCo’s nine operating projects. These are operational projects with actual data gathered from post-completion monitoring activity. On outputs, DevCo slightly missed its 2012 target for new mandates signed, signing up nine, one short of the ten mandates targeted. However all the mandates were in DAC 1, 2 countries and poorer states of India exceeding that target by 18 percent. The target for mandates in fragile countries was exceeded by 36 percentage points (target: 20%, achieved: 56%). The nine cases reviewed were all designed to benefit poor people, and to develop the capacity of public and private stakeholders. DevCo funds were used as agreed, and all the projects reviewed addressed specific market and government failures. DevCo did not meet its combined target (2009-2012) for successfully bid-out projects. Only 10 projects closed successfully over this time, compared to a target of 25. Was this target reasonable? DevCo’s success rate6 in project closings over the period was 60 percent7. This success rate is higher than the IFC non-DevCo projects closed during this period, which was just 42 percent. Notwithstanding, DevCo should still aim to close more projects successfully. This can be done by generating more mandates, and by addressing the factors that contribute to failure. On outcomes, DevCo outperformed its combined target (2009-2012) on increased availability of new or improved infrastructure services. DevCo mandates are now expected to benefit 15.5 million people, 50 percent more than the target. For the nine operating projects, the number of people reported to have benefited from the projects is more than double what was predicted (nine million people versus four million). DevCo outperformed its combined target (2009-2012) on increasing private sector investment (PSI). DevCo mandates are expected to deliver US$4,780 million in PSI, 71 percent higher than the target. Turning to the cases, four of the five that closed have achieved or are on track to achieving expected PSI. The nine operating projects are 47 percent below the predicted PSI. However, the difference is reasonable because these projects all closed within the last nine years, and the PSI is expected to be generated over the total life of the projects. DevCo also outperformed on PSI raised in fragile or post conflict countries by 17 percent. However, DevCo is below target on the percentage of PSI raised in DAC 1, 2 countries. From 2009 to 2012, DevCo was expected to raise 838 percent of total PSI in DAC 1, 2 countries. However, DevCo only succeeded in raising 9 percent9. The PSI raised by the Central Java IPP project in the Philippines (a DAC 3 country) is five times higher than the average PSI raised by other 6 The PIDG DevCo database reports 10 successful projects of 19 concluded (53 percent success rate) 7 12 successfully bid out mandates divided by total 20 concluded mandates. Note that this 12 successfully bid projects figure differs from that reported in the log frame for the same time period (10 successfully bid). This is an example of the types of data discrepancies that we recommend be resolved in the Recommendations Section 10 8 Target in 2009 was 75%, in 2010 – 2012, 85%. So the average for 2009-2012 is 83% of total PSI in DAC 1, 2 countries 9 Note: when Central Java IPP is excluded, DevCo succeeded in raising 33 percent of total PSI in DAC 1,2 countries Page 4 of 6 projects closed during this period. Excluding the Central Java IPP project from the PSI calculation still puts DevCo below the target, raising 33 percent of PSI in DAC 1, 2 countries. On fiscal impact, the DevCo mandates that closed between 2009 and 2012 are expected to deliver increased receipts from sales, or savings, to governments of US$810 million, 16 times the target of US$50 million10. Actual fiscal impact reported on the nine operating projects amounts to US$1,884 million, 13 percent over the predicted amount of US$1,673 million. There are no specific targets on economic or employment effects. Evidence from the five closed projects Castalia visited suggests direct job creation may be somewhat less than expected for these projects. Indirect employment effects cannot realistically be measured. Additional impacts and demonstration effects are strongly in evidence in three of the five closed projects analyzed in depth, in that additional similar projects are moving ahead using ideas and documents developed by DevCo. Evidence from the nine sample cases shows DevCo’s funding was necessary for the transactions, so causation and impact can be attributed to DevCo. Other key contributors to success include commitment from the government client and funding from other donors. Efficiency A review of the successful DevCo projects that started after 2008 showed that, on average, outputs cost about seven percent less than expected. Evidence from the nine projects visited shows that expected outputs are being delivered and at high quality. However, budgeting and financial reporting processes could be improved. Sustainability An assessment of the five cases that had closed suggest sustainability is likely, with the main risk being that governments may not deliver promised payments, or could renege on agreed tariff increases. Conclusions and Recommendations Based on the conclusions from this evaluation, there are four main recommendations to improve DevCo’s operations and performance: 1. On DevCo’s target setting and monitoring process, DevCo’s target for successfully bid-out projects should be more realistic, taking into account DevCo’s performance to date, the difficulty in doing projects in DevCo’s target countries, and how comparable facilities are performing. Also, PIDG should set DevCo’s target for percentage of PSI in DAC 1, 2 countries in dollar terms and reduce the expected PSI per project in these countries. 2. On operational improvements, DevCo should improve its data management processes so that data is accurately and consistently reported in both the PIDG DevCo database and IFC’s database. DevCo should improve budgeting with a database of past costs, and PIDG should allow the DevCo manager some discretion to approve some additional funds on DevCo transactions. These 10 PIDG DevCo Database. Total predicted upfront fees to government plus predicted avoided subsidies for projects that closed from 2009 to 2012 Page 5 of 6 changes will reduce project managers’ tendencies to overestimate the cost of the support needed. 3. To achieve more impacts, DevCo should: (a) Integrate upstream to generate more mandates. IFC teams should engage more with DAC 1, 2 governments in identifying project opportunities and removing initial roadblocks; (b) Integrate downstream to deliver more impact. DevCo project teams should consistently monitor transactions after implementation, and develop capacity in government agencies to manage projects effectively. Successful projects should be scaled up into PPP programs where possible. 4. To ensure project sustainability, IFC teams should maintain relationships until at least two years after services commence to ensure transactions translate into sustained service delivery. Copyright and Legal Disclaimer The material in this publication is copyrighted. IFC encourages the dissemination of the content for educational purposes. Content from this publication may be used freely without prior permission, provided that clear attribution is given to IFC and that content is not used for commercial purposes The findings, interpretations, views, and conclusions expressed herein are those of the authors and do not necessarily reflect the views of the Executive Directors of the International Finance Corporation or of the World Bank or the governments they represent. The views expressed in this publication are those of its authors and do not necessarily reflect the views of the wider institution. Some of the information used in this document may come from publicly available sources such as company websites and publications. The Lessons of Experience series does not represent a commitment by IFC to require projects it finances to take certain or all of the actions specified in this publication. Instead, any issues arising in an IFC-financed project will be evaluated and addressed in the context of the particular circumstances of the project. Page 6 of 6