CLR Review Independent Evaluation Group 1. CPS Data Country: Timor-Leste CPS Year: FY13 CPS Period: FY13 – FY18 CLR Period: FY13-FY19 Date of this review: November 13, 2019 2. Ratings CLR Rating IEG Rating Development Outcome: Moderately Satisfactory Moderately Satisfactory WBG Performance: Good Fair 3. Executive Summary i. This review of the Timor-Leste’s Completion and Learning Review (CLR) of the World Bank Group’s (WBG) Country Partnership Strategy (CPS) covers the original CPS period (FY13-FY17), and the Performance and Learning Review (PLR) of 2016. The PLR extended the original CPS period by one year to FY18 in order to synchronize the CPS strategy with the country’s political cycle. ii. Timor-Leste is a lower middle-income country, with an oil dependent economy. With oil reserves running low, the key challenges facing Timor-Leste are to achieve greater economic diversification and diminish reliance on public sector spending. At the beginning of the CPS period, the political environment was stable and oil prices high. The country was affected by a significant fall in oil prices that started in 2013, and political uncertainty adversely affected economic activity in 2017 and for most of 2018, as public expenditures fell by over one third. On the whole, growth was modest compared to East-Asia Pacific region peers, reflecting both the fall in oil prices and the political uncertainty towards the end of the program period. iii. The CPS was congruent with the Government’s 2011-30 Strategic Development Plan. The Bank Group-supported program had four engagement areas (or focus areas): (i) improving the management and delivery of services in education, health and nutrition, and social protection, (ii) building core infrastructure to connect communities to markets, (iii) supporting economic development for a non-oil economy and (iv) a cross cutting theme of strengthening institutions for quality of spending and inclusive service delivery. These focus areas addressed key development challenges, including improving service delivery and sustaining economic growth. At the PLR stage, several adjustments were introduced including dropping the cross-cutting theme and associated objectives and indicators. iv. The pre-existing portfolio amounted to $46 million, covering projects in education, transport, social development, health, education and governance. During the CPS period, new lending commitments amounted to $75.2 million, comprising two Additional Financing for the Road Climate Resilience Project. Four trust-funded activities for $27.1 million complemented the road resilience project and other sectors not covered by IDA/IBRD lending, including agriculture and nutrition. CLR Reviewed by: Peer Reviewed by: CLR Review Coordinator Juan José Fernández-Ansola Nils Fostvedt, Jeff Chelsky IEGEC Consultant IEGEC Consultant Manager, IEGEC Takatoshi Kamezawa Lourdes Pagaran Senior Evaluation Officer, IEGEC CLRR Coordinator, IEGEC For Official Use Only CLR Review 2 Independent Evaluation Group v. IEG rates the CPS development outcome as Moderately Satisfactory. Road conditions improved in the Dili-Ainaro road corridor. Collaboration between the World Bank and IFC contributed to attracting private investment in seaport infrastructure. Business registration was streamlined and the amount of time for trade transactions was reduced significantly with support from an IFC AS project. The World Bank supported the activities under the equivalency program in education, including staffing and establishing Community Learning Centers; but there is no available evidence indicating that the quality of the equivalency program has improved in terms of the number of learners completing the program. The nutrition of mothers of children under 2 years of age in the targeted poorest communities, as measured by an increase in consumption of iron- rich foods, was achieved. However, there was no progress in the proportion of children 0-6 months old who are exclusively breastfed. The Bank provided advice on ICT reform together with other donors that facilitated the approval of an ICT policy. In all, the program showed uneven results, with a number of achievements but some objectives and indicators not well supported by interventions, or overly broad objectives with project-level indicators and indicators not measuring the intended outcomes. vi. On balance, IEG rates WBG performance as Fair. The original program’s ambition was not matched by similarly ambitious interventions. Program results relied on a limited portfolio, and the broad scope of the ASA program did not take into account the absorptive and implementation capacity of the client. There were areas of the original program—health, social protection, airport infrastructure, land, cross-cutting area of institution building—where the program intended to achieve results with little engagement. At PLR stage, the WBG-supported program was adjusted, improving the links from WBG interventions to program outcomes. On the whole, however, the changes were insufficient to correct the original program’s weak design. Portfolio performance at exit was poor and the active portfolio showed mixed results. IEG’s review of closed projects during the CPS period noted design shortcomings as too ambitious, unfocused, broad, and complex, and indicated that the design did not take fully into account limited local capacity. Both IEG’s previous CLR review and Country Program Evaluation (2010) had raised the need to reduce the complexity of program and project design, but these lessons were not reflected in the design of this CPS. Other donors supported the program either through a multi-donor fund or through parallel financing. IFC complemented the work of IDA/IBRD through parallel work in infrastructure and economic development. However, the CLR noted conflicting advice on the Tibar Bay Port PPP. No Inspection Panel case was recorded during the CPS period. INT substantiated fraud by a firm that won a consultancy contract financed under a project in the roads sector. vii. IEG agrees with the CLR lessons: (i) a more realistic CPS program scope at the outset of the program would have been more appropriate to deliver results; (ii) given slow rate of skill absorption among local and permanent employees, it is critical for sustainability to adopt slower pace of project implementation to allow strengthening of government institutional capacity; and (iii) clarifying the objectives, establishing clear protocols and escalation mechanisms upfront would help minimize the challenge for IFC and World Bank to work together as a WBG team. viii. IEG adds the following lessons: • Program selectivity matters, especially in a weak capacity environment. In Timor-Leste, lack of selectivity led to an ambitious program that proved too demanding for an FCS country with a constrained administrative capacity and modest experience in implementing WBG-supported programs. In the end, the program could not deliver results as designed. In addition, lack of selectivity in the ASA program tested the limited absorptive capacity of the client country and did not advance local capacity. A more selective program that takes into account client capacity and provides implementation support is essential to achieve sustainable results. • A key lesson from this review, which is in line with CLR lessons, is the need to align program ambition with both WBG interventions and institutional capacity. Not heeding this lesson in Timor-Leste led to a program that could not deliver expected results in a number of objectives. • Simplicity in the results framework maximizes the development effectiveness of WBG assistance. In Timor-Leste, the results framework had too many objectives with limited For Official Use Only CLR Review 3 Independent Evaluation Group interventions, and the link of some indicators to supporting interventions was not well established. 4. Strategic Focus Relevance of the WBG Strategy: 1. Congruence with Country Context and Country Program. Timor-Leste is a Fragile and Conflict-Affected State (FCS) since 2006 that became independent in May 2002. It is a lower middle- income country with a GNI per capita of $2,694 (average 2013-17) in current dollars. Its institutional development is not commensurate to its level of income, and capacity to implement policies is quite limited. In many government institutions, foreign consultants have the unusual role of day-to-day policy implementation and administration. Annual GDP growth (non-oil GDP) averaged about 2½ percent during 2013-17, which is lower than East-Asia Pacific region as a whole. Political conditions remained stable until the July 2017 presidential elections. Towards the end of the CPS period (i.e., in 2017 and for most of 2018), political uncertainty due to the absence of a clear winner in the elections led to cuts by over one third in public spending that adversely affected Timor-Leste's economic activity. The political stalemate was resolved with new elections in May 2018. 2. Timor-Leste’s economy is oil-dependent. Until 2013 high oil prices were providing resources to fund public spending almost entirely from resources accumulated in the Petroleum Fund. Then oil prices fell by over 40 percent for the remainder of the program period. Against this background and with oil reserves running low, Timor-Leste faced three key challenges, to: (i) diversify its economy and have sustainable public finances, (iii) develop its institutions and capacity for policy implementation, and (c) foster the development of human capital. 3. The original Bank Group supported program had four focus areas: (i) improving the management and delivery of services in education, health and nutrition, and social protection, (ii) building core infrastructure to connect communities to markets, (iii) supporting economic development for the non-oil economy, and (iv) a cross-cutting area on strengthening institutions for quality of spending and inclusive service delivery. The focus areas addressed key challenges for the country and reflected focus areas of the Government’s 2011-30 Strategic Development Plan, particularly on investing in human capital, funding catalytic infrastructure, sustaining growth, and fostering institutional effectiveness. At the PLR stage, the focus areas were reduced to three to align with the changed environment, and implementation challenges, reflecting complex project design, delayed project effectiveness, and lack of implementation capacity. 4. Relevance of Design. The program’s ambition was not matched by the limited WBG interventions, given weak institutional capacity and the fragility of the country. The original CPS had 4 focus areas and 14 objectives, while the number of WBG interventions was quite small, and program results relied on the inherited portfolio of seven operations, and on other trust-funded operations that were fragmented. There were areas of the original program—health, social protection, airport infrastructure, land, cross-cutting area of institution building—where the program intended to achieve results with little engagement. During the CPS, new lending was even more limited in the form of two additional financing operations for the Road Climate Resilience Project, augmented by four trust-funded operations. IFC complemented the work of IDA/IBRD in infrastructure and economic development. Development partners supported WBG ASA—particularly in the education sector— through a multi-donor trust fund. At PLR stage, the WBG-supported program was adjusted by refocusing the program on fewer objectives and indicators. On the whole, however, the changes were insufficient to reverse the weak original design. Moreover, they involved dropping outcomes and indicators for some areas, such as strengthening institutions, which are key for Timor-Leste and where the WBG has a comparative advantage. Selectivity 5. The WBG-supported program lacked selectivity. The original CPS had four focus areas with 14 objectives and 26 indicators. At PLR stage, the program was trimmed to three focus areas, seven objectives and 15 indicators, in the context of limited WBG interventions and constrained institutional For Official Use Only CLR Review 4 Independent Evaluation Group capacity. The ASA program was too broad in scope—29 products covering many areas—which challenged the modest absorptive and implementation capacity of the country. IEG’s previous CLR review and Country Program Evaluation (2010) had both raised the need to reduce the complexity of program and project design, but these lessons did not percolate down to the preparation of this CPS. In the end, the lack of program selectivity proved too demanding for an FCS country with constrained administrative capacity and modest experience implementing WBG-supported programs. Alignment 6. The program was well aligned with WBG’s corporate twin goals of reducing extreme poverty and boosting shared prosperity. Focus Area I could conceivably contribute to the reduction of extreme poverty and increase shared prosperity by improving the management and delivery of services in education, health and nutrition, and social protection. Under Focus Area II, supporting for economic development could help generate employment growth and raise incomes. 5. Development Outcome Overview of Achievement by Objective: Focus Area I: Service Delivery 7. Focus Area I had two objectives: (i) improve the quality of the equivalency program, (a program to give a second chance to those who have missed out on an education), and (ii) improve infant and maternal nutrition practices in the poorest communities. Objective 1: Improve the quality of the equivalency program 8. This objective was supported by the Second Chance Education Project (FY11), the Education Sector Support Project (FY07) and its Additional Financing (FY10), the Planning and Financial Management Capacity Building Project (FY06), and the Education Sector Dialogue (FY14) non- lending technical assistance. The objective had three indicators: • The Ministry of Education is staffed with number of qualified and trained teachers, and pedagogic team: A total of 20 pedagogic staff and 139 teachers were recruited and trained (compared with target of 20 for pedagogic staff and 139 for teachers). Achieved • Number of Community Learning Centers (CLCs) established, equipped and functioning to implement an education equivalency program in line with the new regulatory framework: The Second Chance Education Project (FY11) helped establish and equip nine CLCs in selected districts. All CLCs are currently functioning (compared with a target of nine CLCs and a baseline of 0 in 2015). Achieved • Number of learners of the equivalency program: A total of 1,670 students benefitted from the equivalency program, of which 58 percent were female (compared with a target of 1,381 of which 55 percent female). Achieved. 9. The indicators are mostly process oriented, and could plausibly contribute to improving the quality of equivalency program. However, there is no evidence whether the quality of the equivalency program has improved. For instance, there is no information on the number of students completing the equivalency education. On balance, IEG rates Objective 1 as Partially Achieved. Objective 2: Improve infant and maternal nutrition practices in the poorest communities 10. This objective was supported by the trust-funded Community-Driven Nutrition Improvement Project (FY15), Health Sector Strategic Plan Support (FY08), the Financial Management Capacity Building Project (FY06), the Human Resources for Health technical assistance (FY15), and the Health Programmatic AAA (FY17). The objective had two indicators: For Official Use Only CLR Review 5 Independent Evaluation Group • Proportion of children 0-6 months of age who are exclusively breastfed increases from 71.3 percent (2015) to 75 percent (2017): 70 percent of children 0-6 months (a slight decline from baseline) were exclusively breastfed in 2017. Not Achieved • Proportion of mothers of children under 2 years of age who consumed iron-rich foods increases from 15.4 percent (2015) to 20 percent (2017): 37 percent of mothers of children under 2 years of age consumed iron rich foods by 2017. Achieved 11. The indicators are from the Community-Driven Nutrition Improvement Project, which covered 50 villages in the lowest living standards group. The WBG had mixed results in improving maternal nutrition practices in poor communities. It did not achieve the 75 percent program target for breastfeeding of children 0-6 months old (in fact there was a modest decline), but reached the target for the proportion of mothers of children under 2 years of age who consumed iron rich foods. The change in baseline and targets for the two indicators in 2016 (baseline and target lowered for breastfeeding, and increased for other nutrition) call into question the reliability of the indicators. On balance, IEG rates Objective 2 as Partially Achieved. 12. Focus Area I is rated Moderately Unsatisfactory. Both objectives were Partially Achieved. The World Bank supported program contributed to improving the quality of the equivalency program by fostering staffing at the Ministry of Education, establishing Community Learning Centers, and increasing the number of learners benefiting from the program. However, there is no available evidence indicating that the quality of the program has improved in terms of the number of learners completing the program. Moreover, the objective of improving infant and maternal nutrition was only partially achieved. The nutrition of mothers of children under 2 years of age as measured by an increase in consumption of iron-rich foods was achieved. However, there was no progress in the proportion of children 0-6 months old who are exclusively breastfed. Focus Area II: Building Core Infrastructure 13. Focus Area II had three objectives: (i) improve road conditions and sustainability of road investment, (ii) increased private sector investment in seaport infrastructure, and (iii) improve the regulatory framework and increased access to ICT. Objective 3: Improve road conditions and sustainability of road investment 14. This objective was supported by the Road Climate Resilience Project (FY11, and its Additional Financing (FY14 and FY17), the trust-funded Building Climate/Disaster Resilience Along the Dili- Ainaro and Linked Corridors (FY15), and the Planning and Financial Management Capacity Building Program (FY06). The objective had two indicators: • Percent of the Dili-Ainaro Road corridor that is in good or fair condition (target 75 percent in 2017 from a baseline of 10 percent in 2012): The Dili-Ainaro project (FY15) achieved 66 percent of the road corridor in good or fair condition by 2017. Mostly Achieved • Climate resilience of the Dili-Ainaro road corridor: percent of drainage on the road corridor in good condition (75 percent in 2017 from a baseline of 20 percent in 2012): About 69 percent of the drainage was assessed to be in good condition by 2017. Mostly Achieved 15. The objective has a broad scope referring to road conditions and road investment, while indicators refer to a specific project on a single road corridor. As a result, we cannot claim that overall road conditions improved. On balance, IEG rates objective 3 as Mostly Achieved. Objective 4: Increased private sector investment in seaport infrastructure 16. This objective was supported by Advisory Services from IFC—Tibar Bay Port—and ASA from the World Bank—Infrastructure Development and PPP (FY17). The objective had one indicator: • Facilitate US$100 million in new private investment for Tibar Bay Port: Tibar Bay Port is a new port that will replace Dili Port with a state-of-the-art facility developed as Timor-Leste’s first PPP. Tibar Bay Port is expected to increase cargo capacity, solve the growing congestion at For Official Use Only CLR Review 6 Independent Evaluation Group Dili Port, a serious economic bottleneck, and help connect Timor Leste to the world. The port will cost an estimated US$490m to construct over the life of the 30-year concession. IFC, in collaboration with the World Bank, helped complete the Tibar Port PPP transaction. After competitive bidding in 2016, the project sponsor (Bolloré Group) committed a total investment of US$360 million during the 30-year concession period, with US$150 million in the construction phase. The remainder of the cost (US$130m) is to be financed by the Government. The PPP project achieved financial closure in 2018 and port construction has started. Achieved 17. There was no prior private investment in the Tibar Bay Port, and the government counted on the PPP to develop the new port. IEG rates objective 4 as Achieved. Objective 5: Improve the regulatory framework for, and promote increased access to, ICT 18. This objective was supported by a Telecommunications TA (FY19), and had two indicators: • Increase mobile phone penetration (from 50 percent in 2012 to 80 percent of the population in 2017): The CLR reports that about 98 percent of the population had access to mobile networks, basic mobile telephony, and data services. Although IEG could not verify this information, mobile phone penetration reached 125 percent in 2017 according to the International Telecommunications Union (ITU). World Bank TA was helpful for the liberalization of the telecommunications market by providing advice on a National ICT Policy and E-Legislation. An ICT project was planned under the original CPS but was dropped at the PLR stage to focus on TA over the remainder of the CPS period. Achieved • Draft legal and regulatory frameworks for online transactions. ICT policy drafted with support from World Bank TA, and approved by the Council of Ministers in February 2017. Achieved 19. The latest telecommunications TA (FY19) provided advice on ICT Policy as an early deliverable completed in May 2017. Other donors, including Australia, also provided policy advice. The indicator on legal and regulatory frameworks is vague because it does not indicate the stage of drafting. IEG rates objective 5 as Mostly Achieved. 20. Focus Area II is rated Moderately Satisfactory. One objective was Achieved, and two Mostly Achieved. Collaboration between the World Bank and IFC contributed to attracting private investment in seaport infrastructure. While a specific Bank project helped improve the Dili-Ainaro road corridor, the road objective is broader in scope than the indicators that are based on the project. There is no indication in the project that it was intended to be catalytic and have an effect on overall road conditions. The World Bank provided TA on ICT policy during the CPS period. The new telecommunications framework allowed two new operators to enter the telecommunications market in 2012. Focus Area III: Economic Development 21. Focus Area III had two objectives: (i) improve sustainable agriculture productivity, and (ii) improve the business environment. Objective 6: Improve sustainable agriculture productivity 22. This objective was supported by a trust-funded Sustainable Agriculture Productivity Improvement Project (FY17), a Planning and Financial Management Capacity Building Program (FY06), and a trust-funded Institutional Reform and Transformation of the Ministry of Agriculture and Fisheries (MAF) (FY15), and had two indicators: • Prepare plans for 50 percent of selected communities with sustainable agriculture development (from a baseline of 0 in 2015): According to the CLR, watershed management plans and agriculture development plans have been prepared by most of the sucos (local communities) in two, out of four, participating watershed areas. IEG could not verify this information. Not Verified For Official Use Only CLR Review 7 Independent Evaluation Group • Number of farmers that adopt improved agricultural technology (target 1200 in 2018): The implementation of the project supporting this objective (Sustainable Agriculture Productivity Improvement Project) was delayed, and hence, the target was not achieved. Not Achieved 23. The objective is broad in scope while the indicators are project specific. Moreover, the indicators are input focused and neither of them measure productivity. IEG rates objective 6 as Not Achieved. Objective 7: Improve the business environment 24. This objective was supported by three IFC AS on: (i) Business Registration and Licensing Reform (FY10), (ii) Trade Logistics (FY14), and micro-finance Tuba Rai Metin (FY13) and Moris Rasik (FY12). IFC also provided lending to Tuba Rai Metin. It had three indicators: • Streamline business registration and licensing process (from 103 days in 2012 to 5 days in 2016): A one-stop-shop supported by WBG contributed to expedite business registration. Yet, it currently still takes 13 days to register a business according to Doing Business 2019. Mostly Achieved • Access to finance doubled, (from 13 percent of the population in 2012 to 26 percent in 2017): The 2018 Financial Inclusion Report by the central bank indicates that nine percent of the population have loan accounts, a decrease compared to baseline. IFC provided financial and advisory services (Tuba Rai Metin and Moris Rasik micro-finance institutions). The scope of the interventions was too narrow to move the needle at a national level. Not Achieved • Reduce the time for real trade transactions by 15 percent (from 16 days in 2012 to 13 days in 2018): A new automated and streamlined process for customs clearance reduced the number of days to 6 days on average for a real trade transaction. IFC provide advisory services on Trade Logistics. Achieved 25. Business registration was streamlined and the amount of time for trade transactions was reduced significantly with support from an IFC AS project. On balance, IEG rates objective 6 as Mostly Achieved. 26. Focus Area III is rated Moderately Unsatisfactory. Of the two objectives, one was Not Achieved and one Mostly Achieved. IEG could not verify CLR information about sustainable agriculture development plans, and there was no progress on the number of farmers that adopted improved agriculture technology. With IFC support, business registration was streamlined and the amount of time for trade transactions was reduced significantly. The indicator on population’s access to finance was not achieved and does not measure the stated objective. Overall Assessment and Rating 27. IEG rates the CPS development outcome as Moderately Satisfactory. Of the seven objectives, one was Achieved, three Mostly Achieved, two Partially Achieved, and one Not Achieved. Under Focus Area I, the World Bank helped improve the equivalency program, and increase the number of learners benefiting from the program. However, there are questions about the quality of the program in terms of the number of learners completing the program. The objective of improving infant and maternal nutrition was only partially achieved because there was no progress in the proportion of children 0-6 months old who are exclusively breastfed. Under Focus Area II, IFC, in collaboration with the World Bank, helped complete a PPP transaction to develop a new port that is expected to solve the growing congestion at Dili Port. The Bank helped improve the Dili-Ainaro road corridor, and contributed TA on ICT policy Mobile phone penetration increased significantly. Under Focus Area III, business registration was streamlined and the amount of time for trade transactions was reduced significantly with support from an IFC AS project. However, there was no progress in the population’s access to finance. For Official Use Only CLR Review 8 Independent Evaluation Group Objectives CLR Rating IEG Rating Focus Area I: Service Delivery Moderately Satisfactory Moderately Unsatisfactory Objective 1: Improve the quality of equivalency Achieved Partially Achieved program Objective 2: Improve infant and maternal nutrition Partially Achieved Partially Achieved practices in the poorest communities Focus Area II: Building Core Infrastructure Satisfactory Moderately Satisfactory Objective 3: Improve road conditions and Mostly Achieved Mostly Achieved sustainability of road investment Objective 4: Increase private sector investment in Achieved Achieved seaport infrastructure Objective 5: Improve regulatory framework and Achieved Mostly Achieved increase access to ICT Focus Area III: Economic Development Moderately Unsatisfactory Moderately Unsatisfactory Objective 6: Improve sustainable agriculture Partially Achieved Not Achieved productivity Objective 7: Improve business environment Mostly Achieved Mostly Achieved 6. WBG Performance Lending and Investments 28. At the beginning of the CPS period, outstanding lending commitments were $46 million consisting of seven Investment Project Financing (IPF) operations, including one Additional Financing. The portfolio covered a wide range of sectors including education, transport, social development, health, education and governance. During the CPS period, new lending commitments amounted to $75.2 million, comprising two Additional Financing for the Road Climate Resilience Project. The original plan for US$114 million included three projects that were dropped, in part reflecting a political stalemate that delayed the preparation of projects. During the CPS period, four trust-funded activities (TFs) were approved for a total of $27.1 million, complementing the road resilience project and other sectors not covered by IDA/IBRD operations, including agriculture and nutrition. 29. During the CPS period, the Timor-Leste portfolio performance at exit was worse than the EAP and Bank-wide averages. In terms of number of projects, IEG rated one out of five closed projects (20 percent) as Moderately Satisfactory or better, compared to the EAP (78.3 percent) and Bank-wide (72 percent). In terms of volume of commitments, Timor-Leste, with 29 percent rated Moderately Satisfactory or better, also fared worse than EAP (86.6 percent) and Bank-wide (83.3 percent). The risk to development outcome measured as percent of commitment volume at high or significant risk is also worse for Timor-Leste (53.5 percent) than EAP (44 percent) and Bank (47.3 percent) averages. Similarly, in terms of percent of number of projects at high or significant risk, Timor-Leste (60 percent) fared worse than both EAP (55 percent) and Bank-wide (57.3 percent). 30. ICR reviews of the five closed projects that exited during the CPS period concluded that poor performance was due to the following factors: too ambitious, unfocused, broad, and complex project design that did not take fully into account limited local capacity, and lack of mitigating measures during design or supervision to deal effectively with limited local capacity. In some projects, implementation improved when in-country Bank support took over supervision. Both Timor-Leste’s CPE (2010) and IEG’s last CLR review had referred to complex design features of Timor-Leste’s portfolio that seem to have persisted over time. 31. Of the active portfolio, the share of projects at risk (by number of projects) averaged 58 percent, higher than FCS countries in EAP (28 percent) and FCS countries as a whole (38 percent). The share of commitments at risk was lower in Timor-Leste (26.8 percent) than in EAP-FCS (30.6 percent) and FCS as a whole (40 percent). Self-ratings of ongoing projects was satisfactory for For Official Use Only CLR Review 9 Independent Evaluation Group the road climate resilience project that had two additional financing operations. The poor ratings of projects at exit call into question the realism of the self-rating of the ongoing operation. 32. During the CPS period (FY13-FY18), IFC made net commitments of US$1.5 million with no new net commitment in FY17 and FY18. The largest project was IFC’s $1 million loan to an existing financial sector client. During the review period, IEG did not validate any Expanded Project Supervision Reports (XPSRs) of IFC investment projects. There were no new MIGA projects or active projects from before the CPS period. Analytic and Advisory Activities and Services 33. During the CPS period, the Bank delivered a total of 29 ASA products, most of them (18) Technical Assistance (TA) pieces. Technical assistance covered a wide range of topics, and its broad scope and volume did not take into account the absorptive and implementation capacity of the client. An infrastructure development and PPP TA was delivered mostly to foreign consultants at the Ministry of Finance and resulted in little if any build-up of local capacity. The programmatic AAA in health faced similar problems. Five pieces were delivered over two years, which gave little time for the client to absorb and implement key policy recommendations. The completion report notes that there was no critical mass of officials able to absorb the key messages of World Bank advice, and as a result, local implementation capacity is not advancing. On the whole, the analytical and TA work suffered from the same pitfalls as the overall program: excessive ambition for a country with limited absorptive capacity or interest in the extensive advice that the World Bank prepared. 34. During the review period, IFC approved seven new AS projects amounting to $11.7 million of IFC funds. IFC acted as an investment advisor for the port PPP project and the project has attracted significant investment. During the review period, IEG validated one Project Completion Report (PCR) of an AS project and assigned Unsuccessful ratings to Development Effectiveness as the related airport PPP project did not materialize. Results Framework 35. The original results framework had significant shortcomings. First, the original program design had 14 objectives and 26 indicators, with some objectives and indicators not well supported by appropriate interventions, including the cross-cutting area of the program. Second, the scope of some indicators was narrower than the objectives they intended to measure (for example, objective 3). Third, some program targets were ambitious, but with limited supporting interventions. For example, the indicator on access to finance was ambitious or vaguely defined (doubling the percentage of the population) based on IFC supporting two micro-finance institutions. This intervention did not have scope of influence to affect the overall population. Fourth, the initial results framework lacked measurable indicators in some areas (e.g. agriculture) which made it difficult to assess progress. Fifth, the link of some indicators to supporting interventions was not well established (for example, objective 5 -increase in mobile access to ASA). Sixth, some indicators do not have baseline or target dates (for example, objective 4, indicator 1; objective 5, indicator 2; objective 7, indicators 1 and 2). 36. At the PLR stage, adjustments were made to the results framework to address design shortcomings and to align the program with implementation capacity by reducing the number of objectives from 14 to 7, and indicators from 26 to 15. CPS objectives were either dropped (health, social protection, airport infrastructure, land, and the cross-cutting area of institution building) or consolidated due to limited Bank Group interventions (for example, private sector development). While the PLR results framework became more realistic by improving the links between WBG activities and program outcomes, the adjustments were not sufficient to reverse the shortcomings of the original design. For example, some of the objectives (e.g. access to finance) maintained a national dimension although interventions were localized and small. Despite the need for clear results in areas such as education, the revised indicators were process oriented. Partnerships and Development Partner Coordination 37. The World Bank plays a significant role in Timor-Leste, along with other important bilateral donors such as Australia. Overall, the Bank Group collaborated with other donors either through a For Official Use Only CLR Review 10 Independent Evaluation Group multi-donor fund or through parallel financing. A multi-donor fund helped finance World Bank work in the education sector, statistical and policy formulation capacity, and on public expenditure analysis. The World Bank, with the European Union and the Millennium Challenge Corporation, supported the Ministry of Finance in conducting the 2018 PEFA (Public Expenditure and Financial Accountability). In road infrastructure projects, the World Bank shared common implementation units with the Asian Development Bank and the Japanese International Cooperation Agency. Safeguards and Fiduciary Issues 38. Five operations were closed and validated by IEG during the CPS, of which four triggered environmental and social safeguards policies in the education, social services, and health sectors. The CLR confirms compliance with the policies throughout the CPS review period, noting some implementation challenges which included complaints related to the use of safety equipment, delays in staffing, and expense tracking. In the lessons learned, the CLR further notes contractors’ noncompliance with the agreed safeguards implementation plans, and the need for constant capacity building as well as staff management. The ICRs and the ICRRs conclude that environmental and social risks were generally minor with negligible negative impacts in all operations. Compliance with the safeguard policies is rated satisfactory with the proper mitigation of the minor implementation issues that occurred. No inspection panel investigation was recorded during the CPS implementation period. 39. During FY13-17, INT reviewed four complaints and opened one case. INT substantiated fraud by a firm that won a consultancy contract financed under a project in the roads sector. Ownership and Flexibility 40. The WBG-supported program and the Government’s 2011-30 Strategic Development Plan were broadly aligned. The WBG carried out dialogue with the Government as well as roundtables with development partners during the preparation of the CPS to provide a broad-based perspective on development challenges facing the country. On this basis, program ownership was adequate, and cooperation between the World Bank Group and the Government fruitful. Such cooperation secured the country its first large Public-Private Partnership (PPP) for Tibar Bay Port, the largest investment in the country thus far outside of the oil and gas sector. Political conditions remained stable until the July 2017 elections, which were followed by a year-long political stalemate due to the absence of a clear winner. According to the CLR, the political stalemate delayed the preparation of projects, leading to the cancellation of some planned projects and difficulties in the implementation of other projects. The WBG demonstrated flexibility at the PLR stage, by recognizing implementation issues and program shortcomings, and taking measures to amend them by consolidating the CPS objectives and indicators. WBG Internal Cooperation 41. IFC complemented the work of IDA/IBRD. World Bank ASA on telecommunications was followed by an IFC telecoms investment. IFC and World Bank work facilitated large private sector investments in port infrastructure. At the same time, the CLR suggests that there was conflicting advice given by IFC and the World Bank to the authorities on the Tibar Bay Port PPP transaction, and, as part of its lessons, suggests ways to avoid such conflict in the future. Risk Identification and Mitigation 42. The CPS and PLR identified risks from (i) political developments and governance, (ii) macroeconomic instability, notably from external shocks, and (iii) from fiduciary issues. Mitigating measures included policy dialogue on macroeconomic and external developments, and collaboration with other development partners for joint project management and audit to mitigate fiduciary risks. The risks identified were appropriate. The mitigation measures were weak because they relied on monitoring and dialogue for the macro instability risk, and flexibility of the WBG engagement for political risk. The main risk that materialized was the political risk. The stalemate after the July 2017 elections delayed the pace of CPS implementation and made the policy dialogue more challenging. The World Bank’s engagement became more narrowly focused on road infrastructure and agriculture. For Official Use Only CLR Review 11 Independent Evaluation Group In other areas, the main work continues through ASA. Macro-fiscal sustainability remains an issue due to declining oil revenues and excessive government withdrawals from the oil fund. The Bank coordinated with the IMF to monitor macroeconomic risks, and the dialogue with the government in this area continues. Overall Assessment and Rating 43. On balance, IEG rates WBG performance as Fair. The original program addressed important issues for the country. However, the ambition of the program was not matched by supporting interventions. The volume of WBG interventions was small, and program results relied on the inherited portfolio of seven operations, and other trust-funded operations that were scattered in coverage. The broad scope of the ASA program did not take into account the absorptive and implementation capacity of the client. There were areas of the original program—health, social protection, airport infrastructure, land, cross-cutting area of institution building—where the program intended to achieve results with little engagement, and without the appropriate interventions or partnerships with other donors. At PLR stage, the WBG-supported program was adjusted acknowledging that some objectives had no corresponding interventions. For example, the outcomes and indicators of the cross-cutting area of institution building were dropped. Some objectives and indicators were not well supported by interventions, or there were overly broad objectives with project-level indicators and indicators not measuring the intended outcomes. The significant shortcomings of the original results framework were partially addressed at PLR stage when the number of objectives were reduced from 14 to 7, and indicators from 26 to 15. The PLR results framework thus became more realistic, improving the links of WBG activities (in the portfolio or expected to start implementation) to program outcomes. On the whole, however, the changes were insufficient to reverse the original weak design. 44. Portfolio performance at exit was worse than the EAP region and the overall World Bank, reflecting lack of realism in project design. IEG’s ICR reviews of the five projects that exited during the CPS period emphasize far too ambitious project agendas, and unfocused, broad, and complex project design that did not take fully into account limited local capacity despite similar lessons having been raised previously by IEG. The Bank Group collaborated with other donors either through a multi-donor fund or through parallel financing. IFC complemented the work of IDA/IBRD. 7. Assessment of CLR Completion Report 45. The CLR is clear and concise and provides a good discussion of the CPS achievements and WBG performance. The CLR could have provided more information on the impact of TA and economic sector work. In addition, it could have explained better why so many projects were dropped from the WBG program. It also would have been helpful if the CLR discussed the conflicting positions of IFC and World Bank regarding Tibar Bay Port PPP transaction. The CLR assessment period goes through FY19, which is not consistent with the IEG-OPCS shared approach. The extended CPS period was up to FY18 and should be the appropriate coverage of the CLR per the IEG-OPCS shared approach. 8. Findings and Lessons 46. IEG agrees with the CLR lessons: (i) a more realistic CPS program scope at the outset of the program would have been more appropriate to deliver results; (ii) given slow rate of skill absorption among local and permanent employees, it is critical for sustainability to adopt proper pace of project implementation to allow strengthening of government institutional capacity; and (iii) clarifying the objectives, establishing clear protocols and escalation mechanisms upfront would help minimize the challenge for IFC and World Bank to work together as a WBG team. 47. IEG adds the following lessons: • Program selectivity matters, especially in a weak capacity environment. In Timor-Leste, lack of selectivity led to an ambitious program that proved too demanding for an FCS country with a constrained administrative capacity and modest experience in implementing WBG-supported programs. In the end, the program could not deliver results as designed. In addition, lack of For Official Use Only CLR Review 12 Independent Evaluation Group selectivity in the ASA program tested the limited absorptive capacity of the client country and did not advance local capacity. A more selective program that takes into account client capacity and provides implementation support is essential to achieve sustainable results. • A key lesson from this review, which is in line with CLR lessons, is the need to align program ambition with both WBG interventions and institutional capacity. Not heeding this lesson in Timor-Leste led to a program that could not deliver results in a number of objectives. • Simplicity in the results framework maximizes the development effectiveness of WBG assistance. In Timor-Leste, the results framework had too many objectives with limited interventions, and the link of some indicators to supporting interventions was not well established. CLR Review Annexes Independent Evaluation Group 13 Annex Table 1: Summary of Achievements of CPS Objectives – Timor-Leste Annex Table 2: Timor-Leste Planned and Actual Lending, FY13-FY18 ($, millions) Annex Table 3: Advisory Services and Analytics for Timor-Leste, FY13-18 Annex Table 4: Timor-Leste Grants and Trust Funds Active in FY13-18 ($, millions) Annex Table 5: IEG Project Ratings for Timor-Leste, FY13-18 ($, millions) Annex Table 6: IEG Project Ratings for Timor-Leste and Comparators, FY13-18 Annex Table 7: Portfolio Status for Timor-Leste and Comparators, FY13-18 Annex Table 8: Total Net Disbursements of Official Development Assistance and Official Aid for Timor-Leste ($, millions) Annex Table 9: Economic and Social Indicators for Timor Leste, 2013-2017 Annex Table 10: List of IFC Long Term Investments in Timor-Leste ($, millions) Annex Table 11: List of IFC Advisory Services in Timor-Leste ($, millions) Annex Table 12: IFC net commitment activity in Timor-Leste, FY15 - FY18 ($, millions) Annex Table 13: List of MIGA Projects Active in Timor-Leste, FY13-18 ($, millions) CLR Review Annexes Independent Evaluation Group 14 Annex Table 1: Summary of Achievements of CPS Objectives – Timor-Leste CPS FY13-FY18: Focus Area I: Actual Results IEG Comments Service delivery 1. CPS Objective: Improved quality of equivalency program Indicator 1: Ministry is staffed The objective was supported by the At the PLR stage, the with Second Chance Education Project objective and indicator number of qualified and trained (P116520, FY11). were revised. teachers and pedagogic team Baseline: Teachers: none; The IEG ICRR: MU of P116520 reports Pedagogic staff: none (2015) that 97 Level I program teachers and 42 Target: Teachers 139; Level 2 equivalency program teachers Pedagogic staff: 20 (2018). were recruited and staffed as of December 2016. In addition, 20 national pedagogical staff members were also recruited and staffed. Achieved Indicator 2: Number of The objective was supported by the At the PLR stage, the community learning centers Second Chance Education Project indicator was revised. established, equipped and (P116520, FY11). functioning to implement an education equivalency program The IEG ICRR: MU of P116520 reports in line with the new regulatory that 9 community learning centers in 8 framework. municipalities were established, equipped, and functioning as of December 2016. Baseline: 0 (2015) Major Target: 9 (2018) Achieved Outcome Indicator 3: Number of learners The objective was supported by the While the targets of Measures of Second Chance Education Project three indicators were the equivalency program (P116520, FY11). achieved, these are output indicators that Baseline: 729 (2014) The ICR: MS of P116520 reports that could plausibly Target: 1381 (2017) o/w female there were 1,670 students who benefitted contribute to improved 55 percent from the equivalency program as of quality of equivalency December 2016, of which 969 were program. However, female (58%). there is no available evidence to indicate the Achieved achievement of the stated objective. 2. CPS Objective: Improved infant and maternal nutrition practices in the poorest communities Indicator 1: Proportion of The objective was supported by the The Health children 0-6 months of age who Community Driven Nutrition project Programmatic AAA are exclusively breastfed from (P145491, FY14), Health Sector Strategic supported the objective 71.3% (2015) to 75% (2017). Plan Support (P104794, FY08) and the by providing following ASAs: Health Programmatic information on ongoing AAA (P145528, FY16) and Human and future health sector Resources in Health (P146118, FY15). reforms (Completion Summary). P146118 The October 2018 ISR: MS of P145491 studied the labor reports that 70% of children <6 months of market among health age were exclusively breastfed. The workers including the survey that collected the information was assessment of skills, conducted in May 2018. The project area competence, and CLR Review Annexes Independent Evaluation Group 15 CPS FY13-FY18: Focus Area I: Actual Results IEG Comments Service delivery for the project 50 villages in the districts of motivation of doctors Baucau and Viqueque are in the lowest (Report). living standards group (P145491 Project Paper). Not Achieved Indicator 2: Proportion of The objective was supported by the mothers of children under 2 Community Driven Nutrition project years of age who consumed (P145491, FY14), Health Sector Strategic iron-rich foods from 15.4% Plan Support (P104794, FY08) and the (2015) to 20%. (2017) following ASAs: Health Programmatic AAA (P145528, FY16) and Human Resources in Health (P146118, FY15). The October 2018 ISR: MS of P145491 reports that 37% of mothers of children under 2 years of age consumed iron-rich foods as of August 2018. The project area for the project 50 villages in the districts of Baucau and Viqueque that fall in the lowest living standards group (P145491 Project Paper). Achieved CPS FY13-FY18: Focus Area II: Actual Results IEG Comments Building Core Infrastructure 3. CPS Objective: Improved road conditions and sustainability of road investments Indicator 1: % of the Dili-Ainaro The objective was supported by the At the PLR stage, the Road Corridor that is in good or Timor-Leste Road Climate Resilience indicator was revised fair condition Project (P125032, FY11) and Additional Financing (P130975, FY14; P252338, Baseline:10% (2012) FY17) and the Target: 75% (2017) Building Climate/Disaster Resilience Along the Dili-Ainaro and Linked Road Corridors project (P144818, FY15). Major The May 2019 ISR: S of P125032 reports Outcome that 72.6 km of the 110 km of total Measures classified roads of the Dili-Ainaro corridor (66%) were in good and fair condition as of May 2019. Mostly Achieved Indicator 2: Climate resilience of The objective was supported by the P144818 built the the Dili-Ainaro road corridor, as Timor-Leste Road Climate Resilience capacity of measured by % of drainage on Project (P125032, FY11) and Additional communities around the road corridor in good Financing (P130975, FY14; P252338, the Dili-Ainaro and condition FY17) and the linked road corridors in Building Climate/Disaster Resilience community- based Baseline: 20% (2012) Along the Dili-Ainaro and Linked Road disaster risk Target: 75% (2017) Corridors project (P144818, FY15). management (DRM) CLR Review Annexes Independent Evaluation Group 16 CPS FY13-FY18: Focus Area II: Actual Results IEG Comments Building Core Infrastructure and adaptation (ICR: The May 2019 ISR: S of P125032 reports S). This was achieved that 68.7% of the drainage in the Dili- through the training of officials and community Ainaro corridor were in good condition as of May 2019. members on DRM and the Mostly Achieved preparation/implementa tion of DRM plans at the suco level. 4. CPS Objective: Increased private sector investment in seaport infrastructure Indicator 1: US$100 million in The objective was supported by the ASA At the PLR stage, the new private investment for the on Infrastructure Development and PPP indicator was revised. Tibar Bay Port facilitated (P149600, FY17) and the IFC AS Tibar Bay Port (596787, FY17). The ASA P149600 supported the objective The CLR reports that the Tibar Bay PPP by providing TA to transaction has been completed which support he capacity of attracted $148 million in private the government’s PPP investments. unit to evaluate and assess infrastructure IFC reported that after a competitive investments and bidding in 2016, the project sponsor establish processes to (Bolloré Group) committed a total identify PPP investment of US$360 million during the opportunities 30-year concession period, with US$150 (Completion Report). in the construction phase. Achieved 5. CPS Objective: Improved regulatory framework and increases access to ICT Indicator 1: # of beneficiaries The objective was supported by the The ASA P150886 with Telecommunication TA (P150886, FY19). provided the increase mobile penetration from government advice on 50% in 2012 to 80% in 2017 The CLR reports that 98% of the a National ICT policy population have access to mobile and E-Legislation networks, basic mobile telephony, and (activity completion data services. The International summary). Although Telecommunications Union (ITU) reports the latest that mobile phone penetration was telecommunications TA 125 percent in 2017. (FY19) was completed outside the CPS period, the advice on Achieved ICT Policy was an early deliverable completed in May 2017. Indicator 2: Draft legal and The objective was supported by the At the PLR stage, the regulatory framework for online Telecommunication TA (P150886, FY19). indicator was revised. transactions prepared. In February 2017, the National Policy for The indicator does not the Information and Communications include a Technologies (ICT) was approved baseline/target year. CLR Review Annexes Independent Evaluation Group 17 CPS FY13-FY18: Focus Area II: Actual Results IEG Comments Building Core Infrastructure (Council of Ministries). The national policy includes provisions for the creation of laws on cybercrime, electronic transactions, and data protection. Achieved CPS FY13-FY18: Focus Area Actual Results III: IEG Comments Economic Development 6. CPS Objective: Improved sustainable agriculture productivity Indicator 1: % of selected The objective was supported by the At the PLR stage, the communities with sustainable Sustainable Agriculture Productivity indicator was revised. agricultural development plans Improvement (P155541, FY17) and the prepared following ASAs: Institutional Reform and The ASA P148698 Transformation of the Ministry of supported the Baseline: 0 (2015) Agriculture and Fishery (P148698, FY18). objective by improving Target: 50% (2018) the Ministry’s service The CLR reports that 2 out of four delivery model to agricultural development plans have been better align with and prepared. IEG could not verify this be more responsive to information. the needs of farmers and fishing The December 2018 ISR: MU of P155541 communities reports that 1 (out of 7) sustainable (November 2017 ISR: community agricultural development plans MS). were prepared and in use as of November 2018. The June 2019 ISR: MU reports that The target of P155541 as of May 2019, 2 development plans was changed from 4 to Major have now been prepared and in use. 7 management plans Outcome in 2018 (June 2018 Measures Not Verified ISR). Indicator 2: Number of farmers The objective was supported by the At the PLR stage, the adopted improved agricultural Sustainable Agriculture Productivity indicator was revised technology Improvement (P155541, FY17) and the from the original: following ASAs: Institutional Reform and % of farmers adopting Baseline: 0 Transformation of the Ministry of improved farming Target: 1200 (2018) Agriculture and Fishery (P148698, FY18). practices The December 2018 ISR: MU of P155541 The indicator does not reports that as of November 2018 there include a baseline were no clients/farmers that adopted year. improved agricultural technology being promoted by the project. Not Achieved 7. CPS Objective: Improved business environment Indicator 1: Business The objective was supported by the IFC The indicator does not registration AS Timor-Leste Business Registration and have a baseline/target and licensing process Licensing Reform Project (565307, FY18). year. streamlined and time reduced CLR Review Annexes Independent Evaluation Group 18 CPS FY13-FY18: Focus Area Actual Results III: IEG Comments Economic Development The PCR of 565307 reports that the Baseline: 103 days average time to register a business was Target: 5 days reduced to 5 days from 83 days, while licensing was reduced from 35 days to 5 days as of April 2017. However, the Doing Business 2019 reports shows that it takes 13 days to register a business down from the 94 days reported in the 2014 report (Doing Business 2019). Mostly Achieved Indicator 2: Access to finance The objective was supported by the IFC is AS Tuba Rai Metin (590227, FY17) and Doubled Moris Rasik AS (582127, FY18). Baseline: 13% of the population The CLR reports that the only 6% of the in FY12 population or 37,964 individuals owned a Target: 26% of the population in loan account (Banco de Timor-Leste). The FY17 2018 Financial Inclusion Report of the Banco Central de Timor-Leste reports that 9% of the population or 58,352 individuals have loan accounts while 413,2353 individuals (62% of the population) have deposit accounts (report). The report mentions that these figures were not adjusted for individuals holding loan or deposit accounts in multiple banks. The PCR of 590227 reports that 22,140 deposit accounts were opened and 30,113 microloans disbursed ($42 million) as of December 2016.The PCR of 582127 reports that 4,021 first time financial clients were reached as of June 2017. Not Achieved Indicator 3: Time to reduce The objective was supported by the IFC The indicator does not trade by 15% AS Timor-Leste Trade Logistic (600075, have a baseline/target ongoing). year. Baseline: 16 days Target: 13 days The FY19 Q4 supervision report of 600075 reports that it takes 6 days to comply with business regulations on trade. This represents a 37.5% decrease from the baseline of 16 days. Achieved CLR Review Annexes Independent Evaluation Group 19 Annex Table 2: Timor-Leste Planned and Actual Lending, FY13-FY18 ($, millions) Approved Proposed Approval Closing Proposed Proposed Project ID Project name IDA FY FY FY Amount Amount Amount Project Planned Under CPS/PLR FY13-17 CPF PLR TP: Road Climate Resilience Proj P130975 2013 2014 2022 40 25.0 - AF Timor-Leste Coffee Sector Dropped 2014 7 Development TP Telecomms and ICT Develop Dropped 2014 7 Project 2016- Dropped Branch Roads 2018 2016- Dropped Tibar Bay Port 2018 Total Planned 54 0 25 Approved Approval Closing Proposed Proposed Unplanned Projects during the CPS Period IDA FY FY Amount Amount Amount P252338 Road Climate Resilience Project 2017 2022 -- 35.2 Total Unplanned -- 35.2 83.5 Approved On-going Projects during the CPS/PLR Approval Closing IDA Period FY FY Amount TL Second Chance Education P116520 2011 2017 5.0 Project P125032* Timor Leste Road Climate 2011 2022 20.0 * Resilience Proj P120890 TP-ESSP Add'l Financing 2010 2013 5.0 P106220 TP - Youth Development Project 2009 2013 2.1 TP-Health Sector Strategic Plan P104794 2008 2015 1.0 Support P095873* TP-Education Sector Support 2007 2013 6.0 * Planning & Fin Mgt Capacity P092484 2006 2014 7.0 Building Total On-going 46 Source: Timor-Leste CPS and PLR, WB Business Intelligence Table 2a.1, 2a.4 and 2a.7 as of 8/21/19 *LIR: Latest internal rating. MU: Moderately Unsatisfactory. MS: Moderately Satisfactory. S: Satisfactory. HS: Highly Satisfactory. ** Parent Project *** Rating from Parent Project CLR Review Annexes Independent Evaluation Group 20 Annex Table 3: Advisory Services and Analytics for Timor-Leste, FY13-18 Product Proj ID Project Name FY Practice Line P150114 Timor-Leste Economic Management and Governance 2018 AA MTI P163648 Timor-Leste Water Sector Assessment and Roadmap 2018 AA WAT P165266 Timor-Leste economic monitoring and reporting 2018 AA MTI P165836 Timor-Leste Public Expenditure Analysis 2018 AA MTI P132567 Land and Resource Governance in Timor-Leste 2017 AA GOV P145528 Timor-Leste Health Programmatic AAA 2017 PA HNP P149600 Infrastructure Development and PPP 2017 TA PPP P160298 Tibar Bay Industrial Park Technical Assistance 2017 TA FCI P117303 Social Accountability in Participatory Transfer and Grant Programs 2016 TA GOV P148080 Timor 2014 Living Standards Survey 2016 TA POV Programa Nasional Dezenvolvimentu Suku (PNDS) Research and P148808 2016 IE N/A Evaluation Program P149406 Timor-Leste Debt Management Reform Plan 2016 TA MTI P150407 Oecusse Economic and Trade potential 2016 EW MTI P151108 Medium Term Expenditure Pressure 2016 TA HNP Timor Leste: NLTA on Strengthening Country Systems for Service P151167 2016 TA GOV Delivery Timor Leste - Customary Systems of Land Management and Rural P118554 2015 EW GOV Dev. P130056 Timor-Leste: Social Protection Administration TA 2015 TA SPL P133265 Building Disaster/Climate Resilience in Timor-Leste 2015 TA URS P133282 Timor Leste Infra spending analysis 2015 EW MTI P146116 Health Equity & Financial Protection 2015 EW HNP P146117 Health Resource Tracking Survey 2015 TA HNP P146118 Human Resources for Health 2015 TA HNP P146119 Institutional Reform of SAMES 2015 TA HNP P147353 Dialogue and TA for improved social protection policy 2015 TA SPL P126300 Timor Leste Social Protection AAA 2014 TA SPL P127404 Responding to Low Food Prod & High Price 2014 TA AGR P130713 Road for Cultural Heritage 2014 TA URS P132187 Education Sector Dialog 2014 TA EDU P133459 Timor Leste 2011/12 Poverty Profile 2013 TA POV Source: WB AO and Standard Reports, 8/22/19 CLR Review Annexes Independent Evaluation Group 21 Annex Table 4: Timor-Leste Grants and Trust Funds Active in FY13-18 ($, millions) Project Approval Closing Approved Project name TF ID ID FY FY Amount Sustainable Agriculture Productivity Improvement P155541 TF A2869 2017 2023 21.0 Project Building Climate/Disaster Resilience Along the Dili- P144818 TF 18187 2015 2019 2.7 Ainaro and Linked Road Corridors in Timor-Leste P148698 Institutional Reform and Transformation of MAF TF 16487 2015 2018 0.5 P145491 Community Driven Nutrition Improvement TF 17708 2015 2019 2.9 P125443 GPE Management Strengthening Project TF 12495 2012 2016 2.8 P125784 Timor Leste Social Protection Administration Project TF 99833 2012 2015 1.5 Building the capacities of procurement institutions to P125360 TF 99233 2011 2014 0.1 achieve development outcomes P104794 Health Sector Strategic Plan Support Project TF 91653 2008 2015 34.2 P095873 Education Sector Support TF 90564 2008 2014 10.2 Planning and Financial Management Capacity Building P092484 TF 93959 2009 2014 25.1 Program Total 100.9 Source: Client Connection as of 509/2019 ** IEG Validates RETF that are 5M and above Annex Table 5: IEG Project Ratings for Timor-Leste, FY13-18 ($, millions) Exit Total Proj ID Project name IEG Outcome IEG Risk to DO FY Evaluated MODERATELY 2013 P095873 TP-Education Sector Support 10.7 SIGNIFICANT UNSATISFACTORY TP - Youth Development MODERATELY 2013 P106220 2.0 SIGNIFICANT Project UNSATISFACTORY Planning & Fin Mgt Capacity MODERATELY NEGLIGIBLE TO 2014 P092484 7.4 Building SATISFACTORY LOW TP-Health Sector Strategic MODERATELY 2015 P104794 1.0 SIGNIFICANT Plan Support UNSATISFACTORY TL Second Chance Education MODERATELY 2017 P116520 4.5 MODERATE Project UNSATISFACTORY Total 25.6 Source: AO Key IEG Ratings as of 08/23/2019 Annex Table 6: IEG Project Ratings for Timor-Leste and Comparators, FY13-18 Total Total RDO % RDO % Outcome Outcome Region Evaluated Evaluated Moderate or Lower Moderate or Lower % Sat ($) % Sat (No) ($M) (No) Sat ($) Sat (No) Timor-Leste 25.6 5.0 29.0 20.0 46.5 40.0 East Asia 23,330.7 252.0 87.6 79.0 56.0 54.0 Pacific World 131,961.0 1,500.0 83.8 74 48.2 41.9 Source: WB AO as of 08/22/2019; *IEG Calculation CLR Review Annexes Independent Evaluation Group 22 Annex Table 7: Portfolio Status for Timor-Leste and Comparators, FY13-18 Fiscal year 2013 2014 2015 2016 2017 Ave FY13-17 Timor-Leste # Proj 6 6 4 4 4 5 # Proj At Risk 5 5 2 1 1 3 % Proj At Risk 83 83 50 25 25 58 Net Comm Amt ($M) 57.92 90.80 70.82 70.55 121.76 82.4 Comm At Risk ($M) 26.0 68.8 5.0 2.7 2.7 21 % Commit at Risk 44.89 75.78 7.06 3.83 2.22 26.8 EAP # Proj 351 354 344 337 359 349 # Proj At Risk 66 65 70 56 62 64 % Proj At Risk 19 18 20 17 17 18 Net Comm Amt ($M) 30,542.3 31,852.5 32,386.0 33,346.1 35,863.3 32,798 Comm At Risk ($M) 5,089.2 5,270.3 6,412.3 4,776.1 5,405.5 5,391 % Commit at Risk 16.7 16.5 19.8 14.3 15.1 16.5 EAP-FCS # Proj 21 25 34 34 44 32 # Proj At Risk 9 8 7 7 13 9 % Proj At Risk 43 32 21 21 30 28 Net Comm Amt ($M) 355.6 842.9 1,618.9 2,023.4 2,889.8 1,546 Comm At Risk ($M) 114.5 203.0 566.7 348.6 1,285.8 504 % Commit at Risk 32.2 24.1 35.0 17.2 44.5 30.6 FCS* # Proj 202 224 225 221 236 222 # Proj At Risk 77 80 88 86 94 85 % Proj At Risk 38 36 39 39 40 38 Net Comm Amt ($M) 9,725.8 10,752.9 11,996.4 14,071.6 17,776.2 12,865 Comm At Risk ($M) 4,116.9 4,009.2 4,690.0 5,396.4 7,734.4 5,189 % Commit at Risk 42.3 37.3 39.1 38.3 43.5 40.1 World # Proj 1,964 2,048 2,022 1,975 2,071 2,016 # Proj At Risk 414 412 444 422 449 428 % Proj At Risk 21 20 22 21 22 21 Net Comm Amt ($M) 176,202.6 192,610.1 201,045.2 220,331.5 224,420.1 202,922 Comm At Risk ($M) 40,805.6 40,933.5 45,987.7 44,244.9 52,549.1 44,904 % Commit at Risk 23.2 21.3 22.9 20.1 23.4 22.2 Source: WB BI as of 8/21/19 Agreement type: IBRD/IDA Only CLR Review Annexes Independent Evaluation Group 23 Annex Table 8: Total Net Disbursements of Official Development Assistance and Official Aid for Timor-Leste ($, millions) Development Partners 2013 2014 2015 2016 2017 All Donors, Total 258.57 249.54 212.29 223.55 231.96 DAC Countries, Total 199.19 183.36 148.64 160.94 163.53 Australia 108.57 89.64 62.98 56.78 69.06 Austria 0.02 .. .. .. .. Canada 0.3 0.52 0.3 0.3 0.13 Denmark 0.17 .. .. .. .. Finland 0.73 0.83 0.43 0.1 0.24 France 0.25 0.56 0.49 0.31 0.22 Germany 7.88 7.89 7.29 7.63 6.55 Hungary .. .. .. .. 0 Ireland .. 0.14 .. 0.06 .. Italy 1.15 0.02 0.07 0.04 0.01 Japan 22.17 19.17 19.26 36.71 24.13 Korea 3.6 4.02 9.01 12.06 11.77 Netherlands .. .. .. .. 0.24 New Zealand 9.19 10.8 14.91 10.77 11.49 Norway 5.56 4.51 2.92 1.49 0.65 Poland 0.01 0.01 0.01 .. .. Portugal 17.33 17.68 12.97 14.58 15.44 Spain 1.54 0 .. .. .. Sweden 0.87 0.34 0.14 0.23 0.4 Switzerland .. .. .. .. 0.04 United Kingdom 0.11 0.07 0.1 0.02 0.3 United States 19.74 27.15 17.76 19.86 22.86 Multilaterals, Total 59.3 65.9 63.42 62.55 68.27 EU Institutions 19.66 22.69 11.5 20.28 27.86 Regional Development Banks, Total 19.03 18.25 25.24 12.6 12.8 Asian Development Bank, Total 19.03 18.25 25.24 12.6 12.8 Asian Development Bank [AsDB] 19.03 18.25 25.24 12.6 12.8 United Nations, Total 9.26 10.51 8.12 5.84 4.76 Food and Agriculture Organisation [FAO] 0.17 .. .. .. .. IFAD 0.56 2.37 0.24 .. .. International Labour Organisation [ILO] 0.93 0.62 1.18 0.56 0.66 UNDP 1.87 2.28 1.06 1.14 0.67 UNFPA 1.84 2.04 1.74 1.48 1.07 UNICEF 1.5 1.21 1.44 1.18 1.15 WFP 1.55 0.86 0.12 0.04 0.06 CLR Review Annexes Independent Evaluation Group 24 Development Partners 2013 2014 2015 2016 2017 World Health Organisation [WHO] 0.84 1.13 2.34 1.43 1.16 World Bank Group, Total 4.2 4.96 11.17 14.4 10.37 World Bank, Total 4.2 4.96 11.17 14.4 10.37 International Development Association [IDA] 4.2 4.96 11.17 14.4 10.37 Other Multilateral, Total 7.14 9.48 7.4 9.43 12.47 Global Alliance for Vaccines and Immunization [GAVI] 0.29 0.95 0.35 0.09 2.61 Global Environment Facility [GEF] 1.32 2.77 3.56 3.73 4.33 Global Fund 5.53 5.77 3.49 5.62 5.53 Non-DAC Countries, Total 0.08 0.28 0.23 0.06 0.16 Thailand 0.07 0.28 0.22 0.06 0.16 United Arab Emirates 0.01 .. 0.01 .. .. Private Donors, Total 0.1 0.01 0.01 0.01 0.01 Bill & Melinda Gates Foundation 0.1 0.01 0.01 0.01 0.01 Source: OECD Stat. DAC2a as of 03/11/2019 Annexes CLR Review 25 Independent Evaluation Group Annex Table 9: Economic and Social Indicators for Timor Leste, 2013-2017 Timor- EAP World Leste Series Name 2013 2014 2015 2016 2017 Average 2013-2017 Growth and Inflation GDP growth (annual %) (11.0) (26.0) 20.9 0.8 (8.0) -4.7 6.5 2.8 GDP per capita growth (annual %) (13.1) (27.8) 18.1 (1.4) (10.0) -6.8 5.8 1.6 GNI per capita, PPP (current international $) 6,620.0 5,460.0 7,120.0 7,110.0 6,330.0 6,528.0 13,550.2 15,738.3 GNI per capita, Atlas method (current US$) 3,540.0 2,870.0 2,980.0 2,290.0 1,790.0 2,694.0 6,660.2 10,617.1 Inflation, consumer prices (annual %) 11.1 0.7 0.6 (1.3) 0.6 2.3 2.2 2.0 Composition of GDP (%) Agriculture, value added (% of GDP) 5.2 7.4 8.9 11.4 .. 8.2 9.5 3.6 Industry, value added (% of GDP) 79.8 69.8 56.9 44.8 .. 62.8 54.8 26.2 Services, value added (% of GDP) 15.2 22.7 32.7 43.3 .. 28.5 34.8 64.3 Gross fixed capital formation (% of GDP) 11.0 15.7 18.4 25.2 .. 17.6 20.4 23.4 External Accounts Exports of goods and services (% of GDP) 94.0 96.3 74.9 57.8 .. 80.8 26.1 31.1 Imports of goods and services (% of GDP) 36.7 59.5 54.4 60.2 .. 52.7 23.8 30.1 Current account balance (% of GDP) 42.3 27.3 7.2 (21.1) (11.5) 8.9 .. External debt stocks (% of GNI) 2.0 2.4 3.2 1.3 1.9 2.2 .. .. Total debt service (% of GNI) 0.0 0.0 0.0 0.0 0.0 0.0 1.9 Total reserves in months of imports 6.8 3.1 3.9 2.8 5.6 4.4 15.9 13.2 Fiscal Accounts /1 General government revenue (% of GDP) 20.2 26.3 33.2 36.8 30.6 29.4 .. General government total expenditure (% of GDP) 23.7 39.7 50.2 71.8 50.0 47.1 .. .. General government net lending/borrowing (% of GDP) 41.7 23.3 3.7 (22.1) (9.4) 7.4 .. .. General government gross debt (% of GDP) 0.1 0.5 1.5 3.1 3.8 1.8 .. .. Annexes CLR Review 26 Independent Evaluation Group Timor- EAP World Leste Series Name 2013 2014 2015 2016 2017 Average 2013-2017 Health Life expectancy at birth, total (years) 68.0 68.3 68.6 68.9 .. 68.4 74.4 71.8 Immunization, DPT (% of children ages 12-23 months) 82.0 77.0 76.0 76.0 76.0 77.4 92.7 84.9 People using safely managed sanitation services (% of pop) .. .. .. .. .. 58.8 38.5 People using at least basic drinking water services (% of pop) 67.4 68.8 70.2 .. .. 68.8 93.0 88.0 Mortality rate, infant (per 1,000 live births) 46.5 44.9 43.6 42.2 40.8 43.6 14.7 31.3 Education School enrollment, preprimary (% gross) 125.7 121.6 116.4 109.5 100.6 114.7 102.9 103.6 School enrollment, primary (% gross) 11.4 13.9 16.0 16.5 17.9 15.1 76.1 47.6 School enrollment, secondary (% gross) 59.7 62.5 68.0 74.3 79.3 68.8 86.2 76.3 School enrollment, tertiary (% gross) .. .. .. .. .. .. 41.0 36.2 Population population, total 1,184,366.0 1,212,814.0 1,240,977.0 1,268,671.0 1,296,311.0 1,240,627.8 2,020,624,749.0 7,357,707,460.4 population growth (annual %) 2.4 2.4 2.3 2.2 2.2 2.3 0.7 1.2 Urban population (% of total) 28.8 29.1 29.5 29.9 30.2 29.5 53.1 53.9 Rural population (% of total pop) 71.2 70.9 70.5 70.2 69.8 70.5 46.9 46.1 Poverty Poverty headcount ratio at $1.90 a day (2011 PPP) (% of pop) .. 30.3 .. .. .. .. Poverty headcount ratio at national poverty lines (% of pop) .. 41.8 .. .. .. .. .. .. Rural poverty headcount ratio at national poverty lines (% of .. 47.1 .. .. .. .. .. .. rural pop) Urban poverty headcount ratio at national poverty lines (% of .. 28.3 .. .. .. .. .. .. urban pop) GINI index (World Bank estimate) .. 28.7 .. .. .. .. .. .. Source: WB World Development Indicators Data Bank 03/13/19 *International Monetary Fund, World Economic Outlook Database, October 2018 CLR Review Annexes Independent Evaluation Group 27 Annex Table 10: List of IFC Long Term Investments in Timor-Leste ($, millions) Investments Committed in FY13-18 Project Cmt Project Project Net Net Primary Sector Name Net Comm ID FY Status Size Loan Equity 33643 2013 Closed TRM Microfinance 500.0 500.0 - 500.0 34884 2016 Active TRM Loan II 1,000.0 1,000.0 - 1,000.0 Sub-Total 1,500.0 1,500.0 - 1,500.0 Source: IFC-MIS Extract as of 1/31/19 Annex Table 11: List of IFC Advisory Services in Timor-Leste ($, millions) Advisory Services Approved in FY13-18 Impl Impl Primary Project Project Name Start End Project Status Business Total Funds ID FY FY Line 602611 Pipeline review 2018 2018 CLOSED CPC 0.03 Timor Leste Tibar Bay Investment Plan 600892 (TBIP) - Spatial, SME, Skills and GVC 2017 2018 TERMINATED TAC 1.08 Cluster Development 600552 Timor-Leste Health PPP 2016 2019 ACTIVE CPC 1.18 600075 Timor-Leste Trade Logistic 2014 2019 ACTIVE EFI 2.53 590227 Tuba Rei Metin AS Project 2013 2017 CLOSED FIG 1.60 591387 Dili Airport PPP 2013 2016 CLOSED CPC 1.46 596787 Tibar Bay Port PPP 2013 2017 ACTIVE CPC 3.90 Sub-Total 11.78 Advisory Services Approved pre-FY15 but active during FY13-18 Impl Impl Primary Project Project Name Start End Project Status Business Total Funds ID FY FY Line 582127 Moris Rasik 2012 2016 CLOSED FIG 1.39 583908 Timor-Leste Public Private Partnerships 2012 2013 TERMINATED CPC 1.71 Timor-Leste Business Registration and 565307 2010 2017 CLOSED EFI 2.33 Licensing Reform Project 557728 Timor Leste Public-Private Dialogue 2008 2013 CLOSED IC 1.99 Sub-Total 7.4 TOTAL 19.2 Source: IFC AS Portal Data as of 02/28/19 CLR Review Annexes Independent Evaluation Group 28 Annex Table 12: IFC net commitment activity in Timor-Leste, FY15 - FY18 ($, millions) 2013 2014 2015 2016 2017 2018 Total Long-term Investment Commitment Financial Markets 0.5 1.0 Total IFC Long Term Investment Commitment 0.5 1.0 - - 1.5 Short-term Finance/Trade Finance / Average Outstanding Balance (GTFP) Source: IFC MIS as of 03/12/19 Note: IFC began reporting average outstanding short-term commitments (not total commitments) in FY15 and no longer aggregates short-term commitments with long-term commitments. IEG uses net commitment number for IFC's long-term investment. For trade finance guarantees under GTFP, average commitment numbers have been used. Annex Table 13: List of MIGA Projects Active in Timor-Leste, FY13-18 ($, millions) Max Gross Contract Enterprise 2013 2014 2015 Sector Issuance No Active Projects Source: MIGA w/ Project Briefs 5/9/19