Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Sust. Growth and Income Convergence DPL(P149609) Report Number : ICRR0021301 1. Project Data Country Practice Area(Lead) Colombia Macroeconomics, Trade and Investment Programmatic DPL Planned Operations: 2 Approved Operations: 2 Operation ID Operation Name P149609 Sust. Growth and Income Convergence DPL L/C/TF Number(s) Closing Date (Original) Total Financing (USD) IBRD-84540 30-Jun-2017 700,000,000.00 Bank Approval Date Closing Date (Actual) 12-Dec-2014 30-Jun-2017 IBRD/IDA (USD) Co-financing (USD) Original Commitment 700,000,000.00 0.00 Revised Commitment 700,000,000.00 0.00 Actual 700,000,000.00 0.00 Prepared by Reviewed by ICR Review Coordinator Group Paul Holden Robert Mark Lacey Malathi S. Jayawickrama IEGEC (Unit 1) PHPROJECTDATATBL Operation ID Operation Name P154821 Colombia Growth and Convergence DPL 2 ( P154821 ) Page 1 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Sust. Growth and Income Convergence DPL(P149609) L/C/TF Number(s) Closing Date (Original) Total Financing (USD) IBRD-84540,IBRD-85580 30-Jun-2017 696,107,840.00 Bank Approval Date Closing Date (Actual) 08-Dec-2015 30-Jun-2017 IBRD/IDA (USD) Co-financing (USD) Original Commitment 700,000,000.00 0.00 Revised Commitment 700,000,000.00 0.00 Actual 696,107,840.00 0.00 2. Program Objectives and Policy Areas a. Objectives Original Objective: The Program Development Objective (PDO) for the first Development Policy Loan (DPO) as stated in the Loan Agreement dated December 15, 2014 (p. 11) was: (i) Improve the access to financial markets for road infrastructure investments and improve access to finance for firms; (ii) improve training, build productive skills and improve their allocation; and (iii) strengthen regulations that affect innovation and business efficiency, as means to support sustained growth and income convergence. 　 The PDO for the second DPO in the Loan Agreement (LA) dated April 15, 2016 modified the original PDO (LA p. 13) to (i) foster solutions to develop infrastructure financing and increase access to finance for firms; (ii) build productive skills and improve their allocation; and (iii) strengthen regulations that affect innovation and business efficiency. However, the substance of the PDO remained essentially unchanged. b. Pillars/Policy Areas There were three pillars: 　 Pillar 1 focused on fostering solutions to promote infrastructure financing and improving access to finance for firms. The Country Partnership Strategy 2012-2016 (CPS1, p. 23) had identified the infrastructure gap and access to finance by firms as key bottlenecks. These were to be addressed by allocating more financing to infrastructure through a concession program; increasing the registration of security interests in the collateral registry following on from a reform of the secured transactions framework; and enhancing capital market development by increasing the number of issuers in the alternative securities market. 　 Page 2 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Sust. Growth and Income Convergence DPL(P149609) Pillar 2 focused on upgrading the quality and allocation of production and training programs in the labor market through better job placement, retraining the unemployed and providing professional training through firms. Weak training and allocation of productive skills was identified in CPS1 as an important constraint to higher productivity, which pillar 2 sought to address. 　 Pillar 3 focused on promoting innovation and improving the regulatory environment by expanding the number of firms utilizing incentives for science, technology and innovation and utilizing regulatory evaluations to reduce the number of regulations and upgrade the regulatory environment. CPS1 (p. 26) identified that the utilization of incentives for innovation was highly concentrated among a small number of large firms. In addition, the regulatory environment for firms was adversely affecting productivity and needed rationalizing. The ICR (p. 8) describes the framework for evaluating existing regulations and the process for introducing new ones. c. Comments on Program Cost, Financing, and Dates The first DPO (P149609) was approved on December 12, 2014 in the amount of US$700 million. It became effective on December 16, 2014. It closed on schedule on June 30, 2017. The actual amount disbursed was US$700 million. 　 The second DPO (P154821) was approved on December 8, 2015 in the amount of US$700 million. It became effective on May 16, 2016. It closed on schedule on June 30, 2017. The actual amount disbursed was US$696.11 million, due to the change in the exchange rate between the US$ and the Euro. . 3. Relevance of Objectives & Design a. Relevance of Objectives The objectives were based on the National Development Plan 2014-2018 and were consistent with the World Bank Group’s Country Partnership Strategy (CPS) for 2012-2016. Two of the strategic areas of the National Development Plan were "Equality of Opportunity for Social Prosperity" and "Sustainable Growth and Competitiveness" (CPS p. 6), whereas the CPS proposed three strategic themes, of which two were Expanding Opportunities for Social Prosperity and Inclusive Growth with Enhanced Productivity (p. 12). Among the desired outcomes, the CPS specified: (i) enhancing skills for shared growth, including increasing education quality and skills formation and the improvement of technical and institutional capacity to monitor and evaluate education policies and programs; (ii) inclusive growth and enhanced productivity, through Page 3 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Sust. Growth and Income Convergence DPL(P149609) human capital development, science, technology and innovation, improved business and property registration, and higher financial inclusion. 　 The objectives were relevant to country circumstances at appraisal. Slowing external demand, falling commodity prices and lower foreign direct investment, were beginning to affect the Colombian economy adversely. Structural bottlenecks included limited access to finance for firms and infrastructure projects, a shortage of skills and human capital that contributed to low productivity, a weak financial system, and onerous regulations that raised the costs of doing business and hampered trade. 　 The objectives remained relevant at closure. The government had requested World Bank support for implementing its reform agenda (Country Partnership Framework, 2016-2021, p. 64) as well as assistance with external financing needs. The objectives in the two DPOs were part of the government’s ongoing reform agenda. 　 The objectives were ambitious, covering infrastructure financing, financial market reform, labor market reform, and reforming regulations governing the operation of firms. Sub-objectives had an even broader scope and included the issuing of securities by sub-prime borrowers, expanding the mutual fund industry, training the unemployed, broadening the scope of technology adoption by firms, reforming the institutions governing tax and customs, improving border clearing of imports and exports, and the trade relationship with Ecuador. A tighter focus with fewer reform areas could have improved outcomes. Rating Substantial b. Relevance of Design The strength of the causal chain between activities supported by the series and expected results is uneven. Overall, the theory of change is less than robust. 　 Increasing the availability of financing for road construction and enhancing access to finance for Colombian firms (the first part of the first objective) was to be achieved through the establishment of an institutional framework to widen private sector participation in the overall infrastructure program. However, the program- supported amount of incremental financing (less than US$4 million), would have no significant impact on reducing road infrastructure bottlenecks. 　 Enhancing access to finance for firms (the second part of the first objective) was to be attained through an already reformed secured transactions framework, an increase in the number of private bonds issuers, and augmenting the number of investors in mutual funds. Secured transaction reforms would allow firms to pledge their moveable property as collateral, hence reducing lending risks. Regarding private bonds, the program focused primarily on SMEs, less well-known investment grade borrowers, and those who had poorer credit Page 4 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Sust. Growth and Income Convergence DPL(P149609) ratings. It is unclear why the issuing of bonds by SMEs or companies with poor credit ratings was considered a priority. There was no assessment of the demand for such securities, or of associated transaction and interest costs. In most countries, SMEs are financed primarily by lending institutions rather than through bond issues. The lack of consideration of the significant risks to investors undermines the theory of change related to this objective. The PDs do not make it clear how increasing the number of mutual fund investors would necessarily improve access to finance for firms and for infrastructure investment. This would depend on how mutual fund managers direct the proceeds from the larger number of investors. They may even be invested outside Colombia (the Authorities had already put in place regulations allowing mutual fund investments in the countries of the Pacific Alliance – Chile, Mexico and Peru). A more relevant target would have been increased investment by mutual funds in infrastructure and company financing in Colombia. 　 Enhancing the skills of the labor force, building productive skills and improving skill allocation (the second PDO) was to be achieved through provision of appropriate training and better matching of job vacancies to the available skill mix. These links are clear. 　 Encouraging innovation and business efficiency and hence sustained growth and income convergence (the third PDO), was to be achieved by increasing tax benefits for technology and innovation, reducing the burden of regulation, and trade facilitation. Tax incentives were expected to encourage more firms to adapt (mostly foreign) technology to their needs. However, the link is tenuous. The PDs do not refer to analytical work that links low productivity of Colombian firms with their failure to innovate. Other causes of low productivity - for example, poor management practices - may have been given insufficient weight. Moreover, the link between faster growth and income convergence is unclear. 　 One of the barriers to innovation in Colombia is weak protection of intellectual property (CPS, p. 26). Yet this does not feature in the program. Burdensome regulations, which tend to drive firms into the informal sector (where productivity is low), were to be addressed through a piloting a system of regulatory evaluation that would identify those regulations that constrain productivity. The evaluation would examine the rationale behind specific regulations and specify their costs and benefits. However, the program falls short of supporting actual changes to the regulatory framework. Rating Modest 4. Achievement of Objectives (Efficacy) PHEFFICACYTBL Objective 1 Objective Improve the access to financial markets for road infrastructure investments and improve access to finance for firms. Page 5 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Sust. Growth and Income Convergence DPL(P149609) Rationale Under DPO1, the prior action consisted of amending credit limits that would allow institutions to increase their lending for a single project from 10% to 25%, provided that the additional 15% be used to fund 4G highway concession investments under government guarantee. A trigger for DPO2 was a reduction in restrictions on pension funds investing in private infrastructure funds and an easing of restrictions on funding in international markets. PD1 indicated that these actions were expected to increase the amount of financing allocated to the 4G program by credit institutions by US$ 3.3 million. Under DPO2, the allocation of financing between types of institutions was broken down: banks US$2.1 million; National Development Bank – US$0.7 million; and Pension Funds US$1.8 million. The ICR reports (20) that the US$3.3 million target was exceeded by early 2018, with a total of US$5.7 million being financed. Although this amount remains very small in relation to infrastructure needs (planned investments in the 4G program total US$26 billion), the task team subsequently explained to IEG that the Colombian Authorities viewed this as a demonstration aimed at proving the feasibility of raising private sector financing for the 4G program. Improving access to finance by firms was to be attained through (i) a new secured transactions framework, which allows firms to pledge their moveable property as collateral (prior action, DPO1); and (ii) promoting issuance of bonds. The collateral registry was linked to that for motor vehicles, and a central registry for electronic invoices was created (both prior actions, DPO2) Triggers involved the definitions and regulations governing credit guarantees and the operation of the registries. The target was the registration of 1 million security interests in the secured transactions registry; the actual number achieved was 1.65 million. The ICR (p. 17) reports that 85% of the registered security interests were for automobiles, but in addition, the factoring of accounts receivable was encouraged by the invoice registry to promote the financing of receivables. A prior action under DPO1 encouraged the issuing of bonds by less well-known investment grade borrowers, as well as SMEs and other sub-prime borrowers through reducing issuing requirements, reducing issuance costs and faster approval times by the regulatory authorities. The issuing of rules and policies for the operation of mutual funds, and of further regulations governing their management, were, respectively, a trigger and a prior action for DPO2. The outcome targets were, first, that 10 new bond issuers would enter the market, and, second, that there would be an increase in the number of investors in mutual funds to 1.5 million. In the event, although there were 10 new bond issues, these were not executed by new entrants, but by existing firms taking advantage of the more relaxed regulations governing issues; the first target was therefore not achieved. The actual number of investors in mutual funds was 1.6 million by March 2018, thereby exceeding the second target. However, as noted in Section 3b above, more relevant to the program’s objectives would have been a target for increased investment by mutual funds in infrastructure and company financing in Colombia. Rating Substantial PHREVDELTBL PHEFFICACYTBL Page 6 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Sust. Growth and Income Convergence DPL(P149609) Objective 2 Objective Improve training, build productive skills and improve their allocation. Rationale Allocation of skills was to be improved through better job placements. A Public Employment Service incorporating a unified national unemployment registry with a system for matching skills with vacancies was established, and training courses for the unemployed, involving learning centers and special human capital formation for indigenous populations, were created (both prior actions, DPO1). Prior actions under DPO2 were (i) specification of the institutional arrangements and responsibilities in the public employment services for the implementation of management and job placement tools; (ii) creation of a national framework of professional qualifications and competencies; (iii) establishment of a system of transfers and credits connecting the different branches of tertiary education; and (iv) establishment of a special educational regime (through the tertiary level) for indigenous communities. There was no indicator to measure whether job placements had better matched skills with vacancies. Rather, the outcome target was expressed in terms of the number of job placements: an increase of 149,000. This was significantly surpassed, with the cumulative number placed in employment between 2014 and 2017 increasing by more than 1.7 million. Whether this indicates improved productive skills and skill allocation is unclear, as is the extent to which it can be attributed to actions supported by the series. It is impossible to ascertain whether these were new jobs that moved workers in the informal sector into the formal sector, or whether these were job placements that would have occurred anyway under the pre- program system. Further outcome target were: (i) 1.12 million persons referred to management placement services (actual achievement 3.38 million); (ii) at least 40 firms providing professional training and the preparation of qualification matrices for different education levels (actual achievement 40); and (iii) four indigenous communities to be certified for the administration of budgetary funds for training (not met - to be certified, communities must become indigenous territories and demonstrate that they can administer budgetary funds; by program closure, no communities had yet qualified, although the ICR reports partial progress towards qualification in a number of instances). A DPO1 target of 60,000 retrained unemployed was dropped under DPO2 because of lack of progress under DPO1. Rating Modest PHREVDELTBL PHEFFICACYTBL Objective 3 Objective Strengthen regulations that affect innovation and business efficiency, as a means to support sustained growth and income convergence. Rationale Page 7 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Sust. Growth and Income Convergence DPL(P149609) There were three sub-objectives: (i) expanding the use of science and technology; (ii) reducing the regulatory burden on businesses; and (iii) trade facilitation. 　 Science and Technology. The system of tax credits for scientific and technological innovation was strengthened (prior action, DPO1). Prior actions under DPO2 were the definition of eligibility criteria, creation of an information platform, the definition of criteria for eligible expenses, and the merging of the Governance System of Competitiveness and Innovation with that for Science and Technology. There were three outcome targets: (i) an increase in the number of projects financed with royalty resources from 0 to 20 (exceeded - by the end of 2017, 47 had been approved); (ii) an increase in the share of budgeted tax benefits utilized from a baseline of16 per cent to 50 percent (exceeded - in both 2016 and 2017, 100 per cent of resources were used); and (iii) an increase from zero to 30 percent in the number of projects and programs designed and implemented under the combined system of governance (this was not met; the ICR reports on page 21 that inter agency disagreements hampered the operation of the unified system). No evidence is provided concerning the impact of these technology programs on business efficiency or on sustained growth and income convergence. Reducing the Regulatory Burden. For the second sub-objective, a more coherent framework for reviewing existing regulations and designing new ones was designed (prior action, DPO1). Triggers were the creation of a Committee for Improving the Production of Regulations, and the reorganization of the National Tax and Customs Administration to streamline processes. Under DPO2, all agencies drafting new regulations were required to consult with the Superintendence of Industry and Commerce, although this was subsequently amended to improving procedures for the issuance of regulations. There was one outcome target: an increase from zero to five in the number of pilots carrying out the new regulatory evaluation process (this was exceeded: four pilots were prepared in 2016 and three in 2017). However, no evidence is yet available on reducing the regulatory burden on Colombian businesses. Trade Facilitation. For the third sub-objective, the National Tax and Customs Administration was reorganized to allow for improved inspection methods for imports, and the single window for international trade was strengthened by permitting the extension of simultaneous inspection methods to imports (both prior actions, DPO1). Under DPO2, Customs procedures were further streamlined, and a system established whereby eligible economic operators would be authorized to benefit from such procedures. The outcome target was 30 economic operators, which was exceeded (40 had been authorized by March 2018). However, simultaneous inspections for imports were not yet operational at the time of ICR completion (p. 13). Moreover, the decline in oil prices and the consequent fall in government revenue, together with a change in Customs leadership, resulted in the postponement of the planned restructuring of the service. No evidence is presented of faster Customs procedures. Rating Modest PHREVDELTBL PHREVISEDTBL Page 8 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Sust. Growth and Income Convergence DPL(P149609) 5. Outcome The projects objectives were substantially relevant in the light of both deteriorating external conditions and the slowing growth and low productivity of the domestic economy and were in accordance with both World Bank and government strategies. Relevance of design was mixed, however, with the large number of sub-objectives contributing to the less than robust theory of change. The outcome of the first of three objectives - improving infrastructure financing and financing for firms - is rated substantial. The secured transactions reform increased the potential for the financing of firms, and the target for the registration of security interests was substantially exceeded, although the goal of promoting bond issues by sub-prime lenders was not met. The value of the instruments issued for infrastructure financing relative to needs was very small, although the institutional and regulatory framework for such lending was tested and strengthened. 　 The efficacy of the second objective - improving worker training, enhancing productive skills and better job placement - is rated modest. Information on the efficacy of training and productive skills is not available. There was no evidence of better matching of job placements and skills. Although the targeted number of job placements was substantially exceeded, it is unclear to what extent this result could be attributed to the program. 　 The third objective is rated as modest because little evidence is provided on results. For example, the impact of the regulation rationalization framework on the regulatory burden facing firms was not specified, while the increase in the number of economic operators benefiting from accelerated customs clearing provides no indication of reduced clearance time. No evidence is provided that program-supported reform enhanced economic growth or income convergence. 　 Overall, the shortcomings are significant, and outcome is assessed as moderately unsatisfactory. a. Outcome Rating Moderately Unsatisfactory 6. Rationale for Risk to Development Outcome Rating The improved legal framework, particularly in the area of secured transactions, supports economic activities and is likely to be sustained. The effectiveness and sustainability of the infrastructure financing program will depend on the extent to which private sector financing will utilize the new institutional framework. Since the job matching program resulted from a change in the institutional structure, it will likely continue. Trade facilitation reforms have been only partially carried out and are vulnerable to exogenous influences, vested interests and leadership of key services. Page 9 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Sust. Growth and Income Convergence DPL(P149609) a. Risk to Development Outcome Rating Modest 7. Assessment of Bank Performance a. Quality-at-Entry The program was prepared in close collaboration with the government and reflects both the CPS 2012- 2016 and the priorities of the government as indicated in the National Development Plan. The objectives were relevant for country’s needs. The program was implemented during a period of macro-economic pressure and was therefore timely. Design of the operations benefited from extensive analytical work performed by the government, the Bank, OECD and others, part of it some years previously. There were no fiduciary or environmental aspects that arose. M&E arrangements were the responsibility of the Government and were discussed by the task team during supervision. Risks were appropriately identified, and mitigation arrangements made; overall risk was assessed as modest. However, the strength of the theory of change reflected in project design was variable: in some instances, the causal chain between actions supported by the series and the expected results was unclear. For example, although the third PDO stated that higher growth and improved income distribution were desired outcomes, the link between these and the prior actions and other policy measures is obscure. Quality-at-Entry Rating Moderately Satisfactory b. Quality of supervision No Implementation Status Reports (ISRs) were prepared since they are not required for operations lasting less than one year. M&E arrangements were delegated to the Colombian government – according to the task team, there was sufficient in-house capacity to monitor the program. During the preparation of DPO2, the Bank team examined progress in implementation and as a result adjusted a number of indictors. Any loss of institutional memory stemming from the lack of formal supervision would be partly addressed through a follow up Development Policy Financing operation that was approved in 2016 which documented progress on the technology portion of the PDOs. Quality of Supervision Rating Moderately Satisfactory Overall Bank Performance Rating Moderately Satisfactory Page 10 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Sust. Growth and Income Convergence DPL(P149609) 8. Assessment of Borrower Performance a. Government Performance The ICR reports that the government was committed to the implementation of the program. The task team indicated that the government performed effectively in monitoring the program. Implementation of the reforms supported by the series was, however, mixed. While actions under the second pillar were mostly carried out, there were some shortfalls in other areas. Achievement under the first pillar was partial, and under the third, there were significant gaps. Trade facilitation was halted by adverse external developments and by changes in the leadership of the Customs Administration. Extending simultaneous inspection methods to imports was postponed because of lack of time to put in place the necessary logistics. Instead, the Authorities proposed streamlining procedures for authorized operators and implementing the multi-modal Andean International Transit of Goods (Transito Aduanero Internacional Multimodal,TIM) with Ecuador. In the event, the TIM was in place for only 18 months while financed through an Inter-American Development Bank project. No alternative plans have been put forward (ICR, p. 19). However, the task team reports that the government remains committed to reform. Colombia took the necessary measures to align with OECD entry requirements, and was admitted in 2018 Government Performance Rating Moderately Satisfactory b. Implementing Agency Performance N/A Implementing Agency Performance Rating Not Rated Overall Borrower Performance Rating Moderately Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design PD1 and PD2 (p. 30 and p. 36 respectively) both state: "The MHCP (Ministry of Finance and Public Credit), DNP (National Planning Department), and DANE (National Administrative Department of Statistics) are responsible for coordinating actions among the concerned agencies." The ICR (p. 14) states that "MHCP and DNP have the authority and capabilities to monitor implementation of the Program, although their staff is overstretched by the multiple functions that they perform". The task team subsequently informed IEG that the Colombian authorities had a high level of monitoring and evaluation capability. 　 Page 11 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Sust. Growth and Income Convergence DPL(P149609) DPL1 had 3 indicators under the first pillar, which was to foster solutions to improve infrastructure finance and to increase access to finance for firms. They were; (i) increasing the amount of financing allocated to 4G concession programs from 0 in 2013 to COP$10 billion in 2016; (ii) that the number of registrations of security interests in the collateral registry exceed 1 million, by 2016; and (iii) that 10 new issuers enter the bond market by 2016. The first of these does not capture the aim of this aspect of the pillar, which was to establish the institutional foundation for private sector infrastructure financing because it provides no information on whether this had been achieved and the actual amount of the financing target was so small as to be inconsequential. The second two indicators are of more relevance. 　 Under Pillar 2, which aimed to improve the quality and allocation of productive skills, the indicators were (i) the number of job placements achieved through the new system; (ii) the number of unemployed retrained; and (iii) the number of firms providing professional training. These indicators do not capture how productive skills were enhanced, although the number of job placements under the new system does go some way towards this, under the assumption that the placement system was more effective. Whether relevant skills were improved skills is unknown. 　 Under Pillar 3, promoting innovation and an efficient regulatory environment, the indicators were; (i) The share of budgeted tax benefits used by firms; and (ii) the number of sectors for which pilot regulatory evaluations occurred, and the evaluation of proposed regulations. The first of these is an input and provides no information regarding how this promoted innovation. The second does measure how the regulatory environment improved. 　 DPL2 revised some of the results indicators and replaced others. 　 Under pillar 1, the indicator for 4G financing was disaggregated by funders. And a new indicator was added related to the number of investors in mutual funds. The former left the DPO1 indicator essentially unchanged. The added indicator provides little information on how access to finance for firms, or for infrastructure financing would be enhanced. 　 Under pillar 2 the indicator regarding the number of unemployed that were retrained was dropped and indicators on the number of qualification matrices defining job skills was added. An additional indicator relating to the number of communities certified to manage transfer resources directly was also added. It is unclear how these indicators would improve the quality and allocation of productive skills. 　 Under pillar 3, new indicators were added: (i) an indicator that measured the number of projects evaluated under a new methodology for formulating, selecting and approving innovation projects; (ii) the number of instruments designed and implemented under a new governance system and (iii) the number of authorized economic operators increase. In addition, the target for the utilization of tax benefits for science and technology increase, and the target for the number of pilots for the regulatory evaluation process was revised upwards. Of the indicators, only the last relates directly to the goal of the third pillar. b. M&E Implementation Page 12 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Sust. Growth and Income Convergence DPL(P149609) The Bank monitored the program through the preparation of the second DPO (ICR p.14), in which progress was recorded in associated Aide Memoires. c. M&E Utilization The ICR provides no information on M&E utilization M&E Quality Rating Modest 10. Other Issues a. Environmental and Social Effects There were no environment implications for either of the DPOs. Positive social impacts could be expected from the focus on employment in Pillar 2. b. Fiduciary Compliance There were no fiduciary or procurement issues reported c. Unintended impacts (Positive or Negative) N/A d. Other N/A 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Program objectives are Moderately Moderately Outcome substantially relevant, but Satisfactory Unsatisfactory relevance of design is modest. Page 13 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Sust. Growth and Income Convergence DPL(P149609) Efficacy of two of the three objectives is rated modest, and the third substantial. Risk to Development Modest Modest --- Outcome Moderately Bank Performance Moderately Satisfactory --- Satisfactory Moderately Borrower Performance Moderately Satisfactory --- Satisfactory Quality of ICR Substantial --- Note When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 12. Lessons The following lessons are taken from the ICR with some adaptation of language: 　 The design and preparation of this DPO series benefitted from continuity with respect to Task Team Leaders and core team members, both with respect to knowledge of the issues and the economy and in regard to relationships with country counterparts; 　 The reform experience in Colombia underlines the need for intensive dissemination and implementation to bring about changes in behavior when institutional reform involves new laws or regulations. The use of the secured transactions reform to extend credit access is a case in point, where the initial use was primarily for financing of motor vehicles. More dissemination could have broadened its use 　 IEG draws the following two lessons: 　 Design of development policy operations can benefit from international experience and research. In this case, expected financing of SMEs and other sub-prime borrowers through issuing instruments such as bonds did not materialize since the risks associated with such instruments were perceived to be too high. Sufficient study of international experience would have illustrated the likelihood of such an outcome. 　 Political economy issues can often undermine reform objectives. Two examples were the difficulties associated with the attempted consolidation of the Committee for Competitiveness with the System for Science, Technology and Innovation, and the restructuring of the National Tax and Customs Administration, both of which undermined the achievement of the third pillar of the program. Intensive consultation, dissemination of the advantages of the reforms, and ensuring that all participants support the reform are necessary to reduce the risks of non-achievement of objectives. Page 14 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Sust. Growth and Income Convergence DPL(P149609) 13. Assessment Recommended? No 14. Comments on Quality of ICR The ICR is well written and explains clearly the issues faced by both the World Bank Group and the Government of Colombia in formulating the reform program. More discussion of the M&E issues would have been welcome, as would a more critical analysis of the labor market and technology issues. Overall, however, this is a sound document. a. Quality of ICR Rating Substantial Page 15 of 15