Document of The World Bank Report No: ICR00002059 IMPLEMENTATION COMPLETION AND RESULTS REPORT (IBRD-78310) ON A LOAN IN THE AMOUNT OF U.S.$20.42 MILLION TO THE REPUBLIC OF COLOMBIA FOR A CONSOLIDATION OF NATIONAL PUBLIC MANAGEMENT INFORMATION SYSTEMS PROJECT June 15, 2015 Governance Global Practice Mexico and Colombia Country Management Unit Latin America and the Caribbean Region CURRENCY EQUIVALENTS Currency Unit = Colombian Peso December 2009 COP$1.00 = US$ 0.0004998 US$ 1.00 = COP$2,000.48 December 2014 COP$1.00 = US$ 0.0004185 US$ 1.00 = COP$2,404.71 FISCAL YEAR January 1 – December 31 ABBREVIATIONS AND ACRONYMS BPIN National Investment Projects Bank (Banco de Proyectos de Inversión Nacional) CPS Country Partnership Strategy DGCPTN Public Credit and Treasury Directorate (Dirección General de Crédito Público y del Tesoro Nacional) DIAN National Tax and Customs Directorate (Dirección de Impuestos y Aduanas Nacionales) DNP National Planning Department (Departamento Nacional de Planeación) FUT Single Territorial Form (Formulario Único Territorial) GDP Gross Domestic Product ICB International Competitive Bidding Process ICR Implementation Completion and Results Report ICT Information and Communication Technology IFR Interim Financial Report IMF International Monetary Fund ISR Implementation Status and Results Report M&E Monitoring and Evaluation MHCP Ministry of Finance and Public Credit (Ministerio de Hacienda y Crédito Público) MS Moderately Satisfactory MUISCA Integrated Tax and Customs Administration Model (Modelo Único de Ingresos, Servicios y Control Autorizado) NTCGP Technical Norm of Quality for Public Management (Norma Técnica de Calidad en la Gestión Pública) PAD Project Appraisal Document PAU Project Administrative Unit PDO Project Development Objective PFM I First Public Financial Management Project PFM II Second Public Financial Management Project PGN National General Budget (Presupuesto Nacional de la Nación) SECOP Electronic Procurement System (Sistema Electrónico de Contratación Pública) SEIGI Specialized Information and Management System (Sistema Especializado de Información y Gestión Integral) SIGOB System of Programming and Management by Objectives and Results (Sistema de Programación y Gestión por Objetivos y Resultados) SIIF- Integrated Financial Management Information System Nación SILOG Logistic Synergy System (Sinergia Lógistica) SINERGIA Public Sector Performance Evaluation System (Sistema Nacional de Evaluación de Resultados de la Gestión Pública) SMGI Integrated Management Monitoring System (Sistema de Monitoreo de la Gestión Integral) STA Single Treasury Account SUG Unified Management System (Sistema Único de Gestión) SUIFP Unified Investment Management System (Sistema Unificado de Inversiones y Finanzas Públicas) Vice President: Jorge Familiar Country Director: Gerardo M. Corrochano Senior Practice Director Mario Marcel Cullell Practice Manager: Arturo Herrera Project Team Leader: Pedro Arizti/Henry Forero ICR Team Leader: Pedro Arizti/Henry Forero ICR Author Adrienne Hathaway COLOMBIA Consolidation of National Public Management Information Systems Project CONTENTS Data Sheet A. Basic Information B. Key Dates C. Ratings Summary D. Sector and Theme Codes E. Bank Staff F. Results Framework Analysis G. Ratings of Project Performance in ISRs H. Restructuring I. Disbursement Graph 1. Project Context, Development Objectives and Design ............................................... 1 2. Key Factors Affecting Implementation and Outcomes .............................................. 4 3. Assessment of Outcomes .......................................................................................... 10 4. Assessment of Risk to Development Outcome......................................................... 16 5. Assessment of Bank and Borrower Performance ..................................................... 17 6. Lessons Learned ....................................................................................................... 18 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners .......... 19 Annex 1. Project Costs and Financing .......................................................................... 21 Annex 2. Outputs by Component ................................................................................. 22 Annex 3. Economic and Financial Analysis ................................................................. 30 Annex 4. Bank Lending and Implementation Support/Supervision Processes ............ 32 Annex 5. Beneficiary Survey Results ........................................................................... 34 Annex 6. Stakeholder Workshop Report and Results................................................... 35 Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR ..................... 36 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders ....................... 37 Annex 9. List of Supporting Documents ...................................................................... 38 Annex 10. Changes to PDO Indicators, January 2013 Restructuring……………...…39 Annex 11. Reallocation of Loan Funds, December 2013 Restructuring…………..….40 Annex 12. Level of Achievement – Final PDO indicators……………………………41 MAP A. Basic Information Consolidation of National Public Country: Colombia Project Name: Management Information Systems Project ID: P106628 L/C/TF Number(s): IBRD-78310 ICR Date: 06/15/2015 ICR Type: Core ICR MINISTRY OF Lending Instrument: TAL Borrower: FINANCE Original Total USD 25.00M Disbursed Amount: USD 20.42M Commitment: Revised Amount: USD 20.42M Environmental Category: C Implementing Agencies: Ministry of Finance and Public Credit National Planning Department National Tax and Customs Department Cofinanciers and Other External Partners: B. Key Dates Revised / Actual Process Date Process Original Date Date(s) Concept Review: 09/29/2009 Effectiveness: 03/29/2010 01/13/2010 05/09/2012 Appraisal: 11/03/2009 Restructuring(s): 01/30/2013 12/13/2013 Approval: 12/17/2009 Mid-term Review: 08/15/2011 08/19-26/2011 Closing: 12/31/2012 12/15/2014 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Satisfactory Risk to Development Outcome: Moderate Bank Performance: Moderately Satisfactory Borrower Performance: Moderately Satisfactory C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Quality at Entry: Moderately Satisfactory Government: Satisfactory Quality of Supervision: Satisfactory Implementing Moderately Satisfactory Agency/Agencies: Overall Bank Overall Borrower Moderately Satisfactory Moderately Satisfactory Performance: Performance: C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments Indicators Rating Performance (if any) Potential Problem Project Quality at Entry No None at any time (Yes/No): (QEA): Problem Project at any Quality of No None time (Yes/No): Supervision (QSA): DO rating before Satisfactory Closing/Inactive status: D. Sector and Theme Codes Original Actual Sector Code (as % of total Bank financing) Central government administration 97 97 Public administration- Information and communications 3 3 Theme Code (as % of total Bank financing) Administrative and civil service reform 3 3 Macroeconomic management 3 3 Public expenditure, financial management and 64 64 procurement Tax policy and administration 30 30 E. Bank Staff Positions At ICR At Approval Vice President: Jorge Familiar Calderon Pamela Cox Country Director: Gerardo M. Corrochano Gloria M. Grandolini Practice Arturo Herrera Gutierrez Nicholas Paul Manning Manager/Manager: Project Team Leader: Pedro Arizti/Henry Forero Edgardo Mosqueira Medina ICR Team Leader: Pedro Arizti/Henry Forero ICR Primary Author: Adrienne Elizabeth Hathaway F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) The objective of the Project was to solidify and expand the gains achieved by the Borrower under PFM II and further improve the transparency, coverage, operation and Interoperability of the Borrower's national public management information systems to allow the Borrower's decision-makers to link performance measures with planning, budgeting, and other managerial actions. The Project was to focus on: i. Enhancing public sector efficiency and transparency by strengthening and expanding the individual management information systems that are the building blocks of an integrated, performance-informed management model, including SIIF-Nación, SUIFP and MUISCA. ii. Support the design and implementation of mechanisms aimed at enabling the inter- operability of individual public management tools and systems. iii. Promoting the full use of information resulting from these integrated PM tools and systems to enhance strategic decision making. Revised Project Development Objectives (as approved by original approving authority) (a) PDO Indicator(s) Original Target Formally Actual Value Values (from Revised Achieved at Indicator Baseline Value approval Target Completion or documents) Values Target Years Information on budget execution—i.e., revenues and expenditures—that is Indicator 1 : available on a daily basis to increase f 85% to 100% of the total amount of the National Budget (Presupuesto General de la Nación – PGN) Information on budget execution—i.e., revenues 100% and expenditures—is reconciliation of Value available daily for 85% of expenditures and quantitative or 100% the total amount the revenues in real Qualitative) National Budget time with SIIF- (Presupuesto General de Nación la Nación – PGN) Date achieved 11/18/2009 12/31/2012 12/15/2014 Comments Achieved. In real time, corresponding to 193 executing agencies – 88 at the (incl. % central level and 105 at the subnational level. achievement) Time lag to produce aggregate financial data for the central government, Indicator 2 : including decentralized entities, has been reduced from up to 35 days to 1 day. 1 day for central 1 day for both Value administration, incl. centralized and quantitative or 1 day centralized entities, and decentralized Qualitative) 35 days for decentralized entities within the entities central government Date achieved 11/18/2009 12/31/2012 12/15/2014 Comments Achieved. Global financial information, corresponding to 193 executing (incl. % agencies – 88 at the central level and 105 at the subnational level. achievement) 100% of consolidated budget information (at the level of individual entities) is Indicator 3 : published online on a monthly basis within the first week of the following month. 100% of 100% of budget consolidated information budget information consolidated online (at the level of (Economic Value Limited availability of individual entities) Transparency quantitative or public financial is published online Portal, Qualitative) administration on a monthly basis www.pte.gov.co) within the first on a daily basis, week of the one day after following month availability. Date achieved 11/18/2009 12/31/2012 12/15/2014 Comments (incl. % Achieved. achievement) Average customs clearance time has been reduced from 48 hours to less than 8 Indicator 4 : hours. Average time (i.e., hours) Value required to obtain 6.7 hours with quantitative or 8 hours customs clearance is 48 electronic payment. Qualitative) hours (2009) Date achieved 11/18/2009 12/31/2012 12/15/2014 Comments (incl. % Achieved, target surpassed. achievement) Tax administration efficiency increases, as reflected by an increase in the ratio Indicator 5 : total tax collection/DIAN’s executed budget from 103.2 to 103.7 Value Total tax collection / quantitative or DIAN’s Executed budget 101.6 106.03 Qualitative) ($101.1 as of Sept. 2009) Date achieved 11/18/2009 12/31/2012 12/15/2014 Comments Achieved, target surpassed. Changed in restructuring. Original indicator: Tax (incl. % administration efficiency increases, as reflected by an increase in the ratio total achievement) tax collection / DIAN’s executed budget from 101.1 to101. 6 by project closing. Indicator 6 : Number of studies completed to strengthen fiscal and macroeconomic planning The Technical viceminister - Value (Macroeconomic policy 12 studies quantitative or general direction - Total of 9 completed. Qualitative) DGPM) has limited technical and institutional capacity to conduct macroeconomic and fiscal studies. Date achieved 01/30/2013 12/31/2013 12/15/2014 Comments Achieved. Indicator changed in restructuring. Original indicator: The newly (incl. % created Center for Fiscal and Financial Studies has been staffed and assigned achievement) budgetary resources. SIIF and SUIFP information on the allocation and execution of budget items is Indicator 7 : fully consistent. Not clear in PAD. From RP#74936-CO: Lack of Value consistent information Implicit in 100 percent quantitative or between the investment indicator name. consistent Qualitative) information system (SUIFP) and the budget execution system (SIIF). Date achieved 11/18/2009 12/31/2012 12/15/2014 Comments (incl. % Achieved. achievement) (b) Intermediate Outcome Indicator(s) Original Target Actual Value Formally Values (from Achieved at Indicator Baseline Value Revised approval Completion or Target Values documents) Target Years SIIF-Nación implemented throughout the central administration (i.e., 63 Indicator 1 : administrative units [AU] within the central administration and 100 AU in decentralized entities) SIIF-Nación implemented in SIIF-I 63 administrative 100% of executing units within the central agencies of the Value administration; several central (quantitative incongruent systems used 100% government and or Qualitative) for financial 100% of public administration by agencies decentralized entities (establecimientos públicos) Date achieved 11/18/2009 12/31/2012 12/15/2014 Comments Achieved. SIIF-Nación is used by 100% of national entities (88 at the central (incl. % level and 105 at the subnational level for a total of 193 executing agencies) achievement) Indicator 2 : Gap between current competences has been gradually reduced. Human resources Gap between Value management within the current (quantitative MHCP is not framed competencies and 4.15 or Qualitative) within the institution's new ones is overall strategy. reduced progressively on a year basis Date achieved 11/18/2009 12/31/2012 12/15/2014 Partially achieved. In Dec. 2010 an evaluation was undertaken to determine the Comments gap between required and actual competencies. The result was a score of 4.4 in a (incl. % scale from 1 to 5, exceeding the baseline of 3.6 for 2009. For 2011 the score was achievement) 4.1. Indicator 3 : Monitoring and Evaluation system in place. No mechanisms in place Value to ensure quality and (quantitative 100% 100% efficiency of MHCP's or Qualitative) management practices Date achieved 11/18/2009 12/31/2014 12/15/2014 Achieved. Sistema de Monitoreo de la Gestión Integral -SMGI- is in place. All Comments modules are working satisfactorily and improvements are being implemented as (incl. % requested by MHCP. Strategic Institutional Plan is aligned to the Strategic achievement) Sectorial Plan. Indicator 4 : Certification of quality has been obtained by SUG. No mechanisms in place Value Certification of to ensure quality and (quantitative quality has been 100% efficiency of MHCP's or Qualitative) obtained for SUG. management practices Date achieved 11/18/2009 12/31/2012 12/15/2014 Comments Achieved. ICONTEC NTCGP 1000:2009 and ISO's 9001:2008 IQNet (incl. % Certification obtained on July 26, 2010 achievement) The ratio between programmed budgetary resources for investment projects and Indicator 5 : requested budgetary resources for investment projects has increased from 81.6% to 92%. Value (quantitative 81.6% 92% 102% or Qualitative) Date achieved 11/18/2009 12/31/2012 12/15/2014 Comments (incl. % Target surpassed. achievement) The ratio between the number of investment projects programmed and the Indicator 6 : number of investment projects registered has increased from 86.8% to 90%. Value (quantitative 86.8% 90% 122% or Qualitative) Date achieved 11/18/2009 12/31/2012 12/15/2014 Comments (incl. % Target surpassed. achievement) The number of high impact DIAN procedures that have been modernized Indicator 7 : increase from 20 to 40 (100%). Value 20 40 66 (quantitative or Qualitative) Date achieved 11/18/2009 12/31/2012 12/15/2014 Comments (incl. % Achieved. achievement) Number of pending obligations (compromisos) normalized has increased from 0 Indicator 8 : to 540,000. Value (quantitative 0 546,227 568,445 or Qualitative) Date achieved 11/18/2009 12/31/2012 12/15/2014 Comments (incl. % Target surpassed. achievement) The average number of physical documents required for customs clearance has Indicator 9 : been reduced from 12 to 5. Value (quantitative 12 5 9 or Qualitative) Date achieved 11/18/2009 12/31/2012 12/15/2014 Comments Partially achieved. Accomplishment of indicator would be confirmed with the (incl. % issuance of a Decree in the context of the Customs Statute. achievement) The number of procedures that are subject to quality audits by DIAN's internal Indicator 10 : quality control procedures has increased from 67 to 199. Value (quantitative 67 199 199 or Qualitative) Date achieved 11/18/2009 12/31/2012 12/15/2014 Comments (incl. % Achieved. achievement) Indicator 11 : Staff Development Plan has been fully implemented Value (quantitative 0% 100% Not applicable or Qualitative) Date achieved 11/18/2009 12/31/2012 12/15/2014 Comments (incl. % Dropped in January 2013 restructuring. achievement) Progress on development of virtual learning program for improving technical Indicator 12 : capacity of DIAN staff Value (quantitative 21.54% 100% 100% or Qualitative) Date achieved 12/31/2010 12/31/2013 12/15/2014 Comments Achieved. A total of 15 processes were developed. (incl. % achievement) Improve DIAN’s National Transparency Index from 81.25 (2007-2008) to 89.51 Indicator 13 : by project closing. Value (quantitative 81.25 89.51 0.0 or Qualitative) Date achieved 11/18/2009 12/31/2012 12/15/2014 Comments Not achieved. No progress reported as Transparencia por Colombia did not carry (incl. % out the national transparency index for DIAN in 2013. achievement) Number of Systems Services (Servicios Informáticos Electrónicos), transferred Indicator 14 : to DIAN’s technology area increase from 12 to 40. Value (quantitative 12 40 66 or Qualitative) Date achieved 11/18/2009 12/31/2012 12/15/2014 Comments (incl. % Target surpassed achievement) Indicator 15 : Carry out pilots allowing the disaggregation of expenditure information. Value Limited interoperability (quantitative 1 pilot 1 pilot of PM systems. or Qualitative) Date achieved 11/18/2009 12/31/2012 12/15/2014 Comments Achieved. DNP - Pilot completed. 100% interoperation with SUIFP system. (incl. % Ministry of Defense - Pilot completed. 100% interoperation with SILOG achievement) Indicator 16 : Carry out pilots integrating data generated by the different government systems. Value Limited interoperability (quantitative 1 pilot 1 pilot program of PM systems or Qualitative) Date achieved 11/18/2009 12/31/2012 12/15/2014 Comments Achieved. DNP- Pilot completed. 100% interoperation with SUIFP. SECOP - (incl. % 100% implementation of “planos" with SECOP regarding CDPs, commitments achievement) and future fiscal years. Indicator 17 : Full interoperability of SIIF–Nación with SECOP and Public Debt. Value Limited interoperability Implicit in (quantitative 100% of PM systems indicator name or Qualitative) Date achieved 11/18/2009 12/31/2012 12/15/2014 Achieved. SECOP - 100% implementation of “planos" with SECOP regarding Comments CDPs, commitments and future fiscal years. (incl. % Public Debt - MHCP - 100% interoperation through ''archivos planos'' to carry achievement) out public debt payments. Full interoperability of SIIF Nación with the following systems: PAGOS (Banco Indicator 18 : República), SILOG (Defensa) and SOBRETASA (MHCP). Value Limited interoperability 100% 76.6% (quantitative of PM systems or Qualitative) Date achieved 11/18/2009 12/31/2012 12/15/2014 Partially achieved. Interoperability with SILOG -subject to definition of goods and services from the National Procurement Agency. PAGOS -100% Comments interoperability with Banco de la República's payment system. SILOG-Pilot (incl. % completed, 100% interoperability. SOBRETASA (MHCP) – interoperability achievement) through archive mechanism; full interoperability will depend on finalization of modernization of the system’s application. The DGCPTN's integrated management system, SEIGI, is fully operational. No Indicator 19 : other information system is in operation within the DGCPTN. Value (quantitative Not available 100% Not applicable or Qualitative) Date achieved 11/18/2009 12/31/2012 12/15/2014 Comments (incl. % Dropped in January 2013 restructuring. achievement) The DGCPTN's integrated management system, SEIGI, is fully operational. No other treasury and debt management system is in operation within the DGCPTN Indicator 20 : (the current debt system is maintained temporarily to have a record of historical data). Value (quantitative Not available 100% 0 or Qualitative) Date achieved 01/30/2013 12/31/2013 12/15/2014 Comments Not achieved. Indicator added in January 2013. The indicator is subject to the (incl. % definition and implementation of the new business model. achievement) The number of DGCPTN staff that utilizes the new information system has Indicator 21 : increased from 0 to 120. Value (quantitative 0 120 Not applicable. or Qualitative) Date achieved 11/18/2009 12/31/2012 12/15/2014 Comments (incl. % Dropped in January 2013 restructuring. achievement) The percentage of DGCPTN staff that is required to use the new information Indicator 22 : system, and effectively does so, has increased from 0% to 100%. Value (quantitative 0 100% 0 or Qualitative) Date achieved 01/30/2013 12/31/2013 12/15/2014 Comments Not achieved. Indicator added in January 2013. The indicator is subject to the (incl. % definition and implementation of the new business model. achievement) Indicator 23 : The number of methodologies and processes designed for front, middle and back office operations has increased from 2 to 10. Value (quantitative Not available 10 10 or Qualitative) Date achieved 11/18/2009 12/31/2012 12/15/2014 Comments (incl. % Achieved. achievement) The number of staff persons linked to the DGCPTN's Trading Room has Indicator 24 : increased from 0 to 140. Value (quantitative Not available 140 Not applicable or Qualitative) Date achieved 11/18/2009 12/31/2012 12/15/2014 Comments (incl. % Dropped in January 2013 restructuring. achievement) The number of instruments that the DGCPTN is able to register in the system has Indicator 25 : increased from 15 to 25. Value (quantitative 15 25 15 or Qualitative) Date achieved 01/30/2013 12/31/2013 12/15/2014 Comments Not achieved. Indicator added in January 2013. The indicator is subject to the (incl. % definition and implementation of the new business model. achievement) Studies have been conducted on the economic and social impact of global Indicator 26 : warming. Value The DEE has limited 31 studies (quantitative capacity to conduct in- Studies completed completed or Qualitative) depth strategic studies. Date achieved 11/18/2009 12/31/2012 12/15/2014 Comments (incl. % Achieved. achievement) Studies have been conducted on the economic and social impact of alternative Indicator 27 : social security reform alternatives. Value The DEE has limited (quantitative capacity to conduct in- Studies completed. 6 studies completed or Qualitative) depth strategic studies. Date achieved 11/18/2009 12/31/2012 12/15/2014 Comments (incl. % Achieved. achievement) Indicator 28 : Plan for the implementation of Decree 28 is prepared. Value Plan for (quantitative Not available Plan is prepared implementation of or Qualitative) Decree 28 is defined and in place. Date achieved 11/18/2009 12/31/2012 12/15/2014 Comments (incl. % Achieved. achievement) A methodology for the selection of public expenditure projects for evaluation Indicator 29 : purposes is being implemented Methodology is applied for the Value selection of (quantitative Not available Not applicable programs and or Qualitative) projects to be evaluated. Date achieved 11/18/2009 12/31/2012 12/15/2014 Comments (incl. % Dropped in January 2013 restructuring. achievement) Decrees and other legal norms to promote efficiency and effectiveness have been Indicator 30 : drafted. The legal framework Documents with requires inefficient proposed procedures; oversight of Value adjustments to the Suggested legal budget execution is (quantitative legal norms that framework oriented towards or Qualitative) regulate the budget finalized. compliance rather than process have been effectiveness and drafted. efficiency. Date achieved 01/30/2014 12/31/2013 12/15/2014 Achieved. Indicator introduced in the January 2013 restructuring. Decree 4836 from 2011 has been issued aiming to promote the use of SIIF and SUIFP as a main source of information for budgetary decision making. Two projects for Comments decrees were prepared. The first develops the procedures and scope of the (incl. % Medium Term Expenditure Framework and its articulation with other tools for achievement) budgetary programming; the draft of the decree served as the basis for Decree 1610 (2013). The second refers to the use of future periods years for Public Private Partnership projects, and was issued. Tools for managerial decision-making, budget monitoring and evaluation are Indicator 31 : available. Budget execution Tools and information information is on- Information of systems oriented towards line and budget execution is process administration consolidated as a Value available online and and not for managerial tool of information (quantitative consolidated as and decision making. and or Qualitative) information and Insufficient information communication, communication for managerial decision with digital tool. making dashboards for decision making. Date achieved 01/30/2013 12/31/2013 12/15/2014 Comments (incl. % Achieved. achievement) G. Ratings of Project Performance in ISRs Actual Date ISR No. DO IP Disbursements Archived (USD millions) 1 06/25/2010 Satisfactory Satisfactory 3.76 2 02/06/2011 Moderately Satisfactory Moderately Satisfactory 7.73 3 07/05/2011 Moderately Satisfactory Satisfactory 10.03 4 01/23/2012 Moderately Satisfactory Moderately Satisfactory 14.40 5 08/21/2012 Moderately Satisfactory Moderately Satisfactory 18.13 6 04/15/2013 Moderately Satisfactory Moderately Satisfactory 19.34 7 10/19/2013 Moderately Satisfactory Moderately Satisfactory 19.42 8 03/10/2014 Satisfactory Moderately Satisfactory 19.69 9 10/15/2014 Satisfactory Moderately Satisfactory 19.83 10 12/10/2014 Satisfactory Moderately Satisfactory 20.18 H. Restructuring (if any) ISR Ratings at Amount Board Restructuring Disbursed at Restructuring Reason for Restructuring & Approved Restructuring Date(s) Key Changes Made PDO Change DO IP in USD millions The Level 2 restructuring extended the closing date from December 31, 2012 to December 31, 2013, with the goal of allowing time for the development and 05/09/2012 MS MS 16.27 implementation of information technology (IT) systems in the Public Credit and Treasury Directorate under Subcomponent 4.2 of the Project. The Level 2 restructuring revised two PDO indicators and seven intermediate indicators to 01/30/2013 MS MS 19.34 better capture Project results, and also slightly revised the aim of Component 4.3. The Level 2 restructuring 12/13/2013 MS MS 19.59 extended the closing date by ISR Ratings at Amount Board Restructuring Disbursed at Restructuring Reason for Restructuring & Approved Restructuring Date(s) Key Changes Made PDO Change DO IP in USD millions 11.5 months to December 15, 2014; redefined selected activities in Component 4.2, and initiated modernization of the DGCPTN via a business re- engineering exercise and design of a new business model; and carried out a partial cancellation of loan funds originally allocated to Component 3 given that the component’ activities had already been completed. I. Disbursement Profile 1. Project Context, Development Objectives and Design 1.1 Context at Appraisal 1. At the time of Project design, Colombia’s economy – which had been performing well in recent years as a result of sound economic policies, far-reaching structural reforms and favorable external conditions – faced some challenges. The global economic downturn was posing new challenges to Colombia’s near term economic performance, with gross domestic product (GDP) growth having slowed from 7.5 percent in 2007 to just 2.5 percent during 2008 and the structural balance weakening that year in spite of the Government’s decision to cut government spending. Fiscal responsibility was perceived as key to assuring markets that public debt as a share of GDP would continue to decline, and that the public sector’s financing needs would remain modest. 2. The uncertainties of the global economic environment called for a flexible approach to fiscal management. In the short term, advances in the implementation of performance-oriented information mechanisms and integrated fiscal management tools were viewed as something that could greatly enhance the quality of information and decision making for the allocation of budget expenditures, improving the quality of expenditure and the efficiency of public revenue collection. In the medium and long terms, improvements in the quality and availability of information related to expenditure, incomes, investment, debt and other fiscal issues were seen as crucial to improving the transparency, efficiency and efficacy of public management of resources. 3. Progress toward enhancing public sector management and improving transparency in public finances had been substantial over the prior ten years, and included important advances such as the clarification of expenditure management responsibilities among different levels of government, the introduction of modem accounting concepts and procedures, and the improvement of Public Management (PM) tools and systems in areas covering expenditure and revenue, as well as monitoring and evaluation (M&E), macro-programming and debt management functions. The Bank had provided support to the different phases in the development and implementation of these public management systems and tools through several different projects, including the First Public Financial Management Project 1 (PFM I) which provided technical assistance to support incipient government efforts to strengthen resource mobilization, macro-programming, expenditure management, and government management control and public policy evaluation, and included support to the early stages of the performance-informed Public Sector Performance Evaluation System’s (Sistema Nacional de Evaluación de Resultados de la Gestión Pública, SINERGIA) development. A follow-on operation, the Public Financial Management II Project 2 (PFM II) supported reforms in revenue administration 1 P006889, U.S.$30 million 2 P040109, U.S. $35.47 million 1 and public expenditure management at the central government level, including the development of new information systems and strengthening of existing ones 3. 4. As such, the current Project was designed to support the third phase in the development of Colombia’s Public Management systems by addressing some of these remaining structural and other challenges to fully operationalizing existing public management tools and systems and supporting an integrated vision of a performance- oriented public management system by building upon and consolidating the achievements of the PFM I & II operations. Over the years, the Bank had amassed valuable country- specific expertise on the development challenges confronting Colombia’s achievement of ongoing public sector management strengthening goals and was well positioned to support efforts in this area. The Project was fully consistent with both Bank and Government priorities, aligned with both the pillar of Colombia’s National Development Plan for 2006-2010 called “A better state at the service of citizens” and a similarly named pillar (“A State at the Service of its Citizens: Efficient and Effective Government.”) in the Bank’s Country Partnership Strategy (CPS) FY2008-2011 (Report No. 42847-CO). 1.2 Original Project Development Objectives (PDO) and Key Indicators 5. The objective of the Project were to solidify and expand the gains achieved by the Borrower under PFM II and further improve the transparency, coverage, operation and Interoperability of the Borrower’s national public management information systems to allow the Borrower’s decision-makers to link performance measures with planning, budgeting, and other managerial actions. 4 6. The Project was to focus on 5: i. Enhancing public sector efficiency and transparency by strengthening and expanding the individual management information systems that are the building blocks of an integrated, performance-informed management model, including SIIF-Nación, SUIFP and MUISCA. ii. Support the design and implementation of mechanisms aimed at enabling the inter-operability of individual public management tools and systems. iii. Promoting the full use of information resulting from these integrated PM tools and systems to enhance strategic decision making. 3 Systems supported included the development of the Integrated Financial Management Information System named SIIF-Nación; a new Unified Investment Management System (Sistema Unificado de Inversiones y Finanzas Públicas – SUIFP), and the Integrated Tax and Customs Administration Model (Modelo Único de Ingresos, Servicios y Control Autorizado - MUISCA); and enhancement of the Integrated Financial Management Information System SIIF I, the National Investment Projects Bank (Banco de Proyectos de Inversión Nacional - BPIN), and the strengthening of SINERGIA and one of its components, the System of Programming and Management by Objectives and Results (Sistema de Programación y Gestión por Objetivos y Resultados – SIGOB). 4 As per the Loan Agreement dated December 29, 2009. 5 These specific focuses were noted in paragraph 21 of the Project Appraisal Document (PAD), describing the Project’s development objective. However, only paragraph 5, above, appears in both the Loan Agreement and the PAD. As such, project efficacy is evaluated against the objective as defined in the Loan Agreement. 2 1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification 7. No changes were made to the PDO during implementation. In the January 2013 restructuring, two PDO indicators were revised in order to adjust the baseline of PDO#5 to reflect the real 2009 year end value (and consequently adjust the targets so that the percentage increase in efficiency was maintained), and to change the indicator designed to capture increased capacity of the unit responsible for carrying out macroeconomic and fiscal forecasting (PDO#6). See Annex 10 for a summary of PDO indicators and changes. 1.4 Main Beneficiaries, 8. As captured by the PAD’s description of the higher level objectives to which the Project was to contribute, the direct beneficiaries of the Project were expected to be Colombian decision-makers, both within the Presidency and individual agencies who, through the use of performance-based information, would be able to link performance with planning, budgeting and other managerial actions. As the further expansion and consolidation of Colombia’s public management continued to strengthen Colombia’s public sector performance, consolidating fiscal discipline and promoting a better allocation of resources, the general public was to be an indirect beneficiary. 1.5 Original Components 9. Component 1: Strengthening of Public Management Information Systems within the Ministry of Finance and Public Credit (Ministerio de Hacienda y Crédito Público, MHCP) (US$8.8 million of IBRD financing). This component was to support the strengthening of the technical and institutional capabilities of the MHCP, through the provision of technical assistance, training and equipment required for the enhancement, implementation, completion and consolidation of the public management information systems developed under PFM II. 10. Component 2: Strengthening of the Public Management Information Systems within the National Planning Directorate (Departamento Nacional de Planeación, DNP) (US$1.2 million of IBRD financing). This component sought to strengthen the technical and institutional capabilities of DNP, through the provision of technical assistance, training and equipment aimed at increasing the overall efficiency and efficacy of the Government’s public investments and the consolidation of advancements achieved in the development of SUIFP under PFM II. 11. Component 3: Expansion of the National Tax and Customs Directorate’s (Dirección de Impuestos y Aduanas Nacionales, DIAN) Integrated Tax and Customs Administration Model (US$8.0 million of IBRD financing). This component was to support the carrying out of a series of activities aimed at supporting the consolidation, continuity and further development of MUISCA through the acquisition of goods, Information and Communication Technology (ICT) infrastructure, operating costs and the provision of technical assistance. 3 12. Component 4: Improving the Interoperability of Public Management Information Systems (US$5.8 million of IBRD financing). This component was to underpin the carrying out of a series of activities aimed at supporting the integration of key public information systems and promoting the strategic use of information generated by said systems with a view to enhancing policy making and building capacity within the Public Credit and Treasury Directorate (Dirección General de Crédito Público y del Tesoro Nacional, DGCPTN). 13. Component 5: Project Management (US$1.2 million of IBRD financing). This component was to support the strengthening of MHCP and DNP (including through the provision of operating costs and the carrying out of Project staff training programs) underlying the carrying out of Project coordination and management activities. 1.6 Revised Components 14. In the January 2013 restructuring 6, one of the aims of Subcomponent 4.3 was modified from “Provision of technical assistance to support MCHP with the development, establishment, and operation of the Center for Fiscal and Financial Studies” to “Provision of technical assistance to strengthen the Technical Vice-Ministry, especially the General Directorate of Macro Economic Policy, to conduct sound policy making, through the carrying out of high impact macro-economic and fiscal technical studies”, reflecting the change in PDO#6 carried out at that time. 1.7 Other significant changes 15. Over the course of implementation, three Level 2 restructurings took place following Government requests. The first, in May 2012 7, extended the closing date from December 31, 2012 to December 31, 2013, with the goal of allowing time for the development and implementation of information technology (IT) systems in DGCPTN under Subcomponent 4.2 of the Project. In January 2013, a second restructuring took place, revising two PDO indicators and seven intermediate indicators to better capture Project results. Finally, in December 2013 8, a final restructuring was approved to extend the closing date by 11.5 months to December 15, 2014; redefine selected activities in Component 4.2, and initiate modernization of the DGCPTN via a business re-engineering exercise and design of a new business model; carry out a partial cancellation of loan funds originally allocated to Component 3 given that the component’s activities had already been completed, and reallocate loan funds by expenditure category. (See Annex 11 for details on the reallocation of funds.) 2. Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design and Quality at Entry 16. Project design built upon the work of and experiences learned through the PFM I and II Projects. Lessons learned from these past projects – and in particular PFM II, 6 Restructuring Paper No. 74936-CO, January 30, 2013. 7 Restructuring Paper No. 67181-CO, restructuring approved on May 9, 2012. 8 Restructuring Paper No. RES10165, December 13, 2013 4 which included work with the same three agencies (MHCP, DNP, and DIAN) involved in the current Project - were incorporated into design. In doing so, the Project endeavored to focus upon objectives that would be achievable in the short-term timeline of project implementation (rather than structural changes that often take much longer to realize). The Project was expected to be implemented over the course of three years (with an initial expected closing date of December 31, 2012). While many of the Project activities were largely completed by the end of 2012, total project duration was almost two years longer to allow for completion of the activities under Component 4.2, following their revision to ensure the subcomponent’s continuing relevance to the client in the December 2013 restructuring. 17. The Project Administration Unit (PAU) (housed within the MHCP) had been responsible for implementation of PFM II, allowing it to build upon its experience coordinating the required inter-institutional implementation arrangements, as well as familiarity with Bank rules and procedures. As in PFM II, fiduciary responsibilities (procurement, financial management and disbursement) were the responsibility of each of the three participating entities through their respective line units and technical staff, with all reporting to the MHCP. Technical committees were created within each implementing agency with the purpose of ensuring coordination at the technical level. 18. The Project structure (both in terms of the overall organization of components, as well as the activities within shared components) was clearly divided by implementing agency. The MHCP was responsible for implementation of Component 1 and Subcomponents 4.1, 4.2, 4.3, 4.4, 4.5 and 5 (jointly with DNP), as well as overall project coordination in close cooperation with DNP and DIAN. DNP was responsible for Component 2 and Sub-components 4.1, 4.3 and 5 (jointly with MHCP); and DIAN for Component 3. 19. The risk assessment was extensive, identifying a number of potential risks, including a change in authorities and complex implementation arrangements. Following the challenges associated with the implementation of SIIF-Nación in 2009, the risk assessment associated with Component 1 was very comprehensive, and reflected a detailed understanding of the challenges that can arise with the implementation of such systems and the appropriate mitigating actions (the effectiveness of which can be seen in timely completion of the system’s development and implementation). Fewer risks were identified in relation to the other components. Given some of the challenges later encountered (specifically related to Component 4.2, discussed in Section 2.2), it seems that implementation could have perhaps been smoothed through a more careful consideration of the focus at design, or at least, a deeper analysis of the associated risks and the potential strategies for their mitigation. 2.2 Implementation 20. Overall, implementation progressed quite smoothly, with the majority of Project activities being completed more or less within the expected timeline. Component 1 5 21. Given the identified challenges and corresponding risks associated with finishing the development of SIIF-Nación and its rollout within Component 1, particular attention was paid to such activities from the very beginning of implementation in order to ensure proactive support to any challenges encountered. As recommended by the Bank, soon after effectiveness, the MHCP created a team of international experts with experience in the implementation of similar systems to work side-by-side with the Ministry’s technical team, evaluate the system’s current status, and propose the measures necessary to ensure its operation and launch. Following the diagnosis of key issues that could impact the system’s performance in the short-run, by June 2010, a plan for the system’s adjustment and launching was prepared 9 and followed up upon. As a result of this proactive and responsive strategy, the system was successfully launched in January 2011, which in fact served as the basis for the launching of the Government’s Economic Transparency Portal 10 later that year. 22. The successful advancement of SIIF-Nación presented the Government and the Bank with the opportunity to advance on a related agenda - the introduction of a Single Treasury Account (STA). Given that the laws needed to create the legal basis for an STA had not yet been passed at the time of Project design, such work had not been explicitly mentioned at that time. However, following the passing of such legislation in 2011, and given its alignment with the scope of Component 1’s activities - work was undertaken to support the development of the STA’s conceptual model (covering modules for parameterization, payments, and consolidation). This work was leveraged to initiate the related necessary adjustments to the SIIF, and the payments module was launched in January 2015. 23. Activities within Component 1.2 also proceeded largely as planned, with the MHCP obtaining quality management certifications (ISO 9001:2008 and NTCGP 1000:2009) in July 2010 (maintained through a monitoring audit in 2011), developing qualified internal auditors of process quality through training, measuring gaps in the behavioral competencies to support human resources management, and implementing the Integrated Management Monitoring System (Sistema de Monitoreo de la Gestión Integral, SMGI), among other activities. However, no advances were made within the Project regarding the adjustments to the MHCP’s organizational structure to incorporate the Unified Management System’s (Sistema Único de Gestión, SUG) plan, as an analysis for organizational change was instead undertaken and initiated as part of the MHCP’s work under the 2010-2014 National Development plan. Additionally, as a result of the new functions assigned to the MHCP and the expected structural reorganization, it was decided to postpone the external audit to recertify the SUG for three years until after the finalization of the new structure (still pending). Component 2 9 Aide Memoire, Supervision Missions March 14-18, April 14-17, and June 14-18, 2010. 10 The portal provides consolidated budget information to the public, with information posted one day after it becomes available. 6 24. Work within Component 2 proceeded as planned, with most activities being executed in 2010 and 2011. By the time of the mid-term review, SUIFP was consolidated, and activities were focused on promoting its diffusion, with efforts being made to improve the usability of the information systems. As of Project closure, the DNP was continuing to work on the automatization of processes related to the registry and approval of budget processes. Component 3: 25. In turn, while achieving a number of its objectives, implementation of Component 3 did encounter some challenges. 26. Project design was primarily focused on the further development and consolidation of MUISCA, with a focus on modernizing aspects of customs administration. In 2011, the project did not advance as quickly as expected. Legal changes in law mid-way through implementation also contributed to slower than expected progress as DIAN’s attention and internal resources needed to be directed away from the development of new capabilities and towards the continuing operation of existing functions. 27. Imprecision in the original definition of results indicators contributed to difficulties in measuring project progress within the component. As a result, in mid-2013, a series of methodological clarifications to three intermediate indicators 11 and one PDO indicator 12 measurements were discussed and agreed upon by the Bank and the Government. Among the methodological revisions was a clarification to the type of channel for which the PDO indicator related to customs clearance time would be measured (electronic payments, without physical or document inspection), as well as the stage at which a high impact procedures could be considered to have been modernized (in this case, it was determined that processes would be considered modernized when the physical advance was at least 50 percent and there were enough operational elements to allow it to be put into production and its provision of a significant number of services to external and internal users). 28. In spite of advances (detailed in Section 3), implementation progress was ultimately limited by the slower than expected implementation of a new Customs Statute, which has prevented the full rollout of advances such as the complete elimination of paper processes (a change which requires legal modification through the statute before it can be implemented). Additionally, since 2013 there has been an increased focus on further strengthening the robustness and inviolability of the customs system. As a result of both the fact that the current system has more or less fulfilled its natural lifecycle as well as the fact that the security of customs systems needs to be continually revised to 11 The number of high impact DIAN procedures that have been modernized increase from 20 to 40 (100%); Number of Systems Services (Servicios Informáticos Electrónicos) transferred to DIAN’s technology area increases from 12 to 40; and The average number of physical documents required for customs clearance has been reduced from 12 to 5. 12 Average time to clear customs decreases from 48 hours to less than 8 hours. 7 ensure protection against cyberattacks or other associated risks, the Government has decided to look into the possibility of developing a customs module outside of MUISCA that would interact with such. Component 4: 29. Many aspects of Component 4’s implementation also proceeded quite smoothly, with work to pilot interoperability functions between SIIF-Nación and systems including SUIFP, the Electronic Procurement System (Sistema Electrónico de Contratación Pública, SECOP), the Logistic Synergy System (Sinergia Lógistica, SILOG), Public Debt and the Gasoline Tax being successfully implemented, as well as activities targeted toward supporting strengthening of the information collected through the Single Territorial Form (Formulario Único Territorial, FUT) and the DELFOS system, the approval of Decree 4836 (2011) – promoting the use of SIIF and SUIFP, and a web tool serving as a database of unitary costs and values within the public sector being developed, among other things. While the Center for Fiscal and Financial Studies was not able to be implemented as anticipated at design (with neither staff nor budget assigned to the center as planned in 2012 following a change in the Government administration), adjustment to the related PDO indicator in the January 2013 restructuring targeted the completion of twelve studies to strengthen macro-fiscal programming being carried out in MHCP and DNP. 30. However, significant challenges were encountered in the implementation of Subcomponent 4.2 (which targeted the strengthening of the DGCPTN with a focus on the development and implementation of an integrated information and management system) in large part due to an overly broad initial scope of work and numerous changes in leadership. 31. In 2010, implementation was put on hold due to the upcoming change in administration, and delays in the re-initiation of work after the transition were encountered while the Government familiarized itself with the work and revised progress to date, including an International Monetary Fund (IMF) diagnostic. The Internationally Competitive Bidding Process (ICB) for the system was launched in January 2012, but due to a mismatch between the broad scope of work planned and the available budget, it was determined that none of the bids met the requirements and the process was cancelled without awarding of the contract. 32. Following this, the Bank and the Government worked to re-orient the strategy pursued under the subcomponent in order to ensure its relevance and alignment with Government needs, and in 2013, it was decided to focus on the optimization of the DGCPTN’s business model. While the plan for how to do so suffered from some changes (contributing to implementation delays) as a result of changes in administration, under the 11.5 month extension of the closing date in December 2013, it was ultimately decided to contract an individual to design the DGCPTN’s strategic framework, a recommended business model, and an implementation strategy for proposed business model, work which was successfully completed in the second half of 2014. 2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization 8 33. Overall, the design, implementation, and utilization of the monitoring and evaluation (M&E) arrangements are considered to be satisfactory. 34. Overall, M&E design is considered to have been satisfactory. The monitoring and evaluation framework was comprehensive, with the set of indicators being overall well- defined to measure progress in each of the Project’s areas. In retrospect, given the methodological adjustments necessary for measurement of some indicators within Component 3, it seems that some aspects could have benefited from a deeper analysis at design. However, these challenges were limited, and corrected during implementation. 35. Implementation and utilization of the M&E arrangements is also considered to have been satisfactory. M&E was carried out by a coordinator in each executing agency, with the MHCP leading coordination between the three entities for the overall project. A technical committee met every three months to monitor the Project, approve annual operational plans and any changes to financing. Progress reports were provided to the Bank regularly - and with greater frequency than stipulated by the Loan Agreement 13 - providing both the Bank and the counterpart with important information on the advancement toward project objectives. 2.4 Safeguard and Fiduciary Compliance 36. Financial management was considered to be Satisfactory through June 2011 (through Implementation Status and Results Report (ISR) #3). In January 2012 (ISR#4), FM was downgraded to Moderately Unsatisfactory based on the results of a FM supervision mission in August 2011, due to audit report findings that indicated moderate shortcomings in financial management that jeopardized capacity to provide timely and reliable information to manage and monitor project implementation; due to the ongoing implementation of the accounting module in SIIF-Nación, executing entities had not been able to generate their financial reports as of March of that year. The rating was raised to Moderately Satisfactory in August 2012 (ISR#5), as moderate shortcomings in financial management continued to exist but did not prevent the timely and reliable provision of information for project management. Issues with SIIF-Nación continued to pose some challenges for financial management, and the rating was maintained at Moderately Satisfactory for the remainder of implementation. 37. Procurement was satisfactory through June 2011 (ISR#3), and downgraded to MS in January 2012 (ISR#4) due to the risks at that time associated with DNP needing to appoint a procurement coordinator for the project. The rating was maintained during the next ISR due to issues with procurement planning and monitoring, and an increase in the 13 In the October 2010 mission, the Government and the Bank agreed upon submission of progress reports twice a year instead of the one per year required under the Loan Agreement. 9 number of directly contracted consultants, and remained moderately satisfactory throughout the remainder of implementation. 2.5 Post-completion Operation/Next Phase 38. With many areas of work under the Project having been completed by 2012, there is already significant evidence of the continuation of activities well-beyond the scope of the Project. SIIF-Nación continues to successfully operate, and the Bank and the Government have discussed the possibility of financing an extended system, the rollout of the implementation of the new business model developed under Subcomponent 4.2 of the current Project for the DGCPTN and the supporting information system, as well as the development of integration interfaces for external systems (important for the next phase of work to further institutionalize interoperability). Similarly, the SUIFP and MUISCA systems continue to be operated by DNP and DIAN (respectively). Both agencies are continuing efforts to develop additional capabilities and enhance their systems’ interoperability, and DIAN is also now exploring the development of a customs module outside of MUISCA. The Customs Statute is expected to be approved in the near future, allowing for advances in paperless processing, and initial discussions between the Bank and the Government have taken place regarding the possibility of continuing Bank support to DIAN through a reimbursable advisory service. 3. Assessment of Outcomes 3.1 Relevance of Objectives, Design and Implementation 39. Overall relevance is considered to be substantial. The Project was fully aligned with Engagement Area 3 (Inclusive Growth with Enhanced Productivity) of the CPS for FY12-16 14 , which included “Improved Public Sector Management and Equity and Efficiency of Economic Policies” as a Results Area” with Outcome 1 targeting enhanced public sector efficiency and transparency by strengthening and expanding individual management information systems that are the building blocks of an integrated, performance-informed management model and specifically mentioning the Project (and its progress contributing to the subsequent achievement of the corresponding indicator in the 2014 CPS Progress Report). 15 The objectives remain of high relevance to the Government; the Government and the Bank are discussing the potential for a future project that would continue deepening this agenda and aligns with the country’s current OECD accession process. 40. The relevance of design and implementation is also considered to be substantial. Design targeted the consolidation of systems developed through prior projects, as well as activities that were complementary to these efforts to further strengthen key public information management systems in Colombia. While Component 3’s assumption at 14 Country Partnership Strategy for the Republic of Colombia for the Period FY2012-2016, Report No. 60620-CO, June 11, 2011. Discussed by the Board on July 21, 2011. 15 Country Partnership Strategy Progress Report for Colombia for the Period FY12-FY16, Report No. 83966-CO, April 16, 2014. 10 design that customs would be fully implemented within MUISCA did not maintain such high relevance, the Government’s continuing commitment to modernizing and consolidating customs administration (albeit in a separate system) highlights the strong relevance of the objectives pursued by design. Project implementation was highly responsive to the realities encountered, proactively advancing on work critical to project objectives and making the adjustments necessary when challenges that were encountered. 3.2 Achievement of Project Development Objectives 41. Overall, the Project demonstrates a substantial level of achievement of its objective to solidify and expand the gains achieved by the Borrower under PFM II and further improve the transparency, coverage, operation and Interoperability of the Borrower’s national public management information systems to allow the Borrower’s decision-makers to link performance measures with planning, budgeting, and other managerial actions. (Please see Annex 12 for a summary of achievement of PDO indicators.) 42. The objective related to improving the transparency of the Government’s national public management information systems is considered to have been substantially achieved. The successful launch of SIIF-Nación facilitated the development and launching of the Economic Transparency Portal 16 , through which 100 percent of consolidated budget information (including information on income, expenditures, contracting, accounting and regional level data) is published and accessible to the public one day after it becomes available (achievement of PDO#3). The system has helped to strengthen accountability, providing important information for decision-making to government entities, including the President, and moving ahead emphasis is being placed on ensuring that the information is not only available to the public, but that it is also presented in a meaningful way. Similarly, SUIFP has also contributed to transparency in the public investment process, providing timely and good quality information on all stages of public investment (formulation, programming, execution and monitoring). The indicator measuring transparency in DIAN (Intermediate Indicator #13) was not achieved during Project implementation due to the lack of measurement of the targeted index (measurement was not within DIAN’s competencies). 43. In regards to the objective related to improving the coverage of national public management information systems, the objective is considered to have been substantially achieved. With the issuance of Decree 4836 (2011), SIIF and SUIFP were made the primary sources for decision-making related to the budget. With the launching of SIIF- Nación, 100 percent of information on budget execution is now available on a daily basis (achievement of PDO#1) and consolidated budget information available through the Economic Transparency Portal (achievement of PDO#3). The system covers all central government entities, and has also been implemented in 105 subnational entities. With the passing of Decree 2844 (2010), use of SUIFP (which now covers all stages of the investment process, from formulation to monitoring as well as components on budget 16 www.pte.gov.co 11 project and modifications 17) is required by all entities to present projects, and the ratio between the number of projects programmed and the number of projects registered has risen to 122 percent. Additionally, new functions added within the BPIN module allow for definition and monitoring of the value chain associated with investment projects, and a database for registry of investment projects presented for financing through royalties funds was also developed, facilitating compliance with existing agreements. In DIAN, 66 services - covering areas such as implementation of price controls and monthly reports within selective imports, controls and auditing actions, institutional procedures for process management, among others - have been developed and transferred to the technological area. 44. The objective related to improving the operation of national public information systems is also considered to have been substantially achieved. SIIF-Nación, which allows for standardized, real-time public accounting, has facilitated much more rapid production of information, with aggregate information on the central government entities and 105 subnationals now available within one day; previously, it took 35 days for decentralized entities (achievement of PDO#2). The system is now much faster, more stable and efficient, with the most complex process - which previously took seven minutes – now only taking 81 seconds, and covers functions including budget programming, administration of appropriations, annual procurement plans, expenditure management, management of treasury payments, management of treasury revenue, revenue management and accounting management. As previously mentioned, the conceptual model for the STA was also developed under the Project, and is now in the process of being developed and implemented. SUIFP has also brought about improvements in the time related to the budget cycle and responses related to the various stages in the public investment process. Efforts were also made to ensure the usability of the system, with training having been delivered and a focus placed on improving system interfaces and the performance of key processes. Advances in operation can also be seen in the tax and customs system; average customs clearance time has decreased from 48 hours to 6.7 hours with electronic payment (achievement of PDO#4), tax administration efficiency (as measured by total tax collection/DIAN’s executed budget) increased from 103.2 to 106.03 (achievement of PDO#5), and more than 560,000 pending obligations normalized. 45. Other advances contributing to improved operation include the completion of studies to strengthen macro-fiscal programming in both MHCP and DNP (achievement of PDO#6), providing the Government with important information and contributing to overall better macro-fiscal management, as well as the quality certification (and subsequent maintenance of such) of SUG (although as discussed in Section 2.2, this certification has not been maintained), and progress was also made in closing HR competency gaps and the SMGI implemented, making it the single repository of management information in the MHCP and facilitating reporting and the implementation of SUG processes. While work related to the DGCTPN did not advance as expected and 17 Work to automate processes associated with budget procedures that take place during project execution is currently underway. 12 the planned system was not able to be implemented, important progress was made in defining the business model, and is expected to be continued in the future. Other important advances under the Project that have contributed to the better operation of public information systems include work on budget classifiers, and the development of a Bank of Unitary Values and Costs. 46. Finally, the objective related to improving the interoperability of national public information systems is also considered to have been substantially achieved. The information in SIIF-Nación and SUIFP on budget allocations and execution is now fully consistent (achievement of PDO#7). There is 100 percent interoperability between SIIF- Nación and the Bank of the Republic’s payment system, as well as with SECOP (in the areas of CDP, commitments and future fiscal years) and MHCP’s Public Debt (through flat files used to make debt payments. Additionally, pilots were carried out to integrate the data generated by different government systems (between SIIF-Nación and SUIFP, SECOP, the Ministry of Defense’s SILOG, and the Gasoline Tax system). While given the evolving nature of the different systems involved in this effort, work is still needed to continue to develop (and completely institutionalize) full interoperability, the Project’s achievements reflect important advances in this process, and are expected to serve as the foundation for future work. 3.3 Efficiency Overall project efficiency is considered to have been substantial. In 2011, the MHCP carried out initial calculations to estimate the value of annual savings resulting from the implementation of SIIF-Nación. Four areas of savings were identified by the Government, summing to a total of approximately US$105.75 million (COP$205 billion) 18: 1) other public entities no longer needed to develop, acquire, and maintain their own financial management systems (savings of COP$40 billion, U.S$20.63 million); 2) no longer utilizing paper checks (COP$10 billion, US$5.16 million dollars); 3) direct wage transfers to the final beneficiaries in medium and small entities (COP$21 billion, US$10.83 million); and 4) direct transfers of central government resources to the entities executing investment projects (COP$134 billion, US$69.12 million). (See Annex 3 for a description of how the Government calculated these estimations.) The project has also contributed to the operational efficiency of the Government through a variety of channels, which can be seen in a number of ways, including the ability to process payments to contractors more quickly, improved response times of the SIIF-Nación help desk and of public investment processes, and the reduction in customs clearance time to less than seven hours with electronic payments. 47. These savings suggest that project efficiency was quite high. With Project costs (both Bank and Borrower resources) of US$33.85 million, estimated annual savings in each year since 2011 have exceeded total project costs by a factor of more than three. 18 A presentation reporting these estimated savings can be found here: http://www.minhacienda.gov.co/portal/page/portal/HomeMinhacienda/siif/ObjetivoContenido/ASPECTOS %20GENERALES%20david%20morales%20final.pdf 13 Even considering all of the resources also devoted to related work in the PFM I and II projects – and assuming that no savings were realized from work prior to 2011 (a strong assumption) – total costs (US$147.35 million) for all three projects would have been more than covered by the savings generated through use of SIIF-Nación in 2011 and 2012 alone. In terms of implementation efficiency, at U.S. $33.85 million (including both Bank and counterpart funds) project costs also appear to be in line with the average costs of completed IFMIS projects in a database analyzed in Dener et al (2011) (including ICT cost, advisory support, training and project management) – approximately U.S. $25 million. 19 3.4 Justification of Overall Outcome Rating 48. Rating: Satisfactory. The Project’s overall outcome is considered to be Satisfactory (See Table 1 below). The Project made substantial progress towards its development objective of solidifying and expanding the gains achieved under PFM II and further improving the transparency, coverage, operation and interoperability of the Borrower’s national public management information systems over the course of implementation. The advances in SIIF-Nación, SUIFP and MUISCA described above have strengthened and expanded key elements of Colombia’s public management information systems, enabling the full use of such information for strategic decision making (with Government entities including the President using information from SIIF- Nación) and the implementation of interoperability capacities and data integration pilots between key systems. 49. The operation’s objectives remain highly relevant for both the Government and the Bank, and given the substantial relevance of design and implementation, overall relevance is considered substantial. Estimated annual savings from implementation of SIIF-Nación exceed the costs of the entire Project, and if realized over only two years, would more than cover the costs of this Project plus PFMI and II, leading to an overall assessment of substantial Project efficiency. In the context of declining commodity prices (and their subsequent impact on fiscal space), the increased efficiency of public expenditure enabled by Project advances and the fiscal savings generated will become even more important in the future. 19 Cem Dener, Joanna Alexandra Watkins and William Leslie Dorotinksy, Financial Management Information Systems: 25 Years of World Bank Experience on What Works and What Doesn’t, The World Bank, Washington, D.C., 2011. On page 19 14 Table 1. Overall Outcome Rating Item Rating - Relevance Substantial Relevance of Objective High Relevance of Design/Implementation Substantial - Efficacy Substantial - Efficiency Substantial Overall Satisfactory 3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development 50. As a project focused on consolidating key public information management systems at the national level in Colombia, this Project was not necessarily designed to have a direct impact on poverty, gender, or social development issues. Nevertheless, the more effective and efficient management of public resources that it was expected to generate would likely create additional fiscal space to address national priorities such as these, as well as provide tools to monitor the effective direction of resources towards such. (b) Institutional Change/Strengthening 51. The Project’s successful work on strengthening national public information systems in order to increase the availability, accessibility and quality of public information has had a significant impact on institutional strengthening, enhancing transparency, facilitating public access to information, and increasing accountability. In particular, work on the SUG has helped to support the creation of a culture of internal audit and introspection within the MHCP - something of even greater importance now given the temporary pause in external recertification efforts of quality control mechanisms. Systems such as SIIF-Nación, SUIFP, and the Economic Transparency Portal have also significantly contributed to institutional strengthening in Colombia by making increasingly more information about Government operations available to the public, and future efforts point towards ensuring that – in addition to be available – information is delivered in such a way that it is meaningful to a broad range of citizens. 52. Additionally, some of the challenges encountered during implementation – particularly those related to Component 4.2 – led to a process of introspection that has ultimately contributed to the MHCP’s institutional strengthening. The lessons learned in relation to activity planning, and the need to prioritize and clearly coordinate different entities needs in order to arrive at a viable output appear to have been well internalized within the implementing team, and are likely to contribute to more effective planning and execution in the future. (c) Other Unintended Outcomes and Impacts (positive or negative) 53. Progress on the STA is considered to be a positive unintended impact of the Project. As discussed above in Section 2, the successful implementation of SIIF-Nación combined with the passing of legislation in 2011 provided the opportunity for the Project to support the development of the STA’s conceptual model – an important agenda for the 15 Government. These advances were then leveraged to initiate the related necessary adjustments to the SIIF, and launch the payments module. 3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops 54. A survey measuring perceptions of the quality of services provided through SIIF- Nación was carried out in January 2015. The online survey, which was targeted at users registered in training activities between April and December 2014 (3,590 users), received 859 complete responses. The survey found that 92 percent of users reported that the administration of SIIF-Nación did a good or better job of meeting their expectations. 94.6 percent found the software services to be good or better; 88.8 percent that the help desk provided good or better services; 90.8 percent that training was good or better; and 92 percent that information on the web page was good or better. 4. Assessment of Risk to Development Outcome Rating: Moderate 55. Overall, risks to the development outcome are considered to be moderate. Given that many of the Project-funded activities were completed by 2012, and that the Government has continued related and complementary work since through other sources of funding, the continuing relevance and importance of such work is clear. Systems such as SIIF-Nación and SUIFP exhibit stable performance and appear to have been well- institutionalized, suggesting that risks associated with the systems are low. DIAN is committed to continuing work to move towards an integrated customs process (albeit within a new system) – work that is well-aligned with the overall development objective of the operation, if not the specific objective of Component 3 – and there have been initial discussions with the Bank regarding its provision of strategic support through a counterpart funded activity. 56. However, some risks to the development outcome remain. As previously noted, re-certification of the SUG is currently on hold until the completion of MHCP’s restructuring process. While this does not mean that oversight of the management system is not being exercised - the practice of engaging in internal audits has been successfully institutionalized and a supportive culture created - the resumption of external recertification will be important for ensuring the maintenance of external control mechanisms. Interoperability capacities were developed and piloted between SIIF and other key national systems (e.g. SUIFP, SILOG, etc), achieving the objective of developing the tools to enable such. However, given individual system’s continuing evolution and the nature of interoperability efforts in general, how to move forward in the further institutionalization of these capacities will be an important agenda for the next phase of work. 16 5. Assessment of Bank and Borrower Performance 5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry Rating: Moderately Satisfactory 57. The Bank team brought substantial, context specific knowledge to Project design, building upon lessons learned from the PFM I and II operations. Project design represented the key pending areas of work necessary for the consolidation of national public information systems, and demonstrated a strong understanding of the potential risks and necessary mitigating measures, particularly in regards to Component 1. 58. However, there were some challenges which were not anticipated at design – particularly related to Component 3 in terms of the definition of indicators, slower than expected approval of the Customs Statute, and the Government’s decision to explore development of a new module outside of MUISCA –as well as Component 4.2 - which could have been perhaps lessened or prevented through greater consideration of risks at the time of design. (b) Quality of Supervision Rating: Satisfactory 59. Over Project implementation, the Bank team was very responsive, providing the hands-on and proactive support which helped to ensure the success of Project activities and the Project’s continuing relevance during a period of political transition. This support was particularly critical for the successful launching of SIIF-Nación, and following its launch, the team played a critical role in advancing dialogue on the implementation of a STA. The task team benefited from strong consistency over the years, with a stable composition of members that were involved in both the preceding project’s implementation as well as throughout the entirety of the current Project. When challenges presented themselves, the Bank team demonstrated flexibility and worked with the Borrower to find solutions that fit the revised context, ensured continuing relevance to the Government, and maximized progress towards the development objective (e.g. clarification/modification of indicators, adjustments to Subcomponent 4.2, etc.). (c) Justification of Rating for Overall Bank Performance Rating: Moderately Satisfactory 60. Given the overall high relevance of project design as well as the flexible and proactive support provided during supervision but in light of the fact that design could have better anticipated some Project risks which materialized during implementation, overall Bank performance is considered to be moderately satisfactory. 5.2 Borrower Performance (a) Government Performance Rating: Satisfactory 17 61. Strong Government support for the overarching initiatives supported through the Project was essential to the high achievement of Project Objectives. Given the inter- institutional nature of the Project, this support was particularly critical. Changes in administration did lead to some delays in implementation (particularly impacting Component 4.2), as well as pauses in some initiatives (e.g. the pending recertification of the SUG System), but in the end, solutions to these issues were found and there appears to be clear commitment to continuing the lines of work pursued under the Project. While interoperability functions were developed through the Project, it seems that the umbrella of additional Government support could have helped to achieve even further advancements in the interoperability agenda under the Project. (b) Implementing Agency or Agencies Performance Rating: Moderately Satisfactory 62. The three implementing agencies’ (MHCP, DIAN, and DNP) approach to project implementation ensured the overall successful implementation of the Project in spite of unanticipated challenges that were encountered. The three worked proactively to adjust project activities to shifts in priorities and institutional structures, finding forms that were both aligned with the achievement of Project objectives and institutional realities. Demonstrating a clear understanding of the cultural shifts often required to institutionalize the systems strengthened through the Project, their emphasis on training and communication played a critical role in consolidation efforts. 63. Some fiduciary challenges were encountered during implementation, with moderate shortcomings in financial management and areas for improvement in the area of procurement being noted by the Bank. The implementation of Subcomponent 4.2 also experienced difficulties due to the original breadth of the activity’s scope and shifts in the implementation strategy. (c) Justification of Rating for Overall Borrower Performance Rating: Moderately Satisfactory 64. The Government’s and implementing agencies’ overall demonstrated strong performance, ensuring the successful implementation of Project activities. Given the implementation challenges encountered, overall Borrower performance is considered to be moderately satisfactory. 6. Lessons Learned 65. Continuity in operational teams can have a substantial impact on Projects’ success in achieving development outcomes. Team composition was quite stable (both on the Bank and the Borrower’s side), with a number of team members having also been involved in PFMII. This continuity allowed for implementation to hit the ground running following effectiveness, and helped to support the successful progress forward on activities even when challenges were encountered. 18 66. Similarly, strong coordination between the three agencies (MHCP, DNP, and DIAN), as well as with the Bank, played an important role in smoothing project implementation. Having worked together in the implementation of PFM II, the Project’s institutional arrangements and coordination mechanisms were well-established at launching, helping to avoid challenges that are often observed in these types of projects and ensuring progress towards the development objective. Close and constant contact between the counterpart and Bank team facilitated the provision of high quality technical support and proactive adjustments to the implementation context. 67. Creating and maintaining interoperability – the ability of information systems to have harmonized concepts and data, and automatically exchange information - is a process that requires continual reassessment, evolution, and a phased approach. Efforts undertaken within this Project to connect the national government’s various public information systems represent an important first step towards interoperability in Colombia, with significant advances having been made. It will be important to continue to work on the next phase of this agenda in the future and further strengthen its institutionalization through the creation of national standards. 68. Thorough and complete risk assessments - and the ability to adjust quickly when challenges arise - can play a critical role in ensuring the successful implementation of complicated activities. While there were a number of challenges that needed to be overcome in order to successfully complete implementation of SIIF- Nación, their clear identification at design allowed for the proactive rollout of actions designed to mitigate these risks and ensured the system’s launch in 2011. 69. When designing a component or subcomponent that is highly concentrated in IT investment, it is important to either engage in a deep analysis of the desired functionalities during the preparation phase in order to ensure a clear understanding of the scope of the system, or to build in some flexibility in the allocation of Project funds in the event of greater than expected costs. In the case of Subcomponent 4.2, the lack of initial clarity in the system’s scope contributed to delays in implementation, and the eventual need to redesign the planned activities. 70. Loans do not have to be big to have a big impact. While relatively small in size compared to other World Bank loans to middle income countries, this Project had a large impact thanks to its high relevance to the Government, strong implementation arrangements, and the application of the Bank’s past lessons learned from implementation of integrated financial management systems. 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners (a) Borrower/implementing agencies 71. As a part of the last Interim Financial Report (IFR), the Borrower prepared a substantial closing report presenting each component’s outputs, main achievements, lessons learned, as well as a description of the implementation process, which served as an important input to this ICR. No comments on the ICR were received prior to its finalization. 19 (b) Cofinanciers N/A (c) Other partners and stakeholders N/A 20 Annex 1. Project Costs and Financing (a) Project Cost by Component (in USD Million equivalent) Actual/Latest Appraisal Estimate Percentage of Components Estimate (USD (USD millions) 20 Appraisal millions) 21 Component 1 8.8 13.17 149.66 Component 2 1.2 1.17 97.5 Component 3 8.0 14.68 183.5 Component 4 5.8 3.18 54.82 Component 5 1.2 1.65 137.5 Total Baseline Cost 25.00 33.85 135.4 Physical Contingencies 0.00 0.00 0.00 Price Contingencies 0.00 0.00 0.00 Total Project Costs 25.00 33.85 Front-end fee PPF 0.00 0.00 .00 Front-end fee IBRD 0.00 0.00 .00 Total Financing Required 25.00 33.85 135.4 (b) Financing Appraisal Actual/Latest Type of Estimate Estimate Percentage of Source of Funds Cofinancing (USD (USD Appraisal millions) millions) Borrower 22 0.00 13.32 International Bank for Reconstruction 25.00 20.42 81.68 and Developmen 23t 20 Source: PAD 21 Source: Government of Colombia, IFRs Sustantivo a diciembre 31 de 201, April 2015 22 Source: Government of Colombia, IFRs Sustantivo a diciembre 31 de 201, April 2015 23 Source: World Bank Client Connection 21 Annex 2. Outputs by Component Component 1 • SIIF operational and in production with the following functionalities: budget programming, administration of appropriations, annual procurement plan, expenditure management, management of Treasury payments, management of Treasury revenue, income management and accounting management. • User services provided through a call center to address operational, functional, and technical issues and provide support. • Optimization and adjustments to the SIIF-II, allowing for a 90% improvement in the system’s response time. • Sustainability of SIIF-Nación is guaranteed through functional financing for maintenance costs. • Training mechanisms (both in person and virtual) have been developed to close knowledge gaps among users of the system • IT infrastructure necessary for launching and effective functioning of SIIF-Nación supported through acquisition of software licenses. Additionally, an alternate site was established (using resources other than project resources) to provide data security and protect the system. This work included the construction of a back-up location and the development of security and emergency protocols. • Economic Transparency Portal now available to the public, allowing for online access to information on each stage of the State’s budget execution. The system incentivizes a culture of transparency on the part of the State and society, and supports citizen oversight of public resources as well as citizen participation in the decision-making process. • ICONTEC and IQNet certification in the rule ISO 9001:2008 and in ICONTEC NTCGP 1000:2009 certification in July 2010. • Maintenance of Quality Certification by ICONTEC in the rules 9001:2008 and NTCGP 1000:2009 during 2011 and 2012. • Acquired and implemented the process portal, which is both a tool as well as a methodology for designing and documenting processes, and which facilitates their consultation, administration, and improvement. • Acquired, parameterized and put into production the Integrated Management Monitoring System (SMGI) • Implemented an information system for the administration of human capital in the MHCP, which allows for the administration of personalized development plans and their execution, as well as the evaluation of behavioral competencies and the execution of a survey of climate and culture. • Results obtained from the measurement of external client satisfaction and presented to the High Directorate along with the strategy to define the respective improvement actions. • Now have virtual courses which allow for the training of public servants in the Unified Management System (SUG) and the Integrated Management Monitoring System (SMGI) 22 Component 2 • Four modules of the system covering the public investment cycle (formulation, programming, execution and monitoring) in operation. • New Adjusted General Methodology, through which information on projects is entered into the bank of projects. Allows for orderly registry of information on Project formulation and evaluation, facilitating ex-ante evaluations and supporting decision making. o Methodological Guide for the Formulation of Indicators o Guide for Regionalization of National Investment o Guide for the seven steps in the value chain and frequently asked questions o Manual of SUIFP procedures. • Budget processes through the system, monitoring of investment projects through the system • Physical, financial and chronological monitoring of public investment from three perspectives: sectorial, regional and transversal. Projects with indicators and goals that allow for results to be observed, the preparation of reports, and presentation of information. Indicators allow for measuring objectives, products, activities and ends programmed in the budget. • Virtual course in Public Investment Management • Multimedia course in Project Theory and the Adjusted General Methodology • Virtual course with the National University in Public Investment Management. • Diploma in Public Investment Management with the Escuela Superior de Administración Pública • Manual of budget processes • Memories of the National Project Database • Manuals and guides for system users • Advances in the Red Juntos Guide and the Guide for the National Consolidation Plan • Monitoring reports of information • Manual of National Public Investment, Manual of Procedures for the Management, Programming and Execution of Cooperative Resources, Manual of Procedures of the National Bank of Programs and Projects. • Guide for the Regionalization of Investment Expenditure in the National General Budget, which has typologies for the identification of investment projects which could be regionalized, the methodology for the regional distribution of resources, the timeline of times and activities in the process, and considerations that should be taken into account to develop the exercise during budget programming. • Presentations and documents for dissemination through the DNP’s webpage and network: o Conceptual Framework for the Training Strategy in the Adjusted General Methodology – Royalties o Guide: Preventative simulation of investment and development, and project horizons and cycles. o Guide: Eliminating development limits, identification stage 23 o Guide: Creation of wealth from potential to real, preparation module o Guide: Approximation of change, evaluation module o Guide: Following the development wave, programming module o Users Manual o Manual of Procedures – SUIFP – General Royalties System o Guide of Frequently Asked Questions o Help in the Adjusted General Methodology o In person trainings on Fridays and massive in person trainings on specific themes. o Accompaniment and technical assistance in the national territory involving the development of necessary technology and training in budget topics (formulation and management of projects) to guarantee the agility, consistency and transparency of budget processes and the assignment of expenditures, as well as efficiency in the provision of services by DNP. o Telephone support in conceptual themes, the use of technological tools, the Adjusted General Methodology, and SUIFP’s Bank of Projects, among others • Decree institutionalizing the changes promoted by SUIFP, Decree No. 2844 (August 5, 2010), with which the organic laws of the Budget and National Development Plan were regulated, with the objective of improving information on public investment, and articulating existing information systems in the SUIFP. • Increase in the regularization, standardization, and legal definition related to public investment and the Colombian budget structure. • Interoperability with Sisconpes, Orfeo, Sismeg and SIIF Component 3 • Expansion of DIAN’s Integrated Management System (MUISCA): o Returns and/or compensation o Returns of the Value Added Tax (DAI) o Implementation of Version 2 of selectivity for returns on sales o Implementation of Version 2 of selectivity for returns on income o Simplified consultation on tax obligations o Single Tax Registry – Online registry o Single Tax Registry – 3 Steps o Single Tax Registry – Simplification – account activation o Single Tax Registry – Certified mail service o Single Tax Registry – AVP modules o Single Tax Registry – Strengthening and synchronization of roles (improvement) o Single Tax Registry – Stabilization o Creation of electronic payment subsystem o Complaints and suggestions o Invitations to pay and communication campaigns o Customs transit o Guarantees o Authorization of Customs Users 24 o Certified reports, annulments and adjustments to the Provider o Annulments certified to the provider o Through entry into effect of the tax reform in Law 1607 (2012)  Form 360  Returns - tax reform 2012  Adjustments to various forms (350, 490, 360, 300, 110, 400, 140, 315)  Exogena 2013 and 2014  Contracts on the importation of technology • Strengthening of DIAN’s institutional capacity to support the and facilitate external trade operations, through the development of technological services to: o Implement price controls within selected imports o Implement monthly reporting on selected imports o Authorized Economic Operators • Achievement of effective controls and oversight actions, through the development of technological services: o Official sanctions and liquidations o Customs violators o Program reports for oversight of Form 1561 o Massive loading of information of the District Secretariat of Economy (external) o External loading of natural declarers in 2012. o Integrated Information Service for Tax, Customs, and Exchange Analysis o Fiscal prize – the receipt awards you o Discussion of Administrative Acts (Legal management) • Through entry into effect of the Free Trade Agreement with the United States: o Implementation of control of quotas (first arrival, first served) o Implementation of control of quotas (Export Trade Company) o Reports on use of tariff quotas (first arrival, first served) o Reports on use of tariff quotas (Export Trade Company) o Registry and control for the Rice Tariff Quota • Strengthening of institutional procedures to sustain and develop information technology that facilitates the management of processes which provide services and information security, through the development of technological services for: o DBA consultations o Architecture o Massive loading of information o Migration of the Import Risk Evaluation service from the MINT database to MSELEC o Maintenance, adjustments and tests of the service of risk evaluation o Implementation of price controls within selected imports, o Implementation of monthly reports of selected imports o Electronic services (asynchronous web services) o Web reports (implementation of asynchronous web services for consultations – 2010 o Adding Letterhead to contextual Documents 25 o F1524 – Notification Updates o Web Module for Consultation of Documents o Numbering for Administrative Acts o •Support in the migration of quota control and selectivity services (imports, exports, returns) to the new database platform o Control panel dashboard o Improved presentation layer MUISCA o Strengthening of Architecture - Seeds Project (Key) o Implementation of Web Filtering and monitoring configuration o Control cell International Mobile Station Equipment Identity o Consultation reports for cellular International Mobile Station Equipment Identity • Progress on development of computer services focused on (i) identification of cases of apparent deviation from the corresponding obligations and determination of the fines (sanctions), (ii ) the definition of the legal status and the applicable legal framework regulating goods subject to customs processes (definition of legal status of goods) (iii) tax authorizations, (iv) customs offenders, (v) Control in trade zones (vi) laboratory samples administration (vii) portfolio management, (viii) management, storage and disposal of goods seized, confiscated and abandoned (ix) Institutional representation in litigation and (x) planning, monitoring and evaluation of institutional management • Increases in efficiency of customs administration – importation processes through o Start in the implementation of electronic transmission of certificates of origin with countries like Mexico, Chile, Ecuador and Brazil. o Constant and online consultation and verification of certificates issued by other entities, as well as import licenses and registrations through the Foreign Trade Single Window (VUCE). o Advance electronic receipt of cargo manifests. o Elimination of country of origin export declaration of the goods as a supporting document. o Elimination of prior authorization by DIAN for some goods such as construction machinery for mineral extraction (Decree 723 of 2014). • Within the goal of process and human resources modernization o In 2011, DIAN received certification in the Quality Management System under Norms NTCGP 1000:2009 and ISO 9001:2008 o Plan for Virtual Training for the development of staff competencies in the 15 processes in the entity’s Quality Management System and Internal Control through the following programs: Mission Processes:  Portfolio Management  Control and Settlement  Customer Support  Mass Management  Revenue Collection  Marketing  Legal Management 26  Customs Operations Control Processes  Disciplinary Investigation  Internal Control Support Processes  Financial Resources  Physical Resources  Computer Services  Human Resources Strategic Processes  Corporate Intelligence • 100% (66 services) of coordination of the development of computer services was transferred from external consultants to personnel within the Subdirectorate for Management of Informational and Telecommunication Technologies. Component 4 • Interoperability through web services o SUIFP (DNP). In 2012, a web service was put into production for the process associated with Modification Letters for the budget planning process. o MUISCA (DIAN). DIAN has finished the definition of the requirements for interoperability to generate withholding vouchers, to be carried out once the updating of MUISCA to the current regulations is finished. o Interoperability with the Surcharge System. In the last trimester of 2012, the functional area of the Directorate for Fiscal Support took the decision to modernize the application for the administration of the gasoline and combustible oil for motors surcharge. As such, the work plan for interoperability of SIIF-Nación with the new application is linked to the timeline of implementation of the new system. o Interoperability with SECOP system (Electronic public procurement system). In 2012, the coordination process with the National Contracting Agency was started to define a catalogue of goods and services to be used to acquire and administer goods. In the area of interoperability, there are no requirements defined by the agency. o Interoperability with the SMGI system. The development of the web service was completed by the provider, which was necessary for the consultation of the defined indicators. During the first half of 2013, it was put into operation in a testing environment in MHCP. • Strategic framework for the DGCPTN constructed and validated, defining the mission, vision, values, objectives, and strategic initiatives • Structured a Business Model, and decided to adjust the base model to include three key components important for its impact: 1) services or products; 2) risks; 3) 27 regulations, laws, decrees and other norms. Value chain and process inventory elaborated for each area. • Strategy and plan for implementation of the recommended business model defined, identifying the general requirements for implementation, impacts, and risks by area. A schematic summary was also prepared of the norms and involved areas for proposals for changing the norms proposed in each area’s strategy. • Although not carried out using loan resources, the DGCPTN has consolidated a set of methodological tools that allows for better estimation and analysis of risks as administrator of the Nation’s financial assets and liabilities. o Assessment methodology for contingent liabilities and reconciliations for judicial proceedings against state entities o Methodology for assessment and monitoring of risks in state contracts o Contingent valuation methodology for guarantees in public credit operations o Financial strategy to reduce the fiscal vulnerability of the State in the event of a natural disaster o Allocation methodology for counterparty quotas – National entities - National Entities o Allocation Methodology for counterparty quotas - Foreign entities o Methodology for estimating exchange rates and interest rates o Methodology for determining the optimal policy for debt issuance o Market Risk Methodology VAR o Methodology for estimating cost overruns in road infrastructure projects (PPPs) o Methodology for forecasting oil and gas production. • Study of alternatives related to the selection of the model to reform the General Royalties System (financed with local funds) • Study on monitoring, projections, and analysis of areas including balance of payments, capital account flows, current accounts, and levels of foreign investment • Estimation of the economic impact of norms to be issued, allowing for the forecasting, modeling, and simulation of different macroeconomic variables in the framework of the projected political economy • Conceptual models for fiscal monitoring of the national central government, general government, and consolidated public sector • Collection of information sources for monitoring the health sector • Study om “Implications of the adoption of the Public Financial Statistics Manual of 2001 of the IMF for fiscal statistics: PPPs and leasing” and a document about the analysis of the system that will automate the model for fiscal monitoring, including the description of data sources, general structures of the solution, definition of entries and exits from the system • Study on export promotion policies and measurement of the impact of international exchange rate volatility in the purchasing decisions for international inputs in relation to the productivity of Colombia • Study analyzing the different options for the creation of an area in charge of managing and unifying work histories in territorial entities and in the national 28 government (active or past), and identifying the best place for the development of this objective. • Diagnostic of the databases containing information on work histories and strategies for the unification and validation of such information. • Contracting of different forecasting studies of economic variables, calculation of statistics and economic series, construction and updating of models. Also carried out evaluations of general and specific public policies – including proposed reforms to the pension and health system, proposed tax reforms, trade policies, royalties, policies to reduce business and labor informality, on the effect of the global economic revision on Colombia and the economic impacts of climate change on the economy, among others. • Six DNP staff updated their knowledge of econometric methods • Work on Single Territorial Form and improvement of the DELFOS system, including support to the formulation and application of the Quality Evaluation Framework of Data from the Single Territorial Form, creating a quality system for the Intersectorial Commission of the Single Territorial Form, and adjusting the software development processes corresponding to the DELFOS system to the requirements of the SUG and international standards. • Process Guide for Software Life Cycle prepared for the MHCP, with the agreement to pilot it in the Fiscal Support Directorate • Training of territorial entities, support and training to strategic users of the Single Territorial Form, maintenance of the DELFOS system, and production of viability report of the departments, with the participation of a contractor financed by the subcomponent. • Study trips and workshops for personnel of the Fiscal Support Directorate. • Reinforcement to elaboration and launching of the monitoring and control mechanisms for use of resources from the General Participations System • Action plan to increase the capacities of the Fiscal Support Directorate to monitor and supervise the Public Management System related to the Single Territorial Form and DELFOS • Decree 4836 (2011), supporting the use of SIIF and SUIFP, promoting the use of budget information contained in them as the primary source of information for budget decisions. • Transparency portal with a management dashboard covering executing of resources by entities by fiduciary agreements and inter-administrative covenants, information on appraisals and revenue collection, as well as national income. • Web tool, called the Bank of Unitary Costs and Values, 29 Annex 3. Economic and Financial Analysis As noted in the main text, in 2011, the MHCP carried out initial calculations to estimate the value of annual savings resulting from the implementation of SIIF-Nación. Four areas of savings were identified, summing to a total of approximately U.S. $105.75 million (COP $205 billion): 1) other public entities no longer needed to develop, acquire, and maintain their own financial management systems (savings of COP $40 billion, U.S$ 20.63 million); 2) no longer utilizing paper checks (COP$10 billion, U.S. $5.16 million dollars); 3) direct wage transfers to the final beneficiaries in medium and small entities (COP $21 billion, U.S.$10.83 million); and 4) direct transfers of central government resources to the entities executing investment projects (COP $134 billion, U.S. $69.12 million). Annual savings from the first were calculated by determining the entities that had previously used their own financial management systems, but had transitioned to using SIIF-Nación and no longer needed to develop or maintain their own systems. The 2011 budget was then examined to determine the value of the budget allocated to investment projects in all types of software (approximately COP $100 billion), and several entities asked the percentage of their software spending which had previously been destined to financial management systems. Based on their response that they had used approximately 40 percent of these funds for their financial management systems, it was estimated that the same ratio was assumed to apply to the total COP $100 billion allocated to software investment across the Government). 24 Annual savings from the second source originated from payments made by entities which had joined SIIF, and were no longer making payments via physical checks, with MHCP estimating that approximately 4 million physical checks had been issued each year, at a cost of 2,500 Colombian pesos each. Annual savings from the third and the fourth sources reflect estimates of savings from no longer making wage and investment transfers to beneficiaries through commercial banks (which required reciprocity of approximately five days), but instead making the transfer directly. These savings suggest that project efficiency was substantial. With Project costs (both Bank and Borrower resources) of U.S. $33.85 million, estimated annual savings in each year since 2011 have exceeded total project costs by a factor of more than three. Even considering all of the resources also devoted to related work in the PFM I and II projects – and assuming that no savings were realized from work prior to 2011 (a strong assumption) - total costs (U.S. $147.35 million) for all three projects would have been more than covered by the savings generated through use of SIIF-Nación in 2011 and 2012 alone. In terms of implementation efficiency, at U.S. $33.85 million (including both Bank and counterpart funds) project costs also appear to be in line with the average 24 As the budget lines considered from the 2011 budget only included investment projects (i.e. they did not include maintenance costs), this estimate is believed to represent a sub-estimation of savings of this nature. 30 costs of completed IFMIS projects in a database analyzed in Dener et al (2011) (including ICT cost, advisory support, training and project management) – approximately U.S. $25 million.25 25 Cem Dener, Joanna Alexandra Watkins and William Leslie Dorotinksy, Financial Management Information Systems: 25 Years of World Bank Experience on What Works and What Doesn’t, The World Bank, Washington, D.C., 2011. On page 19 31 Annex 4. Bank Lending and Implementation Support/Supervision Processes (a) Task Team members Responsibility/ Names Title Unit Specialty Lending Amparo Ballivian Lead Economist LCSPP Fabian Carrillo Consultant LCSPS Manuel Fernando Castro Senior Evaluation Officer IEGCS Cem Dener Senior Public Sector Specialist ECSP4 Jeannette Estupinan Sr Financial Management Specialist LCSFM Enrique Fanta Ivanovic Senior Public Sector Specialisist LCSPS Henry Forero Ramirez Senior Information Officer TWICT Thomas Columkill Garrity Public Sector Specialist PRMPS America Teresa Genta Fons Lead Counsel LEGLA Christian Yves Gonzalez Economist LCSPE Amador Jong Wook Lee E T Consultant PRMPS Gladys C. Lopez-Acevedo Senior Economist LCSPP Karla Lopez Flores Language Program Assistant LCSPS Javier Julio Madalengoitia Consultant LCSPS Alencastre Jose M. Martinez Senior Procurement Specialist ECSO2 Maria Del Carmen Minoso Senior Operations Officer ECSO1 Edgardo Mosqueira Medina Sr Public Sector Spec. ECSP4 TTL Geoffrey Shepherd Consultant LCSPS Milton M. Von Hesse Consultant CLALA Maria Cecilia Zanetta Consultant HDNED Carmen Zuleta Calderon Consultant LCSPS Supervision/ICR Pedro Arizti Senior Public Sector Specialist GGODR TTL Gustavo Adolfo Canales Consultant LCSPS Azul Del Villar Baston Consultant LCSPS Senior Financial Management Jeannette Estupinan GGODR Specialist Enrique Fanta Ivanovic Senior Public Sector Specialist GGODR Henry Forero Ramirez Senior Public Sector Specialist GGODR TTL Sheila Grandio ET Consultant GGODR Alejandro Guerrero Ruiz Adrienne Elizabeth Hathaway Research Analyst Arturo Herrera Gutierrez Practice Manager GGODR Eduardo Huerta-Mercado Consultant GGODR Herrera Raul Felix Junquera-Varela Lead Public Sector Specialist GGODR Eguiar Lizundia Gonzalez Consultant GGODR 32 Carmen Machicado Operations Officer GEEDR Jose M. Martinez Senior Procurement Specialist LCSPT Lars Christian Moller Program Leader AFCE3 Silvia Moran-Porche Senior Financial Management Xiomara A. Morel GGODR Specialist Lead Public Sector Development Edgardo Mosqueira Medina GGODR TTL Specialist Ronald E. Myers Consultant GSURR Jenni Amanda Pajunen ET Consultant LCSPS Monica Penuela Jaramillo Consultant GGODR Jose M. Rodriguez Alvarez Senior Public Sector Specialist GGODR David Santos Ruano ET Consultant GGODR Geoffrey Shepherd Consultant LCSPS Maria Fernanda Tellez Consultant GGODR Fanny Weiner Public Sector Specialist GGODR (b) Staff Time and Cost Staff Time and Cost (Bank Budget Only) Stage of Project Cycle USD Thousands (including No. of staff weeks travel and consultant costs) Lending FY2009 5.93 63,469 FY2010 20.09 147,297 Total: 26.02 210,765 Supervision/ICR FY2010 18.44 131,049 FY2011 40.48 218,826 FY2012 24.63 184,024 FY2013 17.59 109,151 FY2014 16.33 92,817 FY2015 10.21 57,693 Total: 127.68 793,561 33 Annex 5. Beneficiary Survey Results As discussed in the main text, a survey measuring perceptions of the quality of services provided through SIIF-Nación was carried out in January 2015. The online survey, which was targeted at users registered in training activities between April and December 2014 (3,590 users), received 859 complete responses. The survey found that 92 percent of users reported that the administration of SIIF-Nación did a good or better job of meeting their expectations. 94.6 percent found the software services to be good or better; 88.8 percent that the help desk provided good or better services; 90.8 percent that training was good or better; and 92 percent that information on the web page was good or better. 34 Annex 6. Stakeholder Workshop Report and Results Not applicable 35 Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR As a part of the last Interim Financial Report (IFR), the Borrower prepared a substantial closing report presenting each component’s outputs, main achievements, lessons learned, as well as a description of the implementation process. This report served as an important input for the preparation of this ICR. A summary of the Borrower’s ratings by component is presented below. No comments on the ICR were received prior to its finalization. Component Rating Component 1 – Subcomponent 1.1 Highly Satisfactory Component 1 – Subcomponent 1.2 Highly Satisfactory Component 2 – Subcomponent 2.1 Highly Satisfactory Component 3 Satisfactory Component 4 – Subcomponent 4.1 Satisfactory Component 4 – Subcomponent 4.2 Unsatisfactory Component 4 – Subcomponent 4.3 Highly Satisfactory Component 4 – Subcomponent 4.4 Highly Satisfactory Component 4 – Subcomponent 4.5 Highly Satisfactory Component 5 – Subcomponent 5.1 Highly Satisfactory Component 5 – Subcomponent 5.2 Highly Satisfactory 36 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders Not applicable 37 Annex 9. List of Supporting Documents Aide Memoires, Consolidation of National Public Management Information Systems Project, 2010-2014. Consolidation of National Public Management Information Systems Project Implementation Status Reports, 2010-2014. Consolidation of National Public Management Information Systems Project. Loan Agreement. December 29, 2009. Loan No. 7831-CO Country Partnership Strategy for the Republic of Colombia for the Period FY2008-2011. Report No. 42847-CO. Discussed by the Board in April 2008. Country Partnership Strategy for the Republic of Colombia for the Period FY2012-2016, Report No. 60620-CO. Discussed by the Board on July 21, 2011. Country Partnership Strategy Progress Report for Colombia for the Period FY2012-2016, Report No. 83966-CO. April 16, 2014. Implementation Completion and Results report on a Loan in the Amount of US$35.47 Million to the Republic of Colombia for the Public Financial Management Project II Specific Investment Loan (SIL). June 22, 2010. Report No. ICR00001481 Project Appraisal Document on a Proposed Loan in the Amount of US$25 Million to the Republic of Colombia for a Consolidation of National Public Management Information Systems Project, November 18, 2009. Report No. 50902-CO Ministry of Finance and Public Credit, Republic of Colombia. Apectos Generales del Sistema Integrado de Información Financiera – SIIF-Nación. http://www.minhacienda.gov.co/portal/page/portal/HomeMinhacienda/siif/ObjetivoConte nido/ASPECTOS%20GENERALES%20david%20morales%20final.pdf Ministry of Finance and Public Credit, Republic of Colombia. Resultados encuesta de percepción de calidad servicios, Proceso Administración del SIIF-Nación. December 2014 Republic of Colombia. Informes Intermedios Financieros (Interim Financial Reports (IFRs)). 2010-2015 Restructuring Paper on a Proposed Project Restructuring of Consolidation of National Public Management Information Systems Project. Restructuring Approved on May 9, 2012. Report No. 67181-CO Restructuring Paper on a Proposed Project Restructuring of Consolidation of National Public Management Information Systems Project. January 30, 2013. Report No. 79436- CO Restructuring Paper on a Proposed Project Restructuring of Consolidation of National Public Management Information Systems Project. December 13, 2013. Report No. RES10165 38 Annex 10. Changes to PDO Indicators, January 2013 Restructuring PAD January 2013 Restructuring Project Description, pg 9 and Arrangements for Results Monitoring, pg 46 PDO#1. Information on budget execution—i.e., revenues and PDO#1. Continued expenditures—that is available on a daily basis to increase from 85% to 100% of the total amount of the National Budget (Presupuesto General de la Nación – PGN) (Target: 100% reconciliation of expenditures and revenues in real time with SIIF-Nación) PDO#2. Time lag to produce aggregate financial data for the central PDO#2. Continued government, including decentralized entities, has been reduced from up to 35 days to 1 day. (Target: 1 day for both centralized and decentralized entities within the central government) PDO#3. 100% of consolidated budget information (at the level of PDO#3. Continued individual entities) is published online on a monthly basis within the first week of the following month. (Target: 100% of consolidated budget information (at the level of individual entities) is published online on a monthly basis within the first week of the following month.) PDO#4. Average customs clearance time has been reduced from 48 PDO#4. Continued hours to less than 8 hours. (Target: 8 hours) PDO#5. Tax administration efficiency increases, as reflected by an PDO#5. Tax administration increase in the ratio total tax collection / DIAN’s executed budget from efficiency increases, as reflected 101.1 to101. 6 by project closing. (Target: 101.6) by an increase in the ratio total tax collection/DIAN’s executed budget from 103.2 to 103.7 PDO#6. The newly created Center for Fiscal and Financial Studies has PDO#6. Dropped. been staffed and assigned budgetary resources. (Target: The Center for New: Number of studies Fiscal and Financial Studies has been staffed and given budgetary completed to strengthen fiscal resources.) and macroeconomic planning (Target: Total of 9) PDO#7. SIIF and SUIFP information on the allocation and execution PDO#7. Continued of budget items is fully consistent. 39 Annex 11. Reallocation of Loan Funds, December 2013 Restructuring Re-allocation of Loan Funds by Expenditure Category Category Original Allocation December 2013 (U.S. $) Restructuring (U.S. $) (1) Goods, works, training, non-consultant 15,000,000 15,000,000 services and operating costs under Parts A., D.1, D.2, D.3 (c), (d) and (e), D.4, D.5 and E of the Project (2) Goods, training, non-consultant services and 2,000,000 2,000,000 operating costs under Parts B, D.3(a) and (b) and E of the Project (3) Goods, training, non-consultant services and 8,000,000 6,608,546 operating costs under Part C of the Project Partial Cancellation 1,391,454 Total 25,000,000 23,608,546 Re-allocation of Loan Funds by Component Component Original Allocation December 2013 (U.S. $) Restructuring (U.S. $) (1) Strengthening of Public Management 8,632,910 8,632,910 Information Systems within MCHP (2) Strengthening of Public Management 1,169,167 1,169,167 Information Systems within DNP (3) Expansion of DIAN’s Integrated Tax and 8,000,000 6,608,546 Customs Administration Model (MUISCA) (4) Improving the Interoperability of Other Key 5,709,555 5,709,555 IT Systems and Strategic Use of Information (5) Project Management 1,488,368 1,488,368 Partial Cancellation 1,391,454 Total 25,000,000 23,608,546 40 Annex 12. Level of Achievement – Final PDO Indicators Final PDO Indicators Level of Achievement PDO#1. Information on budget execution—i.e., revenues and Achieved. 100 percent. (In real time, expenditures—that is available on a daily basis to increase from corresponding to 193 executing 85% to 100% of the total amount of the National Budget agencies – 88 at the central level and (Presupuesto General de la Nación – PGN) (Target: 100% 105 at the subnational level). reconciliation of expenditures and revenues in real time with SIIF-Nación) PDO#2. Time lag to produce aggregate financial data for the Achieved. 1 day. (Global financial central government, including decentralized entities, has been information, corresponding to 193 reduced from up to 35 days to 1 day. (Target: 1 day for both executing agencies – 88 at the central centralized and decentralized entities within the central level and 105 at the subnational level). government) PDO#3. 100% of consolidated budget information (at the level of Achieved. 100% of budget individual entities) is published online on a monthly basis within information consolidated online the first week of the following month. (Target: 100% of (Economic Transparency Portal, consolidated budget information (at the level of individual www.pte.gov.co) on a daily basis, one entities) is published online on a monthly basis within the first day after availability. week of the following month.) PDO#4. Average customs clearance time has been reduced from Achieved, target surpassed. 6.7 hours 48 hours to less than 8 hours. (Target: 8 hours) with electronic payment. PDO#5. Tax administration efficiency increases, as reflected by Achieved, target surpassed. 106.03 an increase in the ratio total tax collection/DIAN’s executed budget from 103.2 to 103.7 PDO#6. Number of studies completed to strengthen fiscal and Achieved. 9 studies completed. macroeconomic planning (Target: Total of 9) PDO#7. SIIF and SUIFP information on the allocation and Achieved. 100 percent consistent execution of budget items is fully consistent. 41 MAP 42