Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PY DPL-DDO(P' 51007) Report Number: ICRRoo21723 i. Project Data Operation ID Operation Name P151007 PY DPL-DDO Country Practice Area(Lead) Paraguay Macroeconomics, Trade and Investment L/C/TF Number(s) Closing Date (Original) Total Financing (USD) IBRD-846io 31-Dec-2017 I00,000,000.00 Bank Approval Date Closing Date (Actual) i8-Mar-2015 31-Dec-2017 IBRD/IDA (USD) Co-financing (USD) Original Commitment 100,000,000.00 0.00 Revised Commitment oo,ooo,ooo.oo 0.00 Actual I00,000,000.00 0.00 Prepared by Reviewed by ICR Review Coordinator Group Hjalte S. A. Sederlof Robert Mark Lacey Malathi S. Jayawickrama IEGEC (Unit ) 2. Project Objectives and Policy Areas a. Objectives The Project Development Objective (PDO) was to support the government of Paraguay to: (i) strengthen sustainability, equity and transparency in fiscal management; (ii) improve targeting of social programs; and (iii) increase access to financial services. Page I of 13 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PYDPL-DDO(P'51007) b. Pillars/Policy Areas There were three pillars: Pillar 1: Public financial management, aimed at supporting the government's efforts to enhance fiscal sustainability through revenue enhancing measures, enhanced tax equity and neutrality, and better public financial management and transparency; Pillar 2: Social protection, aimed at more efficient implementation of the main social protection programs - a non-contributory pension program and a conditional cash transfer program - by improving targeting mechanisms and improving benefit delivery for both programs; Pillar 3: Financial inclusion, aimed at increasing access to financial services by lowering barriers to opening savings accounts by introducing simplified processes to open and manage basic savings accounts. c. Comments on Program Cost, Financing, and Dates A US$ oo million single-tranche stand-alone IBRD Loan was approved on March I8, 2015. It became effective on March 17, 2016, and closed on December 31, 2017. It was fully disbursed. 3. Relevance of Objectives & Design a. Relevance of Objectives The DPL objectives are relevant to the country situation, government policy and World Bank strategy. While Paraguay has been experiencing strong economic growth and a steady decline in poverty, it is an open economy that is vulnerable to exogenous (economic and climatic) shocks. The overarching challenge at the time of program development was, and still is, how to maintain and build on past achievements, especially when faced with deteriorating fiscal balances. To begin to address this, the government in 2014 drew up a long-term National Development Program (NDP, 2014-2030), including public sector reforms, to raise growth and ensure continued poverty reduction. The Bank's Country Partnership Strategy (CPS) for FYI 5- FY 8, still current, is aligned with the NDP and contributes to its overall poverty and equity targets. The CPS has been built around three results areas - resilience to risk; boosting pro-poor delivery of public services; and integrative markets. The DPO contributes to the two former ones through its focus on public financial management and financial inclusion, and on pro-poor delivery of goods and services. The PDO in turn is aligned with both the NDP and the CPS through its efforts to strengthen sustainability, equity and transparency in fiscal management, improve targeting of social programs and increase access to financial services. Page 2 of 13 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PY DPL-DDO(P1 51007) Rating High b. Relevance of Design The results chain was mostly straightforward: better fiscal management included the adoption of a comprehensive set of measures to increase tax revenues and stabilize fiscal balances, and secure resources for the country's two social assistance programs, which would be better targeted. Extending formal banking services, especially electronic banking, to the poor, aimed at facilitating a more efficient and transparent transfer of social assistance funds to beneficiaries, as well as increasing their inclusion in the financial system. Better fiscal balance was to be achieved by the application of a set of tax policy reforms introduced as prior actions. They included the adoption of higher tax rates and a wider tax base in applying the Tax on Value Added (VAT), improved tax compliance through risk-based processing of VAT claims, improved tax equity by extending the corporate tax to agriculture, better cash management with the introduction of a Single Treasury Account, improved fiscal transparency through public availability of information on budget execution; and improved financial performance of state-owned enterprises (SOE) through stronger audit and financial management arrangements. Safety nets were expected to be strengthened through improved beneficiary targeting of retiree social assistance. In addition, safety net strengthening included measures to ensure timely and cost-effective receipt of cash transfers by families participating in the country's cash transfer program. The former was to be addressed by placing greater emphasis on the poverty line as an indicator; while the latter would be achieved by means of simplified processes to open and manage basic savings accounts. Financial inclusion of poor families receiving cash transfers was to be achieved by the channeling of benefits through the new basic savings accounts. The legal framework for mobile money was also to be strengthened, with a view to extending electronic payment services to currently under-served segments of the population. There were six prior actions related to the first pillar, most of them strong, precise, and relevant. The second and third pillars had two prior actions each, and these were less precise, but underpinned activities that, if completed, would have achieved the respective objectives. A DPO was chosen to help address volatile growth and reduce the uncertainties volatility brought to fiscal balances. A deferred draw-down option (DDO) was included as a further precautionary risk management measure to protect essential public spending against exogenous shocks. (The loan was subsequently drawn down in three tranches - two in 20 16 and Page 3 of 13 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PYDPL-DDO(P1 51007) one in 2017.) Institutional capacity constraints were taken into account by addressing a limited number of interlinked policy issues, and relying on ongoing capacity enhancement, including through Bank-financed technical assistance. The macroeconomic framework was considered adequate for purposes of the operation. Growth was expected to remain moderate, and the fiscal deficit, while increasing, was expected to remain contained. The IMF has not had a program with Paraguay since 2008, but carries out annual Article IV consultations. A Debt Sustainability Analysis (DSA) prepared by the IMF in 20 13 for the annual Article IV indicated that the trajectories of public debt were sustainable. Rating Modest 4. Achievement of Objectives (Efficacy) Objective i Objective To strengthen sustainability, equity and transparency in fiscal management Rationale Rationale. The objective was to be achieved by (i) introducing revenue enhancing measures, including raising the value added tax and expanding the tax base, and increasing the efficiency of tax collection; (ii) improving tax equity and neutrality by introducing a new tax on agricultural income and eliminating previous exemptions; (iii) establishing a Treasury Single Account (TSA) strengthening cash management and controls on budget execution; (iv) enhancing fiscal transparency by publishing information on budget execution, procurement, and salaries; (v) improving SOE's commercial performance and financial viability. Increasing taxation. The VAT rate relative to gross value added was increased from 5 percent to i0 percent for financial sector transactions; in addition, a VAT rate of 5 percent relative to gross value added was established for agricultural and livestock products (both were prior actions). As a result, VAT revenue from financial sector transactions rose from a baseline value of 3-9 percent of transactions to 9.6 percent (end-20 17), compared to a target of 6.3 percent. VAT revenue from agriculture, livestock and forestry rose from a baseline of zero to 1.3 percent (end-2017), compared to a target of 1.4 percent. To improve compliance behavior (reduce fraud) and reduce VAT refunds, risk-based processing of refunds was introduced (prior action). As a result, the number of taxpayers processed reached 389 (large) firms against a target of 3 1, a baseline of zero, Page 4 of 13 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PYDPL-DDO(P' 51007) and a total of 75 1 firms. This seems to miss the purpose of risk-based processing of refunds, which would be expected to focus on the largest and most challenging cases, not on as many firms as possible. Moreover, while this exceeded the number of firms processed, there is no indication of any reduction in revenue losses or in the practice of fraudulent invoicing. Improving tax equity. To increase tax equity and neutrality (the former by increasing the emphasis on direct taxation, and the latter by making taxation more uniform among sectors by bringing in agriculture), collection of agricultural income taxes was revised. As a result, 330 billion Guaranies (Gs) in taxes on agricultural income were collected (end-2017) compared to a target of Gs 5 18 billion and a (pre-revision) baseline of Gs 72 billion. There is no assessment on the shortfall, i.e. why it did not achieve its equity objective. Financial management. To ensure effective control over cash balances and improve efficiency in managing cash and borrowing, the government established a TSA (prior action). The introduction of a TSA was to enable effective liquidity management and strengthen controls on budget execution. As a result, interest payments on central government intra-year borrowing operations fell by 95 percent between 20 14 and end-201 7, from Gs 4 1 billion to Gs 2.25 billion, against a target of Gs 21 billion. Fiscal transparency. To improve transparency and accountability in its public financial management, the government introduced the following measures as prior actions: made accessible monthly budget execution information through the internet; enabled public access to the public procurement process; and made mandatory the publication on the internet on a monthly basis of salaries, subsidies and other allowances paid to public officials. (prior action). SOEs financial performance. As part of a longer-term strategic plan to enhance the operational, commercial and financial performance of the country's nine SOEs, the government introduced measures to eliminate the consolidated arrears between SOEs and government to induce a downward debt trajectory. To initiate this, the government introduced a package of measures as prior actions: a five-year payment plan; liberty to interrupt services to government for non-payment; and as preventive measures mandatory external audits of SOEs; and regulations that prevented the generation of debts for unpaid services between SOEs. By end-20 i7, arrears of public institutions to SOEs for basic services decreased by 5o percent, falling from a baseline of US$124 billion to US$53 billion by end-20 1 7, against a target of US$47 billion. In most instances, targets were achieved. The one exception related to the effects of the introduction of risk-basing in processing VAT and SCT credit requests Rating Substantial Page 5 of 13 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PYDPL-DDO(P'51007) Objective 2 Objective To improve targeting of social programs Rationale The objective was to be achieved by improving the targeting mechanisms of the country's non-contributory pension plan, and its cash transfer program. Targeting for the retiree assistance program was meant to be strengthened through the introduction of more accurate selection criteria, i.e. by giving greater weight to the national poverty line in identifying beneficiaries. The indicator used is the accumulated savings generated by the new targeting mechanism between 2013 and 2017, that is, a measure of the reduction in errors of inclusion. Savings totaled Gs 5 million by end-20 17, against a target of Gs 7 million. This indicator is not altogether adequate, since it focuses on monetary savings rather than reduction in targeting errors due to the new mechanism. Improvement in the delivery of benefits was to be achieved through the introduction of basic bank accounts (Objective 3) that would reduce transactions costs and increase transparency for both government and beneficiaries. Success of this intervention was measured by the share of transfers returned from the account holder to the social protection secretariat. Transfers returned measured 3.4 percent of transfers in 2017, against a target of 2 percent and a baseline of 2.5 percent. The indicator is less than satisfactory, since a number of factors could have influenced the outcome (over-payment by the agency, transactions costs for the beneficiary, an expanded client base). Moreover, as shown below, the goals under Objective 3 were not met, so it is difficult to attribute the result to the prior actions taken. Rating Modest Objective 3 Objective To increase access to financial services Rationale Page 6 of 13 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PYDPL-DDO(P' 51007) Rationale. The objective was to be achieved by reducing barriers to opening savings accounts; and introducing reinforced regulations governing electronic payments. To facilitate access to and use of financial services for lower income groups, the government established simplified procedures for opening and managing basic low-cost savings accounts - the accounts do not have minimum opening and balance requirements, nor do they charge monthly maintenance fees. (prior action). While the number of people without access to financial services was expected to decline as a result, it rose to 48 percent at end-2017 compared to a baseline of 42 percent and a target of 30 percent. The ICR suggests that this may reflect a growth in the adult population over the 2013-2017 period that outpaced the growth in new accounts. This does not seem plausible unless there were significant shifts in adult population flows into the country, or high fertility rates a generation ago. It seems more likely that the numbers reflect the use of different methods in collecting data over the period. To address the strong growth of mobile money in the country, the government strengthened regulations on electronic payment companies and electronic payments to ensure a rules-based market (prior action). It had expected that the share of adults using financial electronic services would reach 40 percent from a base of 28 percent; instead, it reached only 31 percent over the 2013-2017 period. Summary. This objective was not achieved - access to financial services declined; and demand for electronic services turned out to be lower than expected. No assessment appears to have been made on the reasons underlying these results. Rating Negligible 5. Outcome Relevance of objectives is rated high, as the operation aligned with the government's development program and World Bank strategy. Relevance of design is rated modest reflecting weaknesses in the prior actions related to the second and third objectives. Efficacy for the first objective is rated substantial, as objectives were largely achieved. The second objective is rated modest as achievement of objectives was only partial. The third objective is rated negligible - neither target was met, while the first, which directly corresponded to the objective as written, went in the wrong direction. These amount to significant shortcomings, and outcome is rated moderately unsatisfactory. Page 7Of 13 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PYDPL-DDO(P1 51007) a. Outcome Rating Moderately Unsatisfactory 6. Rationale for Risk to Development Outcome Rating The DPL directly supported priority elements of the government's NDP, notably growth and poverty reduction, and fiscal stability. These are reflected in the CPS. A new DPL in the pipeline would continue and expand on themes raised in this operation, continuing to help further develop accountable institutions and fiscal inclusiveness. Elements of risk are (i) a deteriorating fiscal stance, albeit counteracted by donor support and strong government commitment to the NDP objectives; and (ii) limited administrative capacity to successfully implement the reforms (see also Section 7a - Quality at Entry). The risk for the development outcome is rated modest. a. Risk to Development Outcome Rating Modest 7. Assessment of Bank Performance a. Quality-at-Entry The project was strategically relevant, basing its design on priorities spelled out in the government's NDP and, in particular, efforts to improve fiscal balance as a means of achieving longer-term goals of poverty reduction. This gave it internal coherence (Section 3b). It also drew on Bank experience from earlier operations in Paraguay, and on up-front analytical work and technical assistance; and it benefited from close collaboration with other development partners, notably the IMF and the Inter-American Development Bank. Risk was rated moderate, as the operation was designed to take into account capacity constraints in the administration (limiting participation to a few capable local agencies), as well as the availability of technical assistance. The results framework was focused on the main immediate concern of better fiscal management and related improvements in the safety net. In the case of the first objective, links between activities (prior actions) and outcomes were straightforward, although in a few instances, more rigorous measurement would have made assessment of the development impact easier (for instance the reduction in VAT losses). For the second and third objectives, prior actions were imprecisely formulated, and some indicators were inadequate to measure attainment of objectives. Moreover, the second objective was focused on targeting and did not explicitly take into account increased efficiency in benefit delivery, which was included as an additional means to more effectively implement the main social protection programs. Page 8 of 13 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PYDPL-DDO(P' 51007) Quality-at-Entry Rating Moderately Satisfactory b. Quality of supervision The ICR indicates that supervision was carried out in collaboration with the government team and other external partners. Supervision was accompanied by technical assistance that helped implement the various prior actions and identify further reforms that now are being introduced into a follow-up DPL. According to the ICR, collaboration with the Bank was much appreciated by the government. Somewhat more rigor could have been exercised in completing some activities, such as the strong response to risk-based processing of VAT/SCT credits should have been explored, as well as assessing the responses to changes in financial services. Quality of Supervision Rating Moderately Satisfactory Overall Bank Performance Rating Moderately Satisfactory 8. Assessment of Borrower Performance a. Government Performance Government support for the operation appears to have been strong throughout preparation and implementation, reflecting its commitment to addressing immediate issues and implementing the longer-term NDP. This was already apparent during the preparation of the NDP, which the government developed in a consultative process with major stakeholders. Collaboration with implementing agencies was good - it took the form of a consistent process of information collection and monitoring by the responsible agencies, with reporting channeled through the Ministry of Finance. Government Performance Rating Satisfactory b. Implementing Agency Performance Not rated Page 9of 13 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PYDPL-DDO(P' 51007) Implementing Agency Performance Rating Not Rated Overall Borrower Performance Rating Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design All indicators were relevant with regard to determining achievement of objectives. However, in some instances, measurement was challenging, notably in measuring success in poverty targeting: savings from introducing a new system was the measure of success, when a more straightforward measure such as errors of inclusion (which the measure presumably estimates) might have been more straightforward. Likewise, the outcome indicator for reduced debt to SOEs focused on a narrow, immediate, impact of the prior action, but could also have included preventive measures with a longer term impact. The Ministry of Finance through the Directorate of Debt Policy was responsible for monitoring and coordinating with other Government ministries and agencies involved in its implementation. b. M&E Implementation All indicators included in the policy matrix and results framework were monitored, and the quality of the information was generally satisfactory; although it could have been more precise in some instances: notably, better targeting was determined on the basis of savings, rather than reduction in errors. The former option is understandable as the focus was on fiscal probity, albeit the latter is a more conventional measure of success. The ICR points to the data requirements underlying the determination of savings achieved through better targeting, notably its basis in survey techniques and related processes that were disrupted by the rotation of government officials. c. M&E Utilization The results matrix was used to monitor implementation. It also served as basis for discussing progress, as well as relevant policy issues with the government Page io of i3 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PYDPL-DDO(P'51007) M&E Quality Rating Substantial i o. Other Issues a. Environmental and Social Effects No safeguard policies were discussed in the ICR b. Fiduciary Compliance No fiduciary issues were discussed in the ICR. c. Unintended impacts (Positive or Negative) None noted d. Other None noted i i. Ratings Ratings ICR IEG Reason for Disagreements/Comment OutcomeModerately Outcome Usat y Moderately Unsatisfactory --- Unsatisfactory Risk to Development Outcome Modest Modest --- Bank Performance Moderately Satisfactory Moderately Satisfactory --- Borrower Performance Satisfactory Satisfactory --- Quality of ICR Substantial --- Page I I of 13 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PY DPL-DDO(P' 51007) Note When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July r, 2006. The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 12. Lessons The following lessons are drawn from the ICR: Strong government commitment increases opportunity for reform. Strong Government commitment, for instance as reflected in broad stakeholder commitment to a reform agenda is likely to lead to successful outcomes. In Paraguay, the country's long-term reform program had been developed with the participation of a large number of stakeholders, facilitating implementation. Strong analytical work underlies successful substantive reform. Program design should be underpinned by strong analytical work and technical assistance. The DPL was based on prior strong research, which, in turn, provided the necessary rationale for selection of policy and program reform areas. IEG adds the following lesson: Clearly formulated prior actions, with a robust causal chain to the intended results, are important for the success of development policy operations. In this case, the prior actions related to the substantially achieved first objective were mostly precise, relevant and measurable. Those for the second and third objectives, where outcome was much weaker, were few in number and were not formulated with the appropriate degree of precision and detail. 13. Assessment Recommended? No 14. Comments on Quality of ICR The ICR provides an adequate discussion of the background and rationale of the operation to allow for an assessment of the results and ratings of the operation. In some instances, the ICR could have been more moderate in its assessment - for instance in assessing the balance of prior actions in each pillar; and in pointing to the weak institutional capacity on implementation, of which there really is no discussion. Its discussion of equity, on the other hand, is quite thorough. Page 12 of 13 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PYDPL-DDO(P5 51007) a. Quality of ICR Rating Substantial Page 13 Of 13