Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AR La Rioja Public Sect Streng Prog-APL1 (P121836) Report Number : ICRR0021275 1. Project Data Project ID Project Name P121836 AR La Rioja Public Sect Streng Prog-APL1 Country Practice Area(Lead) Argentina Governance L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IBRD-80080 28-Feb-2016 27,276,619.52 Bank Approval Date Closing Date (Actual) 22-Feb-2011 31-Dec-2017 IBRD/IDA (USD) Grants (USD) Original Commitment 30,000,000.00 0.00 Revised Commitment 30,000,000.00 0.00 Actual 27,277,578.50 0.00 Prepared by Reviewed by ICR Review Coordinator Group Judyth L. Twigg Robert Mark Lacey Malathi S. Jayawickrama IEGEC (Unit 1) 2. Project Objectives and Components a. Objectives According to the Loan Agreement (p. 5), the objectives were "to increase tax collection and improve expenditure quality, and to pilot improvements in public service delivery in the area of rural water management." At an August 2015 restructuring, outcome targets were revised (including targets related to tax collection that were revised upward due to inflation), and therefore a split rating is performed in this assessment. At the time of this restructuring, US$ 21.75 million, or 79.7% of total Bank financing, had been disbursed. Page 1 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AR La Rioja Public Sect Streng Prog-APL1 (P121836) The project was the planned first phase of a two-stage, US$ 80 million Adaptable Program Loan (APL) whose objective was "to strengthen the institutions of the public administration of the Province of La Rioja to promote effective and sustainable delivery of essential public services for human and productive development" (Project Appraisal Document, PAD, p. 3). The second operation was to deepen the public sector reform agenda and extend it within agriculture and to other sectors related to human and productive development. b. Were the project objectives/key associated outcome targets revised during implementation? Yes Did the Board approve the revised objectives/key associated outcome targets? Yes Date of Board Approval 07-Aug-2015 PHEVALUNDERTAKENLBL c. Will a split evaluation be undertaken? Yes d. Components The project contained two components: 1. Government Public Sector Reform Program (Eligible Expenditure Programs, EEPs) (appraisal: US$ 24 million; actual, US$ 23.86 million). This component was to follow a Sector-Wide Approach (SWAp) modality, disbursing against EEPs in three areas: tax administration improvement (Ministry of Finance, covering modernization of the provincial tax collection system, and updating of the provincial cadaster); quality of expenditure (Ministry of Finance and Financial Directorates of ministries, covering the introduction of fundamentals for performance-oriented budgeting, and public procurement reform); and service delivery in rural water management (Ministry of Infrastructure/Secretariat of Water, covering strengthening of the province's 45 Water User Associations (WUAs) through a process of consolidation and training, expansion of water supply systems and distribution networks, and conducting of a Sector Environmental Assessment (SEA)). Disbursements were to be linked to achievement of agreed disbursement-linked indicators (DLIs) and the execution of at least 70% of the budgeted EEP expenditures. 2. Technical Assistance (TA) (appraisal, US$ 5.25 million; actual, US$ 2.66 million). This component was to support the preparation and implementation of the government expenditure programs described under the first component, through support for modernization of the tax administration function and related institutions (supporting improvements in the quality of the taxpayer registry, organizational changes to reduce tax arrears and improve tax collection, tax audits and enforcement, and capacity to update the urban cadaster); improvement in the quality and efficiency of expenditure programs (through the introduction of performance-informed budgeting and reform of the public procurement function); and development of a strategy to improve management of and access to water resources by a dispersed rural Page 2 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AR La Rioja Public Sect Streng Prog-APL1 (P121836) population (through the development of the SEA, institutional strengthening of WUAs, improvement of integrated rural water management, and preparation of strategic plans for water resource management and strengthening of extension and TA services). The component was also to support overall project coordination and supervision, including capacity-building for the Project Coordination Unit, monitoring and evaluation (M&E), and audits. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project Cost: Planned project costs were US$ 30 million, with a US$ 0.75 million front-end fee. Actual costs were US$ 27.3 million. Financing: The project was to be financed by a US$ 30 million IBRD Loan, the first of a two-phase APL totaling US$ 80 million. It combined financing of EEPs against which loan proceeds were to flow to the Provincial Treasury; a set of disbursement rules (DLIs) providing a performance framework; and a technical assistance component. US$ 27.3 million was disbursed, and the remaining US$ 2.7 million was cancelled. The second phase of the APL was not pursued due to changes in priorities resulting from a new provincial administration and new World Bank Country Director. Borrower Contribution: No Borrower contribution was planned or made. Dates: The project was approved on February 22, 2011 and became effective on August 1, 2011. A June 20, 2012 restructuring revised some intermediate outcome targets and institutional arrangements. The mid-term review (MTR) was held in May 2014. A December 2014 restructuring that would have revised outcome targets and some institutional arrangements was approved by the Board, but this restructuring was not fully processed due to additional restructuring needs that subsequently emerged and the desire of the Country Director to establish a single reference point for all revisions made after the MTR. A final restructuring on August 7, 2015 contained the changes that had been approved in December 2014 (revision of outcome targets to reflect delays in hiring a technical assistance firm and in enacting a procurement law), as well as changes to the implementation schedule and the project closing date (extended from February 28, 2016 to December 31, 2017). The project closed on December 31, 2017. 3. Relevance of Objectives Rationale At appraisal, rural poverty was relatively high in La Rioja, and the public sector and agriculture were economically important. Limited local tax revenue and low public sector efficiency were key constraints on social and economic development. Tax administration reform was seen as key for mobilization of additional revenue for an inclusive development agenda. Effective expenditure management was limited by shortcomings in budgeting and public procurement. Public water resource management was critical for this arid region, characterized by severe desertification. The project’s objectives were highly relevant to these local conditions. Page 3 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AR La Rioja Public Sect Streng Prog-APL1 (P121836) The national government had prioritized support for provinces with relatively high needs, especially the Norte Grande region. San Juan and La Rioja were selected as initial areas for Bank operations under this strategy. The Provincial Government of La Roija had committed to a multi-annual program of public sector reform (the Integral Provincial Development Plan) intended to improve public service delivery. The program linked horizontal public sector reform with visible improvements in rural water management, the latter intended as a pilot prior to extending reforms to other public services. More recently (November 2017), the national government signed a Fiscal Pact with provincial governments to contain public spending and increase local revenue collection. The objectives were highly relevant to this government strategy, and also to the Country Partnership Strategy at appraisal (CPS, 2010-2012), with outcomes to strengthen the effectiveness, efficiency, transparency, and accountability of public sector management, and to expand performance management, improve the quality of public expenditure, enhance service delivery, and engender trust in public institutions. The project remains highly relevant to the World Bank Group's 2015-2018 CPS. This includes governance as a cross-cutting theme, highlighting the needs for a comprehensive development approach to public sector management and for increased efficiency and transparency in public spending. It prioritized strengthening of public service delivery at the provincial level, noting the responsibility of subnational government for the delivery of basic services, including water. It also supported the project's results-based financing approach, noting that mechanism's contribution toward strengthened governance and accountability. Rating High 4. Achievement of Objectives (Efficacy) PHEFFICACYTBL Objective 1 Objective Increase tax collection Rationale Theory of change: Project design combined cross-cutting horizontal interventions in public administration with a focus on a specific sector (rural water resource management) that would generate public support and political capital to ensure sustainability of the reform. Under the tax collection objective, improvements in human resources, organizational reform, updating of the property registry database, and conducting an urban cadastral census and tax moratorium campaign were to produce an updated urban cadaster and taxpayer registry, as well as an increase in the tax arrears processed for collection or sent to court. These measures were to lead ultimately to improved tax enforcement and investigation capacities and better overall provision of services to taxpayers, and ultimately to increased local tax revenue. Page 4 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AR La Rioja Public Sect Streng Prog-APL1 (P121836) Tax enforcement powers were delegated to the General Office of Provincial Taxes, whose strategic planning functions were strengthened. Tax records were updated, payment points were expanded, an electronic payment system was introduced to facilitate tax compliance, specialized auditors were trained and deployed, and a web-based complaints channel was introduced. 1,359 integral audits for Gross Income Tax were conducted, exceeding the original target of 850 and the revised target of 1,300. 53,105 tax arrears accounts in excess of AR$ 5,000 were processed for tax collection, exceeding the original target of 2,500 and the revised target of 10,000. 1,220 pending tax arrears cases were sent to court, exceeding the original target of 250 and the revised target of 780. Regarding property tax administration, systems analysts and surveyors were hired and trained, allowing for the digitization of plans and parcel files, as well as creation of a cadaster website. 101,807 land parcels were updated and integrated in the urban cadaster, including changes in ownership, updates to the physical parameters of the parcels, and improvements in the General Office of Provincial Taxes, cadaster, and property registry databases. This exceeded the original target of 11,000 and the revised target of 21,000. This almost five-fold exceeding of the revised target may be attributable to extra capacity realized when the General Directorate of Cadaster hired additional information technology specialists, though it may also be the case that the target was not sufficiently ambitious. An updated Cadaster Law was passed in 2017 after elections brought in a new governor and administration, allowing for annual updating of real estate property valuations to incrementally close the gap with market prices. Following these interventions, revenue generated locally by the provincial government, expressed in nominal terms, increased from AR$ 159 million in 2011 to AR$ 1645 million in 2017, exceeding the original target of AR$ 189 million and the revised target of AR$ 782 million (AR$ 993 million when adjusted for inflation). Although direct attribution of increased tax revenues is challenging, the timing of the increased revenues -- from 2014 onward, corresponding to the year the province began to implement improvements in tax collection, enforcement, and compliance -- supports the argument that the project was plausibly responsible for at least some of the observed outcomes As both the original and revised outcome targets were achieved, a split rating is not performed for this objective. Rating Substantial PHREVDELTBL PHEFFICACYTBL Objective 2 Objective Improve expenditure quality (original outcome targets) Rationale Theory of change: A definition of budgeting programs for the provincial administration, introduction of performance indicators as well as an M&E unit, capacity development in budgeting, planning, and procurement, and updating of procurement laws and institutions were to lead to a new programmatic budget Page 5 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AR La Rioja Public Sect Streng Prog-APL1 (P121836) structure, improved strategic and operative planning, strengthening of procurement processes and capacities, and enhanced M&E of the use of public resources. In turn, these outputs were to result in improved expenditure quality: an improved strategic allocation process for public resources, better budget predictability, enhanced performance management, and increased efficiency and transparency in public procurement. Modest progress was made toward improving budget predictability. The Public Sector Reform Program was budgeted. Programmatic structure and physical and financial indicators for the Program were introduced into the budget process and updated as needed. 166 staff were trained in strategic planning and program budgeting, exceeding the original target of 40 and the revised target of 80. The annual number of budget amendments affecting appropriations between budget programs or expenditure categories decreased from 380 in 2011 to 114 in 2017, exceeding the original target of 250 and the revised target of 199. However, since the introduction of fiscal tightening measures in 2016, restrictions have been set on the execution of primary expenditures. As a result, the share of budgeted primary expenditures executed increased from 79.1% in 2011 to 79.32% in 2017, not meeting the original target of 90% or the revised target of 93%. Progress was also made toward improving processes for the strategic allocation of public resources. Ten provincial Secretariats updated their strategic and operational plans, exceeding the original target of eight but not meeting the revised target of twelve. 25 Directorate of Financial Administration units prepared reports on program budgeting and monitoring, exceeding the original target of 12 and the revised target of 21. While the milestone of having a Provincial Integrated Development Plan that fed into the budget formulation process was not achieved, the government was finalizing such a plan at closure, and it was expected that the document would provide inputs for the coming fiscal year's budget exercises. In the project’s final year, 58% of programs in the annual budget were present in the provincial development plan, exceeding the original target of 50% but not meeting the revised target of 70%. Regarding public procurement, an updated Legal Public Procurement Framework was adopted, along with secondary legislation, an operational manual, and standard bidding documents. A Procurement Policy Unit was established within the Ministry of Finance, and an accreditation system was established for public procurement. 80 staff were trained and accredited in this system, exceeding the original target of 16 and the revised target of 46. A Procurement Information System was established, and statistical data on procurement outcomes and methodologies was used by relevant institutions. Terms of reference were elaborated for a Quick Gains study to improve procurement efficiency, and the study was commissioned and produced, identifying a strategy for the rationalization of procurement, and concrete measures to achieve savings in government purchasing. Ten provincial Secretariats presented and monitored annual procurement plans, not meeting the original target of 12 or the revised target of 17. 75% of provincial spending was covered by a procurement audit that verified compliance with the new procurement law and its regulation, meeting the original/revised target. As a result of these interventions, the percentage of procurement carried out through competitive processes in at least two secretariats increased from zero in 2011 to 44.5% in 2017, exceeding the original/revised target of 40%. In 2017, the two agencies that used competitive bidding were the health and road ministries. Page 6 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AR La Rioja Public Sect Streng Prog-APL1 (P121836) Achievement is rated Substantial under the original targets, as the original outcome targets related to strategic allocation of resources and procurement were met. Rating Substantial PHREVDELTBL PHINNERREVISEDTBL Objective 2 Revision 1 Revised Objective Improve expenditure quality (revised outcome targets) Revised Rationale The revised outcome target was not achieved on the main indicator related to strategic allocation of public resources (percentage of programs in the annual budget present in the provincial development plan). As the target for improved budget predictability was also not met, achievement of this objective under the revised outcome targets is rated Modest. Revised Rating Modest PHEFFICACYTBL Objective 3 Objective Pilot improvements in public service delivery in the area of rural water management Rationale Theory of change: Consolidation processes for WUAs, including regulatory status reviews, updating of user registries, water use efficiency assessments, rearrangement of water delivery schedules, and recalculation of water use fees, were to improve quality and sustainability of irrigation and drainage services, improve WUAs' regulatory status and compliance, improve monitoring of usage and usage fee determination/collection, and increase supervision of water distribution processes. Other activities -- sensitization of users on water resource management issues, training of farmers on water management, improvements in the small-scale supply network, and creation of a Provincial Water Plan -- were to further contribute to improved rural water service delivery and management: increased efficiency of collectively managed irrigation schemes, better cost recovery, and a higher number of households in remote areas with access to a drinking water system. A Sector Environment Assessment was conducted, as planned. An Environmental and Social Management Unit was set up in the Provincial Institute of Water, and a legal and normative framework and Water Code were elaborated. A Provincial Water Plan was created, including a diagnostic of the sector in the province that covered the main short- and long-term actions needed to improve management of water. User registries for irrigation services were updated, water delivery schedules were improved, and user fees were Page 7 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AR La Rioja Public Sect Streng Prog-APL1 (P121836) recalculated. 4,145 farmers were trained on management techniques related to access to water, exceeding the original target of 4,000 but not meeting the revised target of 4,800. Of the 28 WUAs covered by the project, 24 became compliant with current irrigation regulations, exceeding the original target of 15 and almost meeting the revised target of 25. All 28 WUAs increased their number of fee-paying users by at least 20%, exceeding the original target of 18 and meeting the revised target of 28. 24 WUAs reduced their time elapsed between irrigation turns by 50% (from an average of 30 days to 15 days), exceeding the original target of 12 WUAs and meeting the revised target of 24 WUAs. 1,182 households in remote areas gained access to a drinking water system, exceeding the original target of 750 and the revised target of 1050. These interventions are now part of the Provincial Institute of Water's strategic plan and have specific financial resources from that organization's budget, complementing revenues generated by user fees. Overall, the number of WUAs that have completed the consolidation process (including regulatory status reviews, water use efficiency assessments, water delivery schedule rearrangements, and recalculation of water use fees) increased from 5 in 2011 to 28 in 2017, exceeding the original target of 18 and meeting the revised target of 28. As both the original and revised outcome targets were achieved, a split rating is not performed for this objective. Rating Substantial PHREVDELTBL PHOVRLEFFRATTBL Rationale With Substantial achievement of all three objectives, overall Efficacy under the original outcome targets is rated Substantial. With Substantial achievement of two objectives and Modest achievement of one objective under the revised outcome targets, overall Efficacy is also rated Substantial under the revised targets. Overall Efficacy Rating Substantial 5. Efficiency The PAD (pp. 12-14, 45-48) stated that the expected economic impact of the project was positive, though only the impact of the tax administration reform was quantifiable, as "improvements in institutional performance do not easily translate into monetary terms." Qualitatively, the project’s benefits were anticipated to accrue from greater fiscal space, adoption of results-based management tools, more efficient information systems, a more qualified workforce, greater internal capacity, and potential savings in procurement. Page 8 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AR La Rioja Public Sect Streng Prog-APL1 (P121836) Quantitatively, the proposed improvements in tax administration only (covering updating of 11,000 urban land parcels, matching and registering 25,000 urban properties with their correct taxpayer identification, recovering AR$ 12 million of tax arrears, and conducting 850 integral tax audits) were expected to raise local tax revenue by AR$ 43.8 million over the 2011-2015 project period, and AR$ 62.1 million (US$ 15.7 million) over the 25- year period of loan repayment, equivalent to about 52.8% of the US$ 30 million loan amount. These calculations assumed a 5% discount rate and an exchange rate of AR$ 3.95 per US dollar, current at the time of the PAD. The ICR (Annex 4, pp. 52-56) carried out an ex post financial analysis of the project's tax administration activities. The actual tax collected from updating of parcels, processing of tax arrears, and conducting of audits was reported and adjusted for inflation using the consumer price index for the San Luis province. Two methods were used, both finding the project financially sound across a range of discount rates. The first method calculated net present value (NPV) gains for actual costs and actual tax collected adjusted for inflation, using the same 5% discount rate used at appraisal. This method found an ex post NPV gain of AR$ 339.3 million. The second method calculated for actual costs and benefits, deflating the amount of tax collection (in 2011 AR$) and using a real interest rate for the discount rate. This method found an ex post NPV gain of AR$ 539.6 million for 2011-2017. Both compare favorably to the AR$ 43.8 million estimated at appraisal. The ICR (pp. 20-21) also reported implementation efficiencies. Project design kept TA activities focused on the government's EEP and DLIs, ensuring that the project's various parts were moving in tandem. Hiring of the technical assistance (TA) consulting firm was consolidated into a single contract, with a view to attracting a larger firm with greater capacity to deliver results; however, the drafting of the terms of reference took longer than expected, resulting in a 2.5-year delay (during which the Bank team compensated for the lack of a TA firm). In addition, due to the small market for such a broad array of services (several different public sector management areas plus water resource management), only one proposal was received. According to the ICR (pp. 20-21), "the limited competition probably resulted in a higher price of the contract." As the ICR argued, though, it is likely that this higher price was compensated by the advantage of having all activities under the same umbrella, reducing transaction costs, and facilitating cross-collaboration between areas, alignment, and on-time delivery. There were also delays in deciding on and implementing an information and communication technology strategy for the project, and in implementing the umbrella financial management information system that the federal government provided to the provincial governments. According to the ICR (p. 24), these delays did not compromise timely achievement of outcomes. Activities were completed on time with no cost overruns. Efficiency Rating Substantial a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Page 9 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AR La Rioja Public Sect Streng Prog-APL1 (P121836) Rate Point value (%) *Coverage/Scope (%) Available? 0 Appraisal 0 Not Applicable 0 ICR Estimate 0 Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome The project's objectives were Highly relevant to local conditions at appraisal, and to government and Bank strategy at appraisal and closing. Under both the original and revised outcome targets, efficacy was Substantial, with key outcome targets met for indicators that were appropriate measures of achievement of each of the three objectives. Efficiency was Substantial, supported by a financial analysis of the tax administration subcomponent and operational efficiencies likely achieved through consolidation of TA activities into a single large contract. These ratings indicate only minor shortcomings in the project's preparation and implementation, leading to an overall Outcome rating of Satisfactory. a. Outcome Rating Satisfactory 7. Risk to Development Outcome New institutional frameworks and processes throughout the entire provincial administration, covering budget, legal, procurement, information technology, and strategic planning reforms, are likely to be sustained. In particular, integration between the cadaster (which now has 100% digital coverage in the urban areas of the province) and the tax registry system is "very unlikely to be reversed" (ICR, p. 27). Sustainability for budget and procurement systems is also very strong. However, political and macroeconomic factors may threaten some achieved outcomes (ICR, p. 28). Argentina has recently committed to continuing and quickening fiscal tightening, reducing the share of budgeted primary expenditures executed due to cuts in public spending, and affecting budget predictability particularly for the delivery of critical public services. With new processes and tools put in place by the project, and with the national government pushing for better financial planning, it is expected that budget predictability will improve, but the current tight fiscal constraints will not allow primary expenditures to increase. In addition, the articulation between planning and budgeting continues to be slow, and although many financial units have produced strategic and operative plans, provincial-level strategic planning still presents challenges. The ICR (p. 28) stated that the Planning Directorate still lacks "the institutional strength and political power to incorporate strategic planning as a regular exercise in the prioritization of resources." Page 10 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AR La Rioja Public Sect Streng Prog-APL1 (P121836) 8. Assessment of Bank Performance a. Quality-at-Entry The project was prepared in close collaboration with provincial authorities, and the financing instrument provided appropriate incentives and a road map for advancing government reform. Project design drew on experiences of previous successful SWAPs in Brazil and the Argentine province of San Juan, including the need for strong sector diagnostics and close client collaboration; the appropriateness of disbursing against EEP expenditures linked to DLIs; the extent to which early visible results are key to sustainable reform (leading to the selection of water management as a pilot); and the risks of overambition of technical assistance components (a risk mitigated by keeping the second component small, with only a few contracts, and limiting activities to those with critical relevance and high client ownership). M&E arrangements and performance indicators were well defined. Overall project risk was assessed as medium, largely due to limited implementation capacity, and adequate mitigation measures were proposed (PAD, pp. 39-40). Quality-at-Entry Rating Satisfactory b. Quality of supervision The Bank conducted thirteen supervision missions as well as several just-in-time technical missions to review specific issues that required particular attention. Implementation Status Reports were candid. Dialogue between the Bank and the government was reportedly effective throughout the project's lifetime, and focused on monitoring and facilitating achievement of results. According to the ICR (p. 27), "the government and World Bank teams developed an effective partnership to push reforms forward." Though there was frequent turnover of task team leadership (four TTLs and two co-TTLs over the project period), handovers were smooth and did not compromise implementation. Support from the Bank team was critically important during the initial 2.5-year period during which there was no TA firm. Quality of Supervision Rating Satisfactory Overall Bank Performance Rating Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design Page 11 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AR La Rioja Public Sect Streng Prog-APL1 (P121836) The project's theory of change was sound. The PAD (pp. 4-5, 8, 16-18) specified output and outcome indicators, with baselines and targets, that were comprehensive measures of achievement of the project’s objectives. EEP amounts, DLIs, and triggers for the second phase of the APL were also well defined. The target for the outcome indicator chosen to measure improved tax collection was expressed in nominal terms due to credibility issues with official statistics at the time; inflation led to the need later to adjust the target upward, and strong intermediate outcome indicators served as effective proxies to measure increased capacity for tax administration and compliance and for recording real property, all preconditions for improved tax collection. Institutional arrangements and responsibilities were clearly spelled out, with the provincial Ministry of Finance in charge of monitoring instruments (progress reports on implementation as well as output and outcome indicators), to be validated by a third party and an external assessment of compliance with DLIs and outcome indicators. b. M&E Implementation The Ministry of Finance reported on indicators as planned, with dedicated staff assigned for this purpose. Data were found by the Bank team to be reliable and of good quality. The ICR (p. 25) noted that "ownership, relevance, and effective use of the results framework during the entire life of the project was substantial and facilitated achievement of desired outcomes." c. M&E Utilization The government used M&E results to monitor compliance with commitments under the project and guide dialogue with the various institutions involved. Achievement of DLIs was used, as planned, as the trigger for budget release to the EEPs. M&E Quality Rating Substantial 10. Other Issues a. Safeguards The project was classified as category B for environmental assessment purposes. As well as OP/BP 4.01 Environmental Assessment, OP/BP 4.04 Natural Habitats, OP 4.09 Pest Management, OP/BP 4.11 Physical Cultural Resources, OP/BP 4.10 Indigenous Peoples, OP/BP 4.12 Involuntary Resettlement, and OP/BP 4.37 Safety of Dams were triggered. La Rioja’s indigenous population was estimated at 0.3-0.5% of the total provincial population, and the Indigenous Peoples safeguard was triggered to ensure understanding of the project’s impact. An Indigenous People Planning Framework was prepared, and a public consultation was held with about 40 civil society organizations in which participants are reported to have expressed agreement with the project. An Environmental and Social Management Framework was published and disclosed in December 2010 (PAD, p. 37), including a Sectoral Environmental Assessment Page 12 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AR La Rioja Public Sect Streng Prog-APL1 (P121836) for water resource management and a Resettlement Framework on a precautionary basis (PAD, pp. 36- 37). The ICR (p. 25) stated that the Bank's monitoring of social safeguards was "continuous and systematic from 2011 to closing," and that performance and compliance was Moderately Satisfactory; the project team later confirmed that some elements of the provincial legal framework still require updating. b. Fiduciary Compliance According to the ICR (p. 26), financial management performance was rated Satisfactory throughout the entire implementation period. Arrangements were in compliance with Bank requirements. Unqualified externally audited financial statements were submitted to the Bank in a timely manner. The final audit report for the project was received on time, and at the time of the ICR, some clarification was awaited from the Project Implementation Unit in the Ministry of Finance. The project team later confirmed that the final audit report was unqualified. Procurement procedures, standard bidding documents, and models of contracts for works, goods, consulting, and non-consulting services were included in the project's Operational Manual and properly used. Procurement Plans were submitted to the Bank, found to be acceptable, and properly implemented. There were some delays in selection of the TA firm (see Section 5), but that contract was eventually implemented successfully. There were no reported cases of misprocurement. c. Unintended impacts (Positive or Negative) None reported. d. Other --- 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Outcome Satisfactory Satisfactory --- Bank Performance Satisfactory Satisfactory --- Quality of M&E Substantial Substantial --- Quality of ICR High --- 12. Lessons Page 13 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AR La Rioja Public Sect Streng Prog-APL1 (P121836) The ICR (pp. 29-31) presented several insightful lessons, including: It is important to ensure that risk mitigation measures do not create other risks. In this case, the risk mitigation measure of bundling technical assistance activities into a single, large contract was smart compensation for limited implementation capacity, but delays then accrued because of a shortage of entities interested and eligible to bid. A more thorough market study during project preparation would have foreseen this development. There is value in direct World Bank support at the subnational level. In this case, institutional strengthening was invaluable for one of Argentina's provinces with the highest share of poor households, and the benefits of the Bank' direct engagement was recognized by high-level provincial authorities and public officials. Subnational public management strengthening is a prerequisite for improvements in service delivery in any single sector. In this case, reforms in cross-cutting public administration platforms and processes have been important foundations for strengthening rural water management, and other sectors (health, education, etc.) should benefit also from these basic public management reforms. IEG adds one additional lesson: Ongoing political economy analysis can help anticipate threats to development outcomes and build in sustainability measures prior to project closing. In this case, fiscal tightening threatens budget predictability and the articulation between budgeting and planning. 13. Assessment Recommended? No 14. Comments on Quality of ICR The ICR was clear, concise, evidence-based, and results-focused. It presented the project's storyline with clarity, marking the shortcomings with the project's results framework and providing additional data where required to demonstrate achievement of objectives. Its economic analysis included full detail on assumptions and intermediate steps in the calculations, and it included an appropriate sensitivity analysis. The insightful lessons articulated, which went beyond the generic lessons often found in ICRs, were drawn directly from the project's experience and should prove useful to teams in the region and beyond. a. Quality of ICR Rating High Page 14 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AR La Rioja Public Sect Streng Prog-APL1 (P121836) Page 15 of 15