IEG Report Number: ICRR14805 ICR Review Independent Evaluation Group 1. Project Data: Date Posted: 09/15/2015 Country: Dominican Republic Is this Review for a Programmatic Series ? Yes No How many operations were planned for the 3 series? How many were approved ? 3 Series ID: S125806 First Project ID : P125806 Appraisal Actual Project Name: Do-3rd Project Costs (US$M): 70.0 70.0 Perform.&accbilty Of Socsctrs Dpl L/C Number: Loan/Credit (US$M): 70.0 70.0 Sector Board: Social Protection Cofinancing (US$M): Cofinanciers: Board Approval Date : 11/17/2011 Closing Date: 11/30/2012 11/30/2012 Sector(s): Other social services (51%); General education sector (17%); Health (17%); Central government administration (15%) Theme(s): Social safety nets (50%); Other accountability/anti-corruption (15%); Public expenditure; financial management and procurement (15%); Education for all (10%); Health system performance (10%) Second Project ID :P121778 Appraisal Actual Project Name: Second Performance Project Costs (US$M): 150.0 150.0 And Accountability Of Social Sectors Deve L/C Number: Loan/Credit (US$M): 150.0 150.0 Sector Board: Social Protection Cofinancing (US$M): Board Approval Date : 11/23/2010 Cofinancers: Closing Date: 11/01/2011 11/01/2011 Sector(s): Other social services (39%), Health (33%), General education sector (28%) Theme(s): Social safety nets (50%), Health system performance (29%), Education for all (21%) Third Project ID :P116972 Appraisal Actual Project Name: First Performance And Project Costs (US$M): 150.0 150.0 Accountability Of Social Sectors Deve L/C Number: Loan/Credit (US$M): 150.0 150.0 Sector Board: Social Protection Cofinancing (US$M): Board Approval Date : 11/17/2009 Cofinancers: Closing Date: 12/10/2010 12/10/2010 Sector(s): Other social services (39%), Health (33%), General education sector (28%) Theme(s): Other public sector governance (36%), Health system performance (29%), Education for all (21%), Social safety nets (14%) Evaluator: Panel Reviewer: ICR Review Group: Coordinator: Lev Freinkman Robert Mark Lacey Lourdes N. Pagaran IEGPS2 2. Project Objectives and Components: a. Objectives: According to the Program Document (PD, p. 45), the Performance and Accountability of Social Sectors (PASS) Development Policy Lending (DPL) series of three operations had four inter-related objectives: a) to enhance the performance of social sectors to promote human capital (health, nutrition, education) for the poorest citizens, through a fundamental redesign of the Government’s Conditional Cash Transfer (CCT) Program, Solidaridad, and its articulation with critical actions in health and education; b) to improve budget management to support the performance of these social sectors within the CCT program; c) to support the gradual implementation of Performance Agreements in social sectors; and d) to enhance transparency and accountability to users in social sectors by strengthening the enabling environment for a better informed demand for improved public sector performance in the social domain. b. If this is a single DPL operation (not part of a series), were the project objectives/ key associated outcome targets revised during implementation? No c. Policy Areas: The policy reforms supported under the PASS series were grouped in the four policy areas consistent with the program’s objectives. These policy areas remained unchanged during the course of the program. (I) Enhancing the performance of social sector agencies to promote equitable access to human capital This policy area supported the reorganization of the country’s social transfer program Solidaridad into a full-fledged and efficient conditional cash transfer (CCT) program and simultaneously providing for expansion of services to CCT beneficiaries, including in education and health.  Reforms in transfer administration included introduction of new transfer eligibility rules (linking them to recipients’ use of health and education services), changes in management responsibilities across key government agencies, establishing an integral system of monitoring and evaluation, introduction of verification mechanisms to ensure beneficiaries’ compliance with their new co-responsibilities, and improvements in targeting mechanisms.  New supply side policies including improvements in monitoring of learning outcomes in primary and secondary education, introduction of the new system of budget transfers to subnational governments aimed at efficiency improvements in public spending in education, and setting-up a new registration system for primary health visits. In addition, the government expanded the non-cash benefits to CCT recipients by broadening the network of businesses that accept the debit card of the program and improved the recipients’ access to the information on their transfer amounts and balances. (II) Improving budget management to support the performance of the CCT program This policy area aimed at boosting budget allocations for social sectors to meet the entitlements introduced under the CCT program and ensuring that the budget for CCT is executed as programmed. The main actions included the development of multi-annual projections of supply gaps in health, nutrition and education, adoption of adequate annual budgets to cover these gaps, and strengthening public financial management (PFM) procedures to ensure implementation of these budgets in a disciplined way. (III) Supporting the introduction of performance agreements in social sectors This policy area was focused on enhancing performance orientation of the budget and management processes in social sectors. The performance management agreements were introduced in the Ministries of Education and Health, backed up by performance-based budgeting and aimed at strengthening incentives for better decision making and stronger accountability for public spending. In parallel, a system of internal management agreements was introduced between the Ministry of Health and regional Departments for Health Services to adopt elements of result-based management. (IV) Improving transparency and oversight in the CCT program This policy area was intended to strengthen public demand for improved performance in social sectors. In particular, the program promoted disclosure of information on both the national budget and CCT program budget performance through a web-based tool. It also supported the introduction of a social auditing scheme to strengthen accountability for the provision of educational and health services at the community level. d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: The financing for this series totaled US$370 million. PASS1 was appraised in September 2009, approved for US$150 million equivalent by the Board on November 17, 2009, was declared effective and disbursed on December 1, 2009, and closed on schedule on December 10, 2010. PASS2 was appraised in September 2010, approved for US$150 million equivalent by the Board on November 1, 2010, was declared effective and disbursed on December 21, 2010, and closed on schedule on November 1, 2011. PASS3 was appraised in September 2011, approved for US$70 million equivalent by the Board on November 17, 2011, was declared effective and disbursed on February 8, 2012, and closed on schedule on November 30, 2012. The Country Partnership Strategy (page 33) originally envisioned that the PASS series would include four operations. PASS4 in the amount of $70 million equivalent was expected to be disbursed in 2012, but eventually was dropped. The ICR (p.16) provides two reasons for this discontinuation: a) deterioration of the macro/fiscal framework (the IMF program was discontinued in early 2012 – see Section 3b below), and b) an assessment that sustainability of the government’s social policies supported by the PASS program had been achieved, making additional policy-based lending unnecessary. The PASS program was strongly aligned with that of other development partners (including IADB and EU), but there were no formal co-financing arrangements. 3. Relevance of Objectives & Design: a. Relevance of Objectives: Substantial. The PASS program was relevant to the objectives of the Country Partnership Strategy for the Fiscal Years 2010-2013 (CPS FY10-13), which was prepared in parallel with the PASS-1 operation. The four development objectives of the program were closely aligned with three out of four strategic objectives of the CPS (strengthen social cohesion and improve access to and quality of social services, enhance quality of public expenditures and institutional development, and build capacity and constituencies for reform). The CPS FY10-13 explicitly recognized the PASS program as the key instrument of delivering IDA support in the area of human capital development (p. 27). The PASS objectives were also closely linked with two main pillars (social cohesion and expenditure quality) of the previous CAS (FY06-09). At the same time, the overall objectives of the PASS series were fully aligned with the social and institutional development pillars of the Government’s National Development Strategy (NDS), which calls for broadening access to key public services, improving the quality of public expenditures, and promoting good governance. Furthermore, the program was consistent with various government sectoral reform strategies, such as a 10-year strategic plan in education (Plan Decenal de la Educación) approved in 2007. b. Relevance of Design: Substantial. The causal chain of the operation linked fiscal and policy support from the Bank to implementation of a comprehensive package of policy reforms supporting a major shift in the country’s social policy. The thrust of the planned reforms was towards a conditional cash transfer (CCT) system that is capable of reaching the neediest citizens and creating incentives for the poor to invest in their human capital. The PASS policy package addressed the problem of an inefficient social protection system in a holistic manner by including interventions to restructure the existing system of social benefits, strengthen incentives of beneficiaries to improve utilization of core public services, increase budget allocations on health and education in order to meet the expected increases in demand, and reform of the benefits administration to ensure improved targeting, monitoring of service quality, and strengthening broader accountability mechanisms. Designing its support to social reforms around the program of high-volume policy-based lending was appropriate, given Dominican Republic’s large needs in both budget support and policy adjustment. The design and selection of specific PASS-supported policies was backed up by considerable amount of analytical work and technical assistance. The prior actions were substantively linked to the program objectives, practically all of them had a meaningful regulatory impact, and they were largely based on the triggers that for PASS-2 and -3 were identified in the Program Documents (PDs) for the previous operations. In a couple of instances the original triggers were dropped, but those were mostly replaced with equally relevant prior actions. There were, however, four moderate shortcomings in program design: a) The original policy matrix in PD for PASS-1 was quite fragmented. It suggested 19 triggers for PASS-2 and 18 for PASS-3. These triggers were then consolidated into ten prior actions for each operation. Such consolidation did not result (with one exception, see below) in weakening of the program content, which suggests that the original program structure was excessively detailed and had to be better focused (ICR, p. 25). b) The reform program in the health sector supported under PASS-2 and -3 was significantly weakened against the original design presented in PD for PASS-1. Out of four original triggers for this sector only one was kept as the prior action for PASS-3. In particular, the PASS program dropped all policy actions to improve vaccination rates, which is a social area where Dominican Republic has shown significant under-performance. c) The original PASS policy matrix did not provide for any policies to improve targeting in the benefit administration. This was added as a prior action under PASS-3. d) The program design was overambitious with respect to the expected pace of development of integrated M&E system. Complexities of developing such a system in a low capacity environment were under-estimated (ICR, pages 13-14). Macroeconomic framework. When PASS-1 was approved, the country’s macroeconomic framework was viewed as sustainable despite considerable government financing needs (6.4% of GDP). In the IMF’s assessment, under an earlier IMF program (2005-08) the government’s prudent macroeconomic policies in a favorable external environment restored confidence and delivered strong macroeconomic performance, with rapid GDP growth and single-digit inflation. A subsequent program - a 28-month Stand-By Arrangement (SBA) - was therefore approved by the IMF board in November 2009. Its implementation in 2010-11 was affected by delays in reducing the energy sector deficit and slower than expected improvements in tax collection. Ultimately, the SBA program was discontinued in early 2012, as the government breached some of its core performance criteria. Moreover, in 2012 a rise in pre-election spending resulted in further widening of fiscal deficit (7.9% of GDP). This deterioration of fiscal performance is one of the reasons why PASS-4, originally programmed for 2012, did not take place (ICR, p.16). 4. Achievement of Objectives (Efficacy): Objective 1: Enhancing the performance of social sectors to promote human capital (health, nutrition, education) for the poorest citizens Substantial Achievement of this objective is measured according to two criteria: a) greater use of social services by the poor and vulnerable, and b) improvements in management of the CCT program itself (monitoring, targeting, benefit payment). Use of social services In health, despite some progress being recorded against the baseline of 2009, most program targets remained unmet. The share of pregnant women among CCT beneficiaries receiving prenatal checks reached only 68% by end-2012 versus a target of 94%, while the share of puerperal women receiving postnatal checks was 44% (vs. target of 50%). The share of children under 2 with the complete vaccination set was 40.3% in 2012 (target – 65%). Progress on ‘health checks for elderly’ was not reported in the ICR due to a lack of data. The only health target met relates to the share of children under 5 who comply with the standard set of medical checks – this indicator reached 65.3% vs. the target of 65.0%. It is worth noting that, as the ICR states, the above data may significantly underestimate those who actually received health services due to weaknesses in the national health reporting (ICR, p.17). In the area of nutrition (three indicators related to specific groups of women and children among CCT beneficiaries), the program targets also were mostly unmet, despite being revised downwards at the time of PASS-3 approval. Specifically, the share of children under five, who receive micronutrient supplements, shows a significant gap against the targets – 12.6% vs. 80%. This is explained in the ICR (p.18) by the delay in the implementation of the nutrition program supported by the World Food Program. In education, the program’s progress towards its own targets was somewhat stronger. Attendance targets were exceeded for all educational levels (with actual attendance rates of 93-96% vs. targets of 70-90%). Enrolment targets were exceeded for compulsory basic education (95.9% vs. 85%), but the performance fell short of targets for pre-school and lower secondary education. Overall, there is a sense that the PASS program made a significant contribution to broadening the poor CCT beneficiaries’ access to critical social services, but the program’s own results framework makes it difficult to appreciate the actual scale of program achievements in this area. This is due to major weaknesses in the M&E system (see section 10 for details). The program targets for access to social services were too ambitious, as they were defined in the absence of reliable baselines. Moreover, progress under the program was measured on the basis of the existing government data collection and reporting system, which resulted in systematic under-estimation of actual improvements. For instance, in the area of pre-school education the share of CCT children enrolled in early childhood programs increased from 14.1% in 2009 to 35.6% in 2012, a significant achievement. Still, it was short of the program target (50%), which was probably unrealistic. In addition, the results framework insufficiently relied on specially designed household surveys and comparisons between the CCT beneficiaries and the control groups. When such comparisons were undertaken (ICR, p.15), they showed significant differences in the rates of improvements between CCT recipients and the control group for a number of indicators of access. Furthermore, as explained by the task team, the results of Program Social Impact Assessment (PSIA) confirmed a major improvement in access among CCT beneficiaries – for instance, compliance with education co-responsibilities, increased primary school enrollment among rural poor by 6 pp., and decreased child labor by 9.2 pp.; while health co-responsibilities doubled incidence of medical checks for children under 5 (from 32 to 66%). Overall, successful mitigation of the negative impact of the 2008-09 crisis on poverty and human capital of the poor represents a significant achievement of the program. In contrast to the previous crisis (2003), extreme poverty did not rise during the recent series of shocks due in part to increases in the CCT transfers and the number of beneficiaries (PD for PASS-2, p. 15). The number of CCT beneficiaries has doubled between 2009 and 2014. Due to its strong targeting performance, the CCT program covers more than 90% of extremely poor households and about 80% of the poor. It is estimated that the transfer program was directly responsible for a one percentage point reduction in the poverty rate (ICR, p. 21) below what it would otherwise have been as a result of the economic and financial crisis. However, these program achievements were unreported in the results framework. Management of the CCT program The Government’s ability to monitor and verify compliance of program beneficiaries with their co-responsibilities has improved. The program target (after the downwards revision in 2011) was to ensure that at least 80% of transfers were paid to the recipients, whose compliance with co-responsibilities was verified. In 2012, 85% of respondents complied with their co-responsibilities in health and 80.7% with co-responsibilities in education. However, at program closure, the Government still did not have in place a permanent system for such verification as the completion of the new M&E system was delayed (See section 10). In 2011-12 verification was undertaken through interim arrangements, relying on regular sample-based audits of beneficiaries. Three other aspects of CCT program management were upgraded. First, the Government significantly expanded the network of stores where CCT beneficiaries can spend their cash transfers (by using their program debit cards). By the end of the PASS series, the respective target (3,067) had been exceeded, with almost 5,000 establishments included in the network. Second, the Government succeeded in setting-up a monitoring mechanism to control potential problems in the benefit payment system. It signed an agreement with a consumer protection organization (pro-Consumidor) to monitor the payment services and issue regular reports. This agreement has been fully operational. In addition, there is an established mechanism in place to deal with beneficiaries’ complaints over the payments. The community scorecards instrument, implemented under the PASS program, revealed a high overall level of satisfaction of CCT beneficiaries with quality of services received under the program. Third, in the area of benefit targeting, the program objective (‘Update the database (SIUBEN) used to select the beneficiaries of the CCT program’) was achieved. The database information was updated for households who were in the original dataset. Moreover, the coverage of SIUBEN was expanded to add more than 300,000 households (about 20% growth in coverage). Overall, improvements in targeting in the course of PASS implementation were quite significant. It is estimated that the share of social assistance spending that targeted the poor increased from 6% in 2005 to 53% in 2011 (ICR, p. 21). On balance, given that the improved management efficiency targets were exceeded and the use of social services by the poor has improved, the efficacy of Objective 1 (“enhancing the performance of social sectors to promote human capital for the poorest citizens”) is rated substantial, despite the fact that several specific targets concerning use of social services, some of which were inappropriate, were not met. Objective 2: Improving budget management to support the performance of social sectors within the CCT program High The program aimed at expanding public spending in social sectors to close the supply gaps in services for CCT beneficiaries. In the event the Government made significant shifts in its fiscal policies, making its budget much more socially-oriented. For instance, budget educational spending (as a share of GDP) almost doubled between 2009 and 2014, from 2.24% to 4.40% (based on the report from the Ministry of Education). The Government developed new budget procedures that effectively prioritize budget allocation to the health, education and nutrition services. In addition, there was a major improvement in budget execution discipline to reduce risks of within-the-year budget revisions. The specific targets under the PASS programs were related to improvements in budget execution and were measured by two specific indices. The first reflected a degree of relative protection of priority budget programs for closing supply gaps from within-the-year revisions (it compared annual budget deviation ratio for priority programs with the same ratio for the entire budget). As the Ministry of Finance followed a policy of almost full protection of priority supply gap programs, the 2012 deviation ratio for these programs was as low as 1.6% (vs. 19.6% for the entire budget), bringing the value of the index to 0.08 (1.6/19.6) vs. the target of 0.85. The second index measured a degree of absolute protection of the same priority budget programs, reflected by the standard budget disbursement ratio. Again, due to the high level of protection of priority programs, the respective target was exceeded: the disbursement ratio in 2012 was 0.98 (best potential outcome - 1) against the target of 0.85. There is, therefore, convincing evidence that: (i) there were considerable actual increases in social spending, which were focused on providing additional funding for priority social programs; (ii) these priority programs were developed, inter alia, to close funding gaps associated with the provision of additional core services in health and education to poor CCT beneficiaries; (iii) the priority social programs were fully funded during budget execution (deviations were almost zero), despite the effects of the economic and financial crisis; and (iv) CCT beneficiaries indeed increased their use of core services. Objective 3: Supporting the gradual implementation of Performance Agreements in social sectors Substantial The Government made gradual progress in introducing the core elements of performance-based budgeting in the operations of social ministries. In late 2011, it was agreed to introduce five performance agreements as part of the 2012 budget. These agreements were signed between the Ministry of Economy, Planning and Development and Ministries of Education (three agreements -- for pre-primary, basic and secondary education programs) and Health (two agreements -- programs for immunization and maternal and infant care). Furthermore, for two other social programs (nutrition and child labor eradication) respective Ministries have put in place their logical frameworks (detailed, structured programs, in which the main performance indicators for the sectoral agencies are identified), as an intermediary step towards the preparation of additional performance agreements. The respective PASS program target provided that by 2012 at least six social programs related to the CCT program would develop log-frame methodology as a basis for introduction of performance agreements with the respective Ministries. The ICR, however, provides no information on the substance/quality of the signed performance agreements. The practical experience of implementation of performance agreements by the social Ministries (after the PASS program closure) remains also unclear from either the program documentation or the IEG meeting with the team. Objective 4: Enhancing transparency and accountability to users in social sectors Substantial The program supported new accountability policies in two separate areas: overall budget transparency and oversight within the CCT program. Budget transparency. The Government has set up a website for publication of the national budget data and made an effort to keep its information up to date while providing increasingly more budget disaggregation along alternative budget dimensions. Local NGOs received training in analysis of budget data. The site’s data coverage is rather comprehensive and goes beyond the social sectors supported by PASS. The website has registered over 15,000 visits in 2012 (against the target of 1,500). However, the website does not yet support registration of its users. Therefore, it remains impossible to monitor progress towards another program target in this area (‘to reach 60 independent registered users of the site’). Overall, the Dominican Republic’s recent progress towards greater budget openness and transparency was substantial. The country’s Open Budget score increased from 12 (out of 100) in 2008 to 29 in 2012. Oversight of the CCT program. Targeting issues (including qualifying criteria for benefits) and the reliability of the payments system were discussed under Objective 1 above. The primary instrument for strengthening accountability of CCT program was implementation of the community scorecard tool. The tool was successfully introduced and became the integral part of the program management system (ICR, p.20). The Government then decided to expand its coverage and approved the 2013 expansion plan. The Government also established local committees to address grievances raised by the scorecards. In 2012 the share of distributed scorecards completed and returned by community committees of CCT beneficiaries was 78.5% vs. the target of 70%. It is worth noting that, according to scorecards, CCT beneficiaries express high level of satisfaction with the program. For instance, 88% of collected responses stated that the program social workers were helpful in facilitating access of beneficiaries to health and education services (information obtained from the team). 5. Efficiency (not applicable to DPLs): 6. Outcome: The outcome rating of the program is satisfactory, reflecting substantial relevance of both objectives and design, high efficacy of the second program objective, and substantial efficacy of the other three objectives. The program helped the Borrower to make significant progress in restructuring its social protection system by introducing a modern program of conditional cash transfers. Introduction of co-responsibility requirements for beneficiaries and expansion in government spending in core social sectors brought about tangible improvements in access to basic services for the poor. In addition, in the course of the program, significant improvements were recorded in the quality of budget management, targeting of social spending, transparency and accountability within the social assistance system, and other governance elements. a. Outcome Rating: Satisfactory 7. Rationale for Risk to Development Outcome Rating: The risk to the outcome rating is moderate given the strong Government’s commitment to the policy reforms supported under the PASS program. After the 2012 presidential elections, the new Administration renewed its commitment to reforms in social assistance and poverty reduction. The Government announced the creation of a Ministry of Social Development to further advance institutional framework for social reform implementation (ICR, p.22). Furthermore, in order to ensure reform sustainability, the Government secured an additional package of assistance from the World Bank under the new Integrated Social Protection and Promotion Project (P147213) approved by the Board in March 2015. The primary risk to the program outcome relates to macroeconomic and fiscal vulnerability of Dominican Republic, which reflects large financing requirements (both domestic and external) and the entrenched deficit of the publicly-owned electricity sector. a. Risk to Development Outcome Rating : Moderate 8. Assessment of Bank Performance: a. Quality at entry: The PASS program was prepared on the basis of a well-established policy dialogue with the Authorities, extensive analytical work, and close coordination with the development partners (EU, IADB, USAID). It was part of the Bank’s strategy to provide the government with a package of budget support, knowledge services and technical assistance (See PD for PASS-1, pp.42-44; ICR, p.12). The PASS program complemented activities undertaken by a number of Bank investment projects in social protection, education, and health (ICR, pp. 3-4). The design reflected recommendations of a 2006 Poverty Assessment, a review of the framework for performance budgeting (2009), and a Policy Note on political economy of reforms in education. PASS-1 was prepared on an accelerated basis to provide urgent support to the Dominican Republic, which was affected by the global financial crisis of 2008-09. A moderate weakness of Quality-at-entry relates to the deficiencies in the program’s Results Framework, including: (i) lack of reliable baseline data at the time of PASS-1 approval; (ii) over-ambitious targets in Program area 1 (access to social services); (iii) gaps in the system of monitoring indicators that did not provide for monitoring critical dimensions of program outcomes, including the poverty impact of reforms in social protection, (iv) excessive reliance on the Government’s own reporting systems that remain under-developed over the most of PASS implementation period; and (v) absence of indicators directly related to program development objectives. There was also a lack of attention to the policies that would improve CCT targeting (this was addressed during the implementation). Quality-at-Entry Rating: Moderately Satisfactory b. Quality of supervision: Four Implementation and Status reports (ISRs) were prepared. The team demonstrated reasonable flexibility during implementation by re-prioritizing and consolidating the set of prior actions for PASS-2 and -3, in particular by dropping those which were completed by the Government earlier than it was originally expected. These modifications overall did not weaken the reform content of the program. Two programmatic Non-lending TA programs, implemented in parallel with the PASS, were useful in supporting participating government agencies in meeting their policy commitments under the program. As part of supervision, the team also undertook an active donor coordination effort. Partnership with the IADB was particularly effective in monitoring and verification of various core program outcomes. Quality of Supervision Rating : Satisfactory Overall Bank Performance Rating : Moderately Satisfactory 9. Assessment of Borrower Performance: a. Government Performance: Government ownership and commitment to the PASS supported reforms was consistently strong. The authorities used the 2008-09 crisis as an opportunity to accelerate the reforms in social protection and raise efficiency and effectiveness of budget spending in social sectors. The Government performed well in timely adoption of the agreed policy actions and was very effective in enhancing the coordinating role of the Social Cabinet that helped to improve collaboration across various government ministries and agencies. The political leadership of the Vice-President office was quite instrumental to the success of the PASS (ICR, pp. 24, 26). High-level political mandate helped counteract interest groups’ resistance to reforms (PD for PASS-3, p.15). The Ministry of Finance developed and adopted new budget procedures that provided protection of budget allocations for priority social programs from sequestration. However, the program implementation was affected by deterioration of the Government’s fiscal performance in 2011-12, which was one of the reasons why PASS-4 did not take place. One minor shortcoming concerned delays in developing a government-management information system, a core part of the M&E system for the program, due to changes in the M&E system and the over-complex sectoral management information systems in the Ministries of Education and Health. The task team confirmed that these information systems had become fully operational after the ICR had been completed. Government Performance Rating : Satisfactory b. Implementing Agency Performance: The implementation of the program was coordinated by the Social Cabinet, the institution attached to the Office of Vice-President. Two special committees were established to enhance inter-agency coordination, both of which chaired by Vice-President. Coordination and inter-agency cooperation was effective. Implementing Agency Performance Rating : Satisfactory Overall Borrower Performance Rating : Satisfactory 10. M&E Design, Implementation, & Utilization: a. M&E Design: The M&E design shows significant weaknesses. First, the results framework did not contain any PDO-level (core) results indicators. Instead it included 22 specific indicators reflecting various particular aspects of program results (presented in Annex 2 of PD for PASS-1). Second, the composition of this indicator set was not well aligned with the program objectives. On the one hand, several indicators were output-oriented, especially for policy areas 3 and 4 (such as e.g. number of performance agreements signed, number of scorecards filled, etc.). On the other hand, various critical parameters of the CCT system being reformed were not covered by these indicators (quality of targeting, degree of CCT beneficiaries’ satisfaction, etc.). Third, at the time of program approval, the program did not contain any baseline due to the lack of reliable household survey data. Such a survey was undertaken only in 2010 and its results only became available at the time of the PASS-3 approval in 2011. Fourth, because of the absence of a reliable baseline, the program targets established under Policy area 1 were over-ambitious and unrealistic. Finally, the M&E design relied excessively on the Government’s own reporting systems (vs. independent survey-based verification), which had weaknesses. This proved especially problematic in the situation when the government decided to develop new sectoral informational systems that took a long time to complete. b. M&E Implementation: The M&E implementation was undertaken primarily on the basis of the Government’s own reporting on the program’s monitoring indicators. It was coordinated by the CCT Inter-sectoral Committee under the Social Cabinet. Most of the agreed monitoring indicators were collected annually and presented in the PD for PASS-2 and -3. Given the delays in implementation of the new government’s information systems, there was a gap in the monitoring of several indicators that were supposed to verify compliance by CCT beneficiaries with their co-responsibilities. This gap was filled with the additional effort to audit the CCT program (sample-based), supported by the development partners (ICR, pp. 15, 18). The limited number of M&E indicators was reported in the ISRs prepared by the team during supervision. The ICR presents information on 19 out of 22 original indicators. c. M&E Utilization: According to the ICR, the program’s M&E system was fully internalized by the government, which enhanced its capacity to use the respective monitoring tools and helped the authorities to make (with delays) progress in monitoring and evaluation of the core social assistance programs (ICR, pp. 15-16). M&E Quality Rating: Modest 11. Other Issues a. Safeguards: No safeguard policies were triggered by this DPO series. b. Fiduciary Compliance: The ICR does not discuss fiduciary issues related to the program implementation. c. Unintended Impacts (positive or negative): There were significant positive unintended impacts under the program. First, the Government started to use the management tools and related infrastructure developed under the PASS (such as targeting and payment mechanisms) in managing its other important social benefits for the poor, including subsidized health insurance and energy subsidies. This resulted in additional improvements in efficiency of the country’s overall social protection system that go beyond the CCT reforms (ICR, p. 19). Second, the new CCT system had strengthened the demand among the poor for obtaining their identification documents (in particular, among the women), which has longer-term implications for broadening their opportunities for social participation (ICR, pp. 21-22). d. Other: 12. Ratings: ICR IEG Review Reason for Disagreement/Comments Outcome: Satisfactory Satisfactory Risk to Development Moderate Moderate Outcome: Bank Performance: Satisfactory Moderately There were moderate shortcomings in Satisfactory Quality at Entry, including a weak results framework. Borrower Performance : Satisfactory Satisfactory Quality of ICR: Satisfactory NOTES: - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 13. Lessons: The following lessons are taken from the ICR with some adaptation of language:  A programmatic approach could be highly effective in supporting national reforms in social protection aimed at step-by-step introduction of the CCT system. The program, however, need not be limited to DPO lending, but would need to utilize complementarities with other Bank instruments, including investment lending, non-lending technical assistance, and advisory work.  Implementation of a reliable M&E system, covering a decentralized system of service providers, is a critical component of transition to the full-fledged CCT system. Development of such an information system in a low capacity environment could become a reform bottleneck if not well-designed and planned in advance. Given the long time needed to complete a modern information system, the option of a simpler interim solution, which uses survey instruments, could be considered at the stage of reform design.  A major economic crisis could provide a window of opportunity to consolidate political support for an ambitious social reform agenda. Availability of timely policy advice from the Bank, based on the high-quality analytical work completed earlier, might be instrumental for utilization of this opportunity and for helping the Government to promptly develop reform implementation plans. 14. Assessment Recommended? Yes No 15. Comments on Quality of ICR: The ICR presents a detailed and frank analysis of program results and implementation experiences, but it has three moderate shortcomings. First, the ICR did not provide ratings for the program efficacy by individual program objective, presenting only the aggregate Outcome rating. Second, while the ICR contains description of M&E developments within the government’s social protection system (ICR, pp.14-16), its discussion of the M&E system used by the team to track progress under the Bank’s PASS operations remains limited. Third, the ICR missed the opportunity to highlight the broader impact of the PASS program on poverty and access to core services in Dominican Republic. This could have been done by bringing in some core results of the Program Social Impact Assessment (2012) that provided in-depth analysis of changes in lives of CCT recipients taken place since the CCT reforms were initiated. a.Quality of ICR Rating : Satisfactory