ICRR 13343 Report Number : ICRR13343 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 03/19/2010 PROJ ID : P103466 Appraisal Actual Project Name : Prsc 2 US$M ): Project Costs (US$M): 42.0 42.6 Country : Mali Loan/ US$M ): Loan /Credit (US$M): 42.0 42.6 Sector Board : PO US$M): Cofinancing (US$M ): Sector (s): Central government administration (38%) General transportation sector (25%) General finance sector (13%) Health (12%) Irrigation and drainage (12%) Theme (s): Public expenditure financial management and procurement (29% - P) Regulation and competition policy (29% - P) Trade facilitation and market access (14% - S) Standards and financial reporting (14% - S) Rural services and infrastructure (14% - S) L/C Number : C4388 Board Approval Date : 02/05/2008 Partners involved : Closing Date : 12/31/2008 03/31/2009 Evaluator : Panel Reviewer : Group Manager : Group : Rene I. Vandendries Kris Hallberg Ismail Arslan IEGCR 2. Project Objectives and Components: a. Objectives: The Program Development Objectives of PRSC 1 and PRSC 2 were essentially the same. The operations supported implementation of Mali's Growth and Poverty Reduction Strategy Framework (GPRSF), including policy measures intended to: (1) strengthen the macroeconomic and budget frameworks, particularly their links to the poverty reduction strategy, and strengthen public finance management, namely management, accountability and transparency of budget execution as well as procurement; (2) support growth enhancing measures in the financial sector, private sector, electricity (only in PRSC 1), transport and transit facilitation, Office du Niger irrigation, and cotton sectors; and (3) improve access to basic social services, notably health and social development, and rural water supply and sanitation . b.Were the project objectives/key associated outcome targets revised during implementation? No c. Components (or Key Conditions in the case of DPLs, as appropriate): As mentioned, the two PRSCs were focused on three major policy areas . Policy area 1: Budget framework, poverty reduction strategy and public financial management - The objective of the PRSCs in this area was on strengthening the link between the poverty reduction strategy and the budgets through development of MTEFs, and on improving the financial management system through better monitoring and reporting. In the procurement area, the focus was on information improvement and better control and regulatory functions. Policy area 2: Support to growth - The areas covered by the PRSCs were the financial sector, private sector development, transport and transit facilitation, the energy sector (only in PRSC 1), the agricultural sector, and the Niger River Irrigation Office (ON). Financial sector - The main objectives were cleaning up the non -performing loan portfolios in the banking system, and further efforts at restructuring two major state -owned banks, the International Bank for Mali (BIM), a commercial bank, and the Housing Bank (BHM). Private sector development - The focus was on a variety of regulations that enhance (or constrain) business activity, as reported in the Doing Business reports . Studies on corporate taxation and labor market competition were also planned under PRSC 2. Transport and transit facilitation - The main objectives were to strengthen road maintenance, strengthen incentives for private sector transport operators and improve customs administration and control . The energy sector - PRSC 1 supported the government's plan to restructure the water and power utility (EDM), following failed privatization efforts in the early 2000s. The cotton sector - The PRSCs focused on continued implementation of a 2005 producer price mechanism to help protect farmer income, and preparation for the privatization of the state cotton company, CMDT . The irrigation sector (ON) - The objectives were to improve ON management and promote irrigation, including private irrigation development . Policy area 3: Access to basic social services - The main focus in this area was on ensuring adequate budget resources for the ministries of health and of social development to help improve social service delivery, and on increasing reliable access to potable water for the rural population . d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: PRSC 1 was approved on March 6, 2007 for US$45 million equivalent, fully disbursed upon effectiveness, and closed on schedule on March 31, 2008. The difference between the appraisal amount (US$45 million equivalent) and actual disbursements (US$45.9 million) is due to exchange rate changes . PRSC 2 was approved on February 5, 2008 for US$42 million equivalent, fully disbursed upon effectiveness, and closed on March 31, 2009 with a three month delay. The difference between the appraisal amount (US$42 million equivalent) and actual disbursements (US$ 42.6 million) is due to exchange rate changes . IDA provided supplemental grant financing for PRSC 2 of US$5 million in December 2008 (additional to the US$ 42 million) to help fill an unanticipated financing gap as a result of a food crisis . There was no cofinancing. 3. Relevance of Objectives & Design: The original intent was a series of three PRSC operations . However, because of limited progress being made in the growth support agenda (policy area 2) the series was discontinued after PRSC 2 and a new PRSC series was initiated with focus on public finance management, governance and the delivery of basic services . The objectives of PRSC 1 and 2 were relevant in the sense that they were in line with the government's objectives, were complemented by several ongoing bank -financed projects in relevant sectors (finance, transport, growth, agriculture) and were closely coordinated with IMF programs . But the relevance of both objectives and design were seriously compromised in two ways . First, the PRSCs were aimed to address too many issues all at once, many of which had been addressed in many earlier Bank operations and proven intractable . In other words, there was no focus. Second, and partly as a result of the above, the design of the PRSCs was complex, focused on processes rather than outcomes, and linkages between these processes and indicators of achievement were often vague . To illustrate, there were 13 prior actions for PRSC 1 and nine for PRSC 2, but most of these were mere indications of intent to reform rather than concrete reform measures --adopt the findings of a report, meet with the banks, study an issue . Furthermore, the number of PDO indicators was inordinately large, 55 in PRSC 1 and 54 in PRSC 2 (the power sector was dropped from consideration ) while data to measure results were often not available, thereby defeating the objective of promoting results -based management. The ICR recognizes these shortcomings when it notes ( in para. 97) that the PRSC series "did not make a real break from the past" in terms of lack of selectivity and a focus on processes rather than outcomes, which led to results that were similar to those achieved during the previous ten years . Lessons from previous operations were clearly not taken into consideration . In sum, the relevance of objectives and design of PRSC 1 and PRSC 2 was modest, at best. 4. Achievement of Objectives (Efficacy): Policy area 1: Budget framework, poverty reduction strategy and public financial management - There was some progress in this area in expanding the MTEF process, thereby improving the dialogue between sector planners and the Ministry of Finance, and strengthening the link between poverty reduction and sectoral strategies. However, alignment of the budget with MTEFs remains deficient and budget implementation remains slow as does reporting on budget execution . In terms of procurement reform, the CPAR was adopted under PRSC 1 and the Procurement Department database was strengthened; a procurement decree was approved under PRSC 2 and databases installed in four ministries. But there has been no implementation progress to date . Efficacy for policy area 1 is rate modest. Policy area 2: Support to growth - In the financial sector, one major commercial bank (BIM) was successfully privatized, but attempts at restructuring of the Housing Bank (BHM) have continued to fail. The target for the ratio of non-performing loans in the banking system's portfolio was not met primarily because of the lack of progress in restructuring BHM. Progress towards improving the investment climate was minimal. There were numerous PDO indicators in this area such as the number of procedures necessary to create an enterprise, the time taken to create an enterprise, tax measures, or the cost of creating an enterprise . The target for the latter indicator was the only one met. There was no significant change in Mali's ranking on the Ease of Doing Business indicators over the PRSC period . Studies planned as part of the operations on corporate taxation and on competitiveness of the labor market were not completed . Little progress was also made towards increasing the efficiency of the transport and transit system . The share of the road network that is maintained has increased marginally from 65 to 66 percent (meeting the target) but road maintenance remains a major issue with 40 percent of the roads that are maintained reported to be in poor condition. There are no data to judge developments in the areas of transit and customs . Road barriers and irregular practices at these barriers remain an issue, and planned regulations were not implemented; nor were planned studies on road taxation and on a competitive freight exchange completed . PRSC 1 supported the government's action plan for the restructuring of the power and water company, including technical studies on institutional arrangements, tariffs, least -cost investment plans, and audits . But the financial situation of the company has worsened further . The government has now adopted a restructuring plan and Bank support will be provided through an investment loan . In the cotton sector area, several of the PDO indicators set were either not met or no data exist to evaluate progress . Most importantly, the cotton parastatal (CMDT) has continued to suffer from poor management and large financial losses, hence requiring further subsidies, contrary to the objectives of the PRSC series . An earlier (2005) reform time table was not implemented as scheduled. Cotton production has declined precipitously . The privatization of CMDT is now embedded in a newly approved law . There has been little progress towards improving the effectiveness of the irrigation department (ON). Irrigation maintenance remains a major challenge primarily because of a lack of government assistance . A new master plan for development of ON was adopted in December 2008, but ON's role and responsibilities in many areas--water management, land tenure, land use --remain unclear. On balance, efficacy in policy area 2 is rated negligible. Policy area 3: Access to basic social services - Some progress was made in this area . PRSC 1 focused on ensuring additional budget resources for the health and social development ministries, while PRSC 2 helped ensure that the budget for the two ministries was aligned with their MTEFs and, in addition, included actions to increase the supply of potable water . The resources allocated to the two ministries were slightly short of target for PRSC 1 but were on target for PRSC 2, and the target rate of assisted childbirths was met for both PRSCs . On the other hand, the coverage of community -based health insurance did not expand, and access to basic social services by the poor continues to be inadequate . The share of the rural population with access to a potable water point was below target for PRSC 1 but was exceeded for PRSC 2, but the share of potable delivery systems which are functional did not improve . Efficacy in policy area 3 is rated substantial . 5. Efficiency (not applicable to DPLs): Not applicable. ERR )/Financial Rate of Return (FRR) a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the re- re -estimated value at evaluation : Rate Available? Point Value Coverage/Scope* Appraisal % % ICR estimate % % * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: While economic growth over the period was a respectable 4.5 percent per annum, it was far from sufficient to have any impact on poverty levels, especially given the continued high level of population growth at about 3 percent per year. The growth was fueled primarily by donor -supported public infrastructure investments and increases in food crop production, and not the result of any potential impact of the PRSCs . On the contrary, This series of PRSCs was a failure in many respects and had to be discontinued . It was overly ambitious in attempting to address too many of Mali's development challenges all at once, while, at the same time, the individual reform proposals were too timid . Some progress was made towards improving public financial management and the delivery of basic social services, but little was achieved in the all -important area of growth stimulation. The combined ratings for relevance and efficacy suggest an unsatisfactory outcome rating . a. Outcome Rating : Unsatisfactory 7. Rationale for Risk to Development Outcome Rating: The risk of reversal in the progress achieved in public financial management is small because the improvements made are by now well entrenched in government policy making . The risk of reversal in the advances made in social service delivery are also small because of expected continued strong support from the donor community as a whole. There is little risk of reversal in the area of support for growth because little was achieved, even though external developments (export prices, the global economic crisis ) may intensify the severity of the challenges. a. Risk to Development Outcome Rating : Moderate 8. Assessment of Bank Performance: In preparing the PRSC series, the Bank worked closely with the government and with the donor community on developing the policy agenda . The Bank was also well-prepared on the basis of its ESW and was simultaneously supporting several projects in relevant sectors . But the program was too ambitious, complex and diffuse, and the reform proposals themselves were too timid to be meaningful . The design included too many indicators, several of them of little relevance and many that were not monitorable . Supervision reports were unable to track developments in the multiple PDO indicators and ended up being superficial . Supervision was also compromised by the high turnover of Task Team Leaders (TTLs): there were four different TTLs over the two and a half years period between the concept review meeting for PRSC 1 and the closing of the series. at -Entry :Moderately Unsatisfactory a. Ensuring Quality -at- b. Quality of Supervision :Moderately Unsatisfactory c. Overall Bank Performance :Moderately Unsatisfactory 9. Assessment of Borrower Performance: While the government's Growth and Poverty Reduction Strategy Framework (GPRSF) stressed the need to accelerate growth (and reduce population growth ) as a prime prerequisite for poverty reduction, it did not develop a concrete agenda for achieving growth and made hardly any progress in tackling long overdue reform measures (in banking, power, cotton, irrigation ) to facilitate growth. Performance of the ministries involved in PRSC implementation was deficient because virtually no effort was made to generate the information and data needed for adequate monitoring of the reform program . a. Government Performance :Unsatisfactory b. Implementing Agency Performance :Unsatisfactory c. Overall Borrower Performance :Unsatisfactory 10. M&E Design, Implementation, & Utilization: It was unrealistic to expect that a monitoring system could be designed given the exceedingly large number of PDO indicators, the preponderance of output /process indicators rather than outcome indicators, the paucity of data and lack of baselines, and the often weak linkages between proposed policy actions and PDO indicators . The ISRs ended up choosing arbitrarily a very few of the indicators to report on . The absence of a credible monitoring and evaluation framework meant that there was no implementation, with incomplete collection of monitoring data and little government ownership . There was also no utilization of M&E : the focus was on prior actions and triggers, rather than on results and use for policy adjustments. a. M&E Quality Rating : Negligible 11. Other Issues (Safeguards, Fiduciary, Unintended Positive and Negative Impacts): The following is a summary of the ratings for the two PRSCs . Ratings ICR IEG Review Reason for Disagreements/Comments Outcome PRSC 1 Moderately Unsatisfactory Unsatisfactory See Section 6. Progress PRSC 2 Moderately Unsatisfactory Unsatisfactory made was minor, almost insignificant. Risk to Development Outcome PRSC 1 Moderate Moderate PRSC 2 Moderate Moderate Bank Performance PRSC 1 Moderately Unsatisfactory Moderately Unsatisfactory PRSC 2 Moderately Unsatisfactory Moderately Unsatisfactory Borrower Performance PRSC 1 Moderately Unsatisfactory Unsatisfactory See Section 9. The PRSC 2 Moderately Satisfactory Unsatisfactory government's reluctance to reform is long-standing ICR Quality PRSC 1 Exemplary PRSC 2 Exemplary 12. 12. Ratings : ICR IEG Review Reason for Disagreement /Comments Outcome : Moderately Unsatisfactory See Section 6. Progress made was Unsatisfactory minor, almost insignificant. Risk to Development Moderate Moderate Outcome : Bank Performance : Moderately Moderately Unsatisfactory Unsatisfactory Borrower Performance : Moderately Unsatisfactory See Section 9. The government's Satisfactory reluctance to reform is long-standing. Quality of ICR : Exemplary NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate . 13. Lessons: The ICR correctly identifies the two prime lessons from this experience . First, to have impact it is vital to be selective in reform proposals, and understand the political economy for their chance of success . Technical analyses prior to a PRSC series should be complemented by political economy analyses that can serve to identify reforms that have a reasonable chance of being implemented . Second, reform proposals should not be confined to statements of intention to reform but rather concretely focused on outcomes . 14. Assessment Recommended? Yes No 15. Comments on Quality of ICR: This is a very well written, clear and, above all, frank ICR . In it, a genuine effort is made to go beyond the superficiality of the ISRs and report candidly on the achievements and failures of this series of PRSCs . It contains all of the information needed to evaluate the program . In spite of the deficient M&E system, the ICR made a commendable attempt to separately assess and rate each operation in the series, and provided a clear presentation of what was achieved in terms of prior actions and performance indicators, including assessment of the relevance of the performance indicators . a.Quality of ICR Rating : Exemplary