Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Cambodia PFM Modernization Project (P143774) Report Number : ICRR0021014 1. Project Data Project ID Project Name P143774 Cambodia PFM Modernization Project Country Practice Area(Lead) Additional Financing Cambodia Governance P151984 L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) TF-15434 14-May-2016 12,000,000.00 Bank Approval Date Closing Date (Actual) 04-Dec-2013 14-May-2017 IBRD/IDA (USD) Grants (USD) Original Commitment 18,796,197.00 18,796,197.00 Revised Commitment 18,796,197.00 18,566,341.45 Actual 18,349,596.97 18,566,341.45 Prepared by Reviewed by ICR Review Coordinator Group Judyth L. Twigg Clay Wescott Malathi S. Jayawickrama IEGEC (Unit 1) 2. Project Objectives and Components a. Objectives According to the Grant Agreement of November 7, 2013, the project's objective was "to enhance public financial management by strengthening: (a) revenue mobilization and (b) budget execution processes through the implementation of the Financial Management Information System (FMIS)." Although Additional Financing (AF) was provided in April 2015, and some key outcome indicators were revised at a May 2016 restructuring, the scope of the project was not changed, and therefore a split rating will not be performed. Page 1 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Cambodia PFM Modernization Project (P143774) b. Were the project objectives/key associated outcome targets revised during implementation? No PHEVALUNDERTAKENLBL c. Will a split evaluation be undertaken? No d. Components The project contained two components: 1. Mobilization of Revenue and Steering Committee Secretariat (SCS) Operations Support (appraisal, US$ 0.5 million; actual, US$ 3.07 million). This component was to support the finalization of a revenue mobilization strategy and monitor its implementation. Support was to be provided to the Department of Economic and Public Finance Policy (Policy Department) of the Ministry of Economy and Finance (MEF) for review and refinement of the strategy, development of sequenced and prioritized implementation action plans, and monitoring of those plans. Support was also to be provided to the MEF's SCS for its task of managing, implementing, and coordinating the country's overall Public Financial Management Reform Program (PFMRP). 2. Improving Budget Execution Processes through the Implementation of the FMIS (appraisal, US$ 11.5 million; actual, US$ 15.28 million). This component was to provide funds for the MEF to acquire an FMIS that would facilitate the financial operations of budget and treasury units. The FMIS was to be built around a core treasury system supporting all budget execution functions, from commitment to payment stages. Support was also to be provided to the MEF, line ministries, and all budget entities to implement specific segments of a newly developed account code structure and to fulfill the requirements of cash basis International Public Sector Accounting Standards (IPSAS). Finally, this component was to provide implementation support for the FMIS, including capacity development for the General Department of the National Treasury (GDNT) and FMIS Project Management Working Group, as well as change management activities to support the transition of organizations and staff within the newly automated environment. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project Cost: Total project costs were estimated at US$ 12 million. Actual costs were US$ 18.35 million. Financing: The project was financed by a grant from the Public Financial Management Trust Fund, supported through contributions from the European Union, Australian Agency for International Development, and Swedish International Development Agency for the period 2013-2016. An unresolved inspection panel case (unrelated to this project) had resulted in a suspension of Bank operations in Cambodia from 2011-2016, leaving the Trust Fund as the only financier of the project across its lifetime. The initial grant agreement was prepared in the amount of US$ 5.9 million to reflect the amount of funding Page 2 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Cambodia PFM Modernization Project (P143774) received by the Bank to date, with planned amendments bringing the amount to US$ 12 million once the remaining committed funds were received. An AF in April 2015 in the amount of US$ 7.5 million brought the total planned financing to US$ 19.5 million. According to the ICR (p. 8), the AF was intended to support additional capacity building, roll out the FMIS to additional ministries, improve budget strategic planning, and fill a financing gap in FMIS implementation support. This gap resulted from an additional planned contribution from the European Union that did not materialize. Exchange rate fluctuations brought the total actual financing to US$ 18.35 million. Borrower Contribution: No Recipient contribution was expected or made. Dates: The project was approved on November 6, 2013 and became effective on December 4, 2013. AF was approved on April 17, 2015. A May 4, 2016 restructuring amended the results framework and extended the project's closing date from May 14, 2016 to May 14, 2017 to allow time for completion of key activities. 3. Relevance of Objectives Rationale Despite earlier progress in public financial management reform, Cambodia was not reaching efficiency and transparency levels at appraisal that were in line with similarly transforming countries in the region. A 2011 Integrated Fiduciary Assessment and Public Expenditure Review identified the need to increase the scope of reforms to improve revenue collection and expenditure policy, leading to the conception and launch of the PFMRP in 2013. The project's objectives are derived directly from the PFMRP, whose key goals are improved financial accountability and effective management of public finance. The objectives are also highly relevant to the Government's National Strategic Development Plan at closing (2014-2018), which identified improved governance as critical to enhancing service delivery. The objectives are also highly relevant to the Bank's Country Engagement Note (2016-2017), which identifies more robust public financial management and specifically expanded system functionality of the FMIS as keys to improving governance for effective delivery of public services (Country Engagement Note, p. 12). Rating High 4. Achievement of Objectives (Efficacy) PHEFFICACYTBL Objective 1 Objective Enhance public financial management by strengthening revenue mobilization Page 3 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Cambodia PFM Modernization Project (P143774) Rationale The theory of change underpinning the revenue mobilization objective was sound. It reasoned that a well- designed revenue mobilization strategy, implemented and monitored rigorously, in conjunction with improved taxpayer services, would generate more resources for the government to finance infrastructure development and improve overall public services. Non-tax revenue sources would be diversified as well. Achievement of this objective was originally to be measured through one key outcome indicator: the revenue-to-gross domestic product ratio. The results framework was later modified to replace this indicator with measures more specifically designed to track progress against the objective: an increase in non-tax revenue sources, and increases in revenue from property and customs taxes. The revenue mobilization strategy (2014-2018) developed under the project organized and facilitated the work of revenue collection agencies, primarily through two Steering Committees established to evaluate tax and non-tax revenues. The Secretariat of these committees produced mid-year and annual monitoring reports. Staff received extensive domestic and international training on taxes and customs. The General Department of Taxation improved and modernized its services to facilitate tax payments at commercial banks or tax branches. The General Department of State Property and Non-Tax Revenues worked with line ministries to develop inter-ministerial agreements to ensure that targeted revenues were collected and reported transparently. As a result, non-tax revenues increased by 181% (from US$ 173 million in 2013 to US$ 314 million in 2016) over the project's lifetime. The number of sources of non-tax revenues increased from 24 to 38, exceeding the target of 37. Revenues from property taxes (from 62.7 billion Cambodian Riel (KHR) in 2013 to KHR 86.7 billion in 2016) and customs taxes (from KHR 3.9 billion in 2013 to KHR 6.9 billion in 2016) increased substantially (exceeding targets, but it should be noted that targets for this indicator appear to have been set in 2016, the same year they were achieved). Although part of this outcome is certainly attributable to robust economic growth (average of 7%) during the project implementation period, it is plausible that the project's implementation and monitoring of the revenue mobilization strategy, as well as its activities to improve taxpayer services and compliance management, helped to leverage economic growth for enhanced revenue outcomes. Rating Substantial PHREVDELTBL PHEFFICACYTBL Objective 2 Objective Enhance public financial management by strengthening budget execution processes through the implementation of the FMIS Rationale The theory of change underpinning the budget execution objective was also sound. Budget execution and compliance were to be improved through the implementation of the FMIS, enabling a comprehensive budget Page 4 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Cambodia PFM Modernization Project (P143774) classification and chart of accounts, effective ex-ante budget control, and timely financial reports. Specifically, project activities -- the acquisition and implementation of FMIS at the GDNT and all 25 provincial treasuries (including budget control and execution modules), implementation of a comprehensive chart of accounts, and related capacity building and change management activities -- were to lead to improved budget classification, the preparation of financial statements with disclosures of standards, and ultimately improved timeliness of locating financial reports on budget execution, payment processing, and closure of financial accounts. Achievement of this objective was originally to be measured through one key outcome indicator: the number of treasuries at the national and provincial levels implementing the FMIS. The results framework was later modified to replace this indicator with measures more specifically designed to track progress against the objective: improved timeliness in locating and reporting financial data; improved timeliness in payment process at the GDNT; and more timely closure of financial accounts at the end of the fiscal year. The FMIS has been fully implemented at MEF and all 25 provincial treasuries. It was supported through five distinct phases -- inception, elaboration, construction, transition, and production -- governed by business process and project management plans and with full transparency on a central FMIS website. Six budget control and execution modules -- Budget Allocation, General Ledger, Accounts Receivable, Accounts Payable, Cash Management, and Purchase Order -- have been in use since January 2016. More than 2000 MEF staff received domestic or international training on procurement, budget execution, budget formulation, FMIS, procurement policies, leadership, and management. (The PAD, p. 26, includes an indicator with the target of increasing the number of female staff participating in training events from a baseline of 5% to a target of 20% that was not mentioned in the ICR.) Seven budget classification structures were uniformly configured in the FMIS, revised, updated, and issued as inter-ministerial agreements at all national and provincial administrations; however, some of the codes (Functional Classification and Geographic Classification) are not yet implemented, although their agreements have been finalized and issued. IPSAS cash basis reports (Budget Income, Budget Expenditure, and Cash Flow Statement) were produced beginning in 2016. The FMIS is interfaced with the country's core banking system. As a result, the time required to locate financial data and produce reports has been reduced from up to two days (in 2013) to a few minutes (no longer than 10 minutes for more complex reports), far exceeding the target of 30 minutes. Financial reports can be extracted from the FMIS from either central offices of the MEF or any of the provincial treasuries. Payment processes have been shortened from 15-20 days (in 2013) to 10-15 working days, as targeted. Salary payments are made via bank accounts in a timely manner (fewer than five working days), on the last day of each month. The time required to close financial accounts decreased from 9-10 months to six months in FY 2016, not reaching the target of four months, with delays due to initial challenges with the introduction of program budgeting reforms and some continued usage of the legacy system. Overall, a comprehensive, country-wide transaction processing system now governs commitments, payments, accounting, and reporting, forming a basis for transparency and accountability of budget management. Rating Substantial PHREVDELTBL Page 5 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Cambodia PFM Modernization Project (P143774) PHOVRLEFFRATTBL Rationale The project's theory of change related to both objectives was sound, with logical and plausible links between planned activities and intended outcomes. Activities were largely implemented as envisaged, and resulting outcomes met or exceeded targets. The project's interventions have taken place in an environment of sustained economic growth, enabling the country to leverage that growth for enhanced revenue outcomes and transparent and accountable budget management. Based on substantial achievement of both objectives attributable in part to project-financed interventions, overall efficacy is rated Substantial. Overall Efficacy Rating Substantial PHREVISEDTBL 5. Efficiency The PAD (p. 20) states only that the project would "improve efficiency, enhance revenue collection, and generate timely and comprehensive financial records and reports," primarily through streamlined business processes, operational efficiencies, improved transparency and oversight, and improved budget integration and reporting made possible by the FMIS. No formal economic analysis was performed at appraisal. The ICR (p. 35) restates the argument that a reliable economic or financial analysis using tools like Net Present Value or Economic Rate of Return is "difficult for such public financial management reform projects," and like the PAD, it treats the benefits qualitatively ("improved budget allocation for key social sectors that are critical to the lives of all Cambodian people"). Administrative efficiency was Substantial. All planned activities were completed within budget, and there were no cost overruns; the one-year extension was implemented at no cost to support an additional transition period for FMIS users. The FMIS contract was completed within the initially estimated US$ 10 million cost, a significant improvement over bids in the US$ 30 million range that had been received and abandoned under a previous project. FMIS implementation was estimated to be US$ 27,000 per user during the project period. This is projected to decrease to approximately US$ 15,000 per user moving forward (taking the ICR's estimates of US$ 11 million for the initial implementation contract, plus a marginal cost increase of US$ 0.3 - 0.4 million to scale up from the current user base of 400 to 700-800 users), in line with comparator World Bank-financed FMIS contracts that have also cost an average of US$ 15,000/user. However, project documents lack clarity about the financing of training activities described in the ICR, leaving some uncertainty around the value achieved for the AF. The PAD does not contain a breakdown of a planned training budget that can be compared to what was actually spent, and it is not clear whether the training activities described in the ICR (pp. 40-43) were financed throughout the project period or only by the AF. In addition, the ICR (p. 8) indicates that the AF would finance the preparation of FMIS rollout to other ministries beyond the MEF, but it is not clear that this took place. Page 6 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Cambodia PFM Modernization Project (P143774) Efficiency Rating Substantial a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal 0 Not Applicable 0 ICR Estimate 0 Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome The project's objectives were highly relevant to country conditions, government strategy, and Bank strategy. The objective to enhance public financial management by strengthening revenue mobilization and budget execution was substantially achieved through effective implementation of an FMIS and revenue mobilization strategy. There was substantial efficiency in the use of project resources. Taken together, these ratings are indicative of only minor shortcomings in the project's preparation and implementation, leading to an Outcome rating of Satisfactory. a. Outcome Rating Satisfactory 7. Risk to Development Outcome The government is committed to building on the institutional and human resource development achieved under the project. It is preparing a next-phase revenue mobilization strategy (2019-2023) that will focus on tax policies and additional sources of revenue. Increased collaboration between key public financial management departments -- notably the MEF and GDNT -- has become a habit, with enhanced trust and willingness to share information (ICR, p. 14). Importantly, the FMIS system has become central to producing financial reports and tracking budget, and the government appears to be committed to sustaining and extending support for the system to all line ministries and several sub-national-level entities, with additional modules added for budget preparation, procurement, and asset management. Post-project closure, it will be important for the government to sustain technical support for the FMIS (which is still dependent on consultants for more advanced functionalities), and even more critically, to provide a platform for cross-institution collaborative problem-solving. The ICR does not state whether the Bank or other donors are considering follow-on operations. Page 7 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Cambodia PFM Modernization Project (P143774) 8. Assessment of Bank Performance a. Quality-at-Entry The identification team focused on finding opportunities for intervention that would help reach transparency and efficiency levels of similar transforming countries in the region (ICR, p. 18). The project built on key lessons learned from an earlier FMIS procurement in the country: the importance of a phased approach for systems procurement and implementation; the need to engage key functional departments in system design, scoping, and acquisition; and the need for significant technical engagement and support from the Bank (PAD, pp. 15-16). A carefully planned and sequenced FMIS acquisition and implementation plan was developed, with the completion of key prerequisites (finalization and approval of a uniform account code structure, adoption of a Chart of Account based on IPSAS) verified at appraisal. Risk identification centered around the procurement and implementation of the FMIS, with mitigation measures including a dedicated sub-component for project implementation support (PAD, Annex 4); according to the ICR (p. 15), the only minor shortcoming was inadequate focus on institutional arrangements for sustainable technical support for the FMIS. An implementation support plan was put in place to ensure donor coordination throughout the sequenced reform process. Quality-at-Entry Rating Satisfactory b. Quality of supervision The Bank supervision team was led by a single Task Team Leader during most of the project period, and its wide and appropriate range of specialists engaged closely with counterparts. Extensive sector expertise, including technical guidance through FMIS procurement and implementation, was provided. Aide Memoire were clear, detailed, and candid. The Bank team identified and addressed early challenges during implementation, including lack of cooperation among skeptical stakeholders (who feared job loss and did not understand the FMIS system). Change Management teams facilitated coordination through study tours and events intended to improve interpersonal relations among staff, and overall enhanced implementation support to resolve recruitment and capacity development issues. Appropriate and timely adjustments were made, including project restructuring, AF, and extension. Overall, the Bank team repeatedly brought together stakeholders from multiple sectors to find solutions to common challenges, taking full advantage of its convening power. However, there is lack of clarity around some indicator targets and their reporting in ISRs (see below, Section 14). Quality of Supervision Rating Satisfactory Overall Bank Performance Rating Satisfactory Page 8 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Cambodia PFM Modernization Project (P143774) 9. M&E Design, Implementation, & Utilization a. M&E Design The PAD contained a comprehensive results framework, including indicators adequate to measure achievement of the project's objectives, specification of data sources, and designation of institutional responsibility for data collection and reporting. The objectives, indicators, targets, and overall results framework were responsive to the government's approach, donor inputs, and the overall public financial management reform agenda. The initial set of key outcome indicators, however, was overly broad and vague, and the linkages between those outcome indicators and intermediate indicators were not solidly articulated. b. M&E Implementation Although there were some occasional delays in the delivery of progress reports by the General Secretariat of the Public Financial Management Reform Steering Committee (GSC), overall M&E reports were timely and reliable in tracking project status. The two project restructurings adjusted the key outcome indicators and the overall results framework to be more internally consistent and directly responsive to the development objectives. Ten implementation support missions monitored progress, collected detailed data, updated progress against baselines, and highlighted issues requiring management attention. Regular external reviews of the project (External Advisory Panel reports, Public Expenditure and Financial Accountability reports, Public Expenditure Tracking Surveys, and other reports prepared by trust fund donors) as well as overall PFMRP progress reviews were used to monitor output and outcome indicators. c. M&E Utilization Ongoing monitoring and analysis were used to inform decision making during implementation, including the decisions to restructure and extend the closing date, as well as allocation of increased resources for training and capacity building and for FMIS implementation. M&E Quality Rating Substantial 10. Other Issues a. Safeguards The project was rated Environmental Assessment Category C. No safeguard policies were triggered. Page 9 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Cambodia PFM Modernization Project (P143774) b. Fiduciary Compliance According to the ICR (p. 18), the project complied with all fiduciary covenants during implementation. Adequate financial management, procurement, and disbursement systems were maintained. Internal control guidelines using country finance systems were followed. Audits were timely and unqualified, including the final project audit carried out by an internationally affiliated firm. The GSC experienced some minor delays due to communications challenges with other concerned departments, but these were overcome early in the project period. All procurement contracts were concluded in a timely manner and in compliance with the Bank's guidelines. c. Unintended impacts (Positive or Negative) None reported. d. Other --- 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Outcome Satisfactory Satisfactory --- Bank Performance Satisfactory Satisfactory --- Quality of M&E Substantial Substantial --- Quality of ICR Substantial --- 12. Lessons The ICR (pp. 20-21) offers several valuable and insightful lessons and recommendations, including: Sequencing matters, especially when attempting complex reform in a low-capacity context. In this case, the predecessor project's (Public Financial Management and Accountability Project, 2006-2013) outcome was rated Unsatisfactory by an IEG Project Performance Assessment Report (PPAR), which observed that the "FMIS contract award collapsed after five years of failed procurement" (p. ix). PFMRP built on this experience, taking a phased approach. The FMIS design started initially with payments and receipts transactions at treasury offices, before rolling out other modules and going into other organizational units. These and other design changes allowed for a successful procurement of an FMIS. Page 10 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Cambodia PFM Modernization Project (P143774) Deliberate and careful change management activities, grounded in high-quality political economy analysis, are central to the success of potentially controversial reforms. In this case, the reform process was put at risk by middle-level managers whose divergent perspectives and competing interests could have significantly impeded implementation. A combination of committed MEF leadership and Bank change management interventions created an environment that respected and engaged all key stakeholders, acknowledging their concerns and providing them with pathways to identify and inhabit their roles in the new system. IEG offers two additional lessons: Clear and well-articulated objectives facilitate the conceptualization and implementation of a coherent, plausible theory of change. In this case, although the initial key outcome measures were overly broad, the clarity of the objectives facilitated the project team's flexible adjustment of the results framework's details. As a result, achievements were trackable, demonstrable, and clearly linked to intended outcomes. Information technology procurements can achieve efficiency by focusing in the initial stages on high-value expenditures. In this case, a relatively small proportion of transactions, but the bulk of expenditures, were covered by the FMIS (see ICR, Annex 7). 13. Assessment Recommended? Yes Please explain This project appears to represent a successful, efficient information technology system procurement that could serve as a model for other projects. It also represents best practices in change management and capacity development that may hold valuable lessons. Although the predecessor project was the subject of a very recent PPAR (2016), a follow-up study offers the opportunity to draw important insights into learning from experience. 14. Comments on Quality of ICR The ICR presents the project's theory of change clearly and concisely, mapping the development objectives explicitly to relevant activities, outputs, and outcome indicators and specifying key underlying assumptions. It is a model ICR under the new guidelines. Its lessons are insightful and well derived from project experience. However, one minor shortcoming is, an intermediate results indicator in the PAD calls for increasing the number of female staff participating in training events. The ICR does not include this indicator in the results matrix, or any quantitative results, mentioning only that the project "supported women by employing and providing them with necessary training at the Budget Departments" (p. 13). Page 11 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Cambodia PFM Modernization Project (P143774) a. Quality of ICR Rating Substantial Page 12 of 12