76064 The World Bank PREMnotes November 2012 NUMBER 22 Special Series on Using M&E to Support Performance Based Planning and Budgeting in Indonesia Mark Ahern, Victoria A. Beard, Anna I. Gueorguieva, and Retno Sri Handini Since 2000, there has been growing interest in reforming Indonesia’s budgeting systems to promote a more performance–orientated process. Indonesia is in the initial stages of this reform. A major challenge is determining the information needs of the central coordinating ministries. To date, these ministries have taken separate paths, developing their own monitoring and evaluation (M&E) systems, which are not linked to the planning and budgeting system, and creating new regulations and institutions to manage them. The result has been underused information and a high reporting burden at all levels. Furthermore, the current system places a greater emphasis on monitoring rather than evaluation. In 2011, representatives from the coordinating ministries participated in a series of high-level round table discussions to identify the steps needed to rationalize and coordinate M&E practices across institutions and to strengthen the links among data collection, evaluation, planning and budgeting. The round table process has confirmed that, while coordination is needed, establishing incentives for the demand and use of M&E information is critical to making the systems effective. This note identifies priority areas for future action building on this finding. Context A number of factors make Indonesia’s devel- opment of a coordinated M&E system particularly In Indonesia, policy makers and planners are in challenging, including its size, geography, and ad- the early stages of identifying the appropriate ministrative organization. Indonesia is the fourth information at the national level so that they can most populous country in the world, with a total better monitor and evaluate the performance population of 238 million in 2010, and roughly of the public sector. Such information is key to 700 unique languages in addition to a national moving the country from a system of planning language (Badan Pusat Statistik 2012). 1 This and budgeting based historically on budgetary geographical complexity is significant because inputs to a new system based on program- government units across the country have varying matic outcomes and performance. The task of capabilities to collect, manage, and transmit data. developing a well-functioning M&E system is The country is organized into territorially based daunting for any country; international experi- political-administrative units (33 provinces and ence suggests the process can take upward of 497 cities/districts), which are responsible for 10 years (Arizti and others 2010). This note about 50 percent of the core public spending summarizes Indonesia’s experience in this tran- in 2012.2 In 2011, at the central government sition by providing a overview of the evolution level, 86 boards/ministries/agencies/commissions of M&E, including the political economy of the were responsible for designing and overseeing M&E activities currently being undertaken, and the implementation of more than 420 national concludes with a discussion of priorities for development programs by 29,000 spending units future action. (Ministry of Finance 2012). FROM THE POVERTY REDUCTION AND ECONOMIC MANAGEMENT NETWORK The historical and political context is impor- for nonconformance�) or pemeriksan (“examina- tant because of its continued influence on the in- tion of the books�), the government of Indonesia’s stitutional culture that shapes M&E in Indonesia (GOI’s) administrative data collection systems today. During Suharto’s presidency (1967–98), and practices are largely geared toward tracking governance was based on a hierarchical structure compliance with rules and regulations rather than in which data were collected from the smallest achievement of results (Huage 2004). territorial-based units (in some cases clusters of 10 households), transmitted upward to increasingly M&E in the Reformasi Era larger units, and ultimately reported to the manag- ing ministry or office in the central government. The resignation of President Suharto in 1998 It should be noted that similar governance struc- ushered in a period of dramatic political and tures exist throughout East and Southeast Asia. administrative reform in Indonesia, known For a number of programs, this system worked nationally as reformasi (Aspinall and Mietzner effectively at relatively low cost. For example, 2010; Crouch 2010). This period brought sig- trained local volunteers (kader) collected informa- nificant changes in the judiciary, legislature, and tion on contraceptive use, basic indicators of early executive offices as well as a system of competitive childhood health, and household poverty. The elections. Another key reform that supports the system, however, did have limitations, including consumption of M&E information is a political allowing for only a one-way transmission of infor- environment that is much more tolerant of free mation from the bottom to the top that limited speech and an open news media. Civil society was opportunities for those who implemented the severely restricted under President Suharto, but programs to communicate performance nuances in the period since his resignation there has been and inhibited horizontal learning. Building on the significant growth in the number and diversity oversight perspectives of pengawasan (“watching of nongovernmental organizations (NGOs). A Figure 1. Timeline of Changes Affecting M&E in Indonesia Government President Ministry of Regulation established a Planning 20/2004 special unit (BAPPENAS) stipulates that (UPK4) to starts to line ministries provide short- implement should submit term program a national quarterly and output effort 2000 annual budget information to monitor performance (INPRES programs 2003 reports 2005 2007 1/2010) 2010–11 (PP 39/2006) No systematic M&E effort Law 17/2003 Deputy of 2009 Performance- 2004 Establishment 2006 on public of Minimum Performance Based �nance Service Evaluation Budgeting stipulates that Standards (BAPPENAS) Legislation budget should for subnational established (PP 90/2010) based on governments performance by the Series of Ministry of ministerial- Home Affairs level capacity- building workshops and policy coordination Source: Authors’ illustration. roundtables 2 PREMNOTE NOVEMBER 2012 major legislative reform during this same period alizing M&E because of the need to assess program was what some have referred to as “big-bang� performance. decentralization that transferred core public Almost in parallel, in 2004 the Law on Na- service functions and devolved political authority tional Development Planning was passed. The to cities and districts. law, among others, mandated that the National The changes in the post-Suharto period have Development Planning Agency (BAPPENAS), a created both opportunities and obstacles in terms coordinating ministry responsible for national of M&E. First, democratization coupled with in- planning, implement an annual work plan (RKP, creased freedom of speech and freer news media Rencana Kerja Pemerintah) to bridge the gap have created new expectations for transparency between the five year plan and the annual budget and accountability in the public sector. A new cycle. The law further mandated that the indi- national dialogue about corruption has emerged, vidual ministries create their own work plans, and recent emphasis has been put on using pro- including basic output indicators; however, ini- gram and ministerial performance as the basis tially these work plans were not linked to budget for allocating resources. The production of Mini- allocations. Eventually, BAPPENAS and Ministry mum Service Standards at the subnational level of Finance (MOF) issued decrees (PP 20 and 21) in for key sectors, such as in health and education, an effort to link the annual work plans to resource also emerged in 2005 (World Bank 2008, 2012). allocation in 2004. These changes support moving the M&E agenda In 2005, the Ministry of Home Affairs forward. However, simultaneously, political and established the Minimum Service Standards administrative decentralization has raised serious (MSS) for most of the key service sectors. This questions about the authority of the central gov- was a significant initiative that carried with it an ernment to mandate reporting from lower levels implicit commitment by the GoI to monitor and of government. It is against this complex backdrop evaluate the MSS indicators over time. However, that analysts, policy makers, and planners need to the use of MSS information by decision makers is understand the evolution and consider the future limited and there is not yet a link to the resource of M&E in Indonesia. allocation system. BAPPENAS first mandated the collection The Evolution of M&E of budget-related M&E data in 2006, when it During the post-2000 period of reform and de- issued a decree (PP 39) that required each na- centralization, the development of M&E systems tional development program to report quarterly. was driven largely by regulations and decrees Almost from its inception, implementation of issued by different central coordinating agencies. this regulation ran into difficulties. One issue New regulations and decrees were created at a was that the data collection was not linked to a rapid pace, thus sometimes creating problems in particular budget or planning process, meaning terms of harmonization with regulations issued that there was not a strong incentive for program by other departments within the same agency managers or budget and planning officials to use or other agencies. Because the new systems were the M&E information provided or give feedback often not linked to the decision-making processes on how it might be improved. Another issue was of the government, they also lacked credibility. that the regulation requires that the reporting be An important development occurred in conducted quarterly at the level of the 29,000 2003 when Law 17/2003 on state finance was spending units, which overburdened all parties promulgated. Among others, the law mandated: involved and resulted in much of the information (i) implementing a budget classification system reported being copied from other reports. The that included performance measures; (ii) unifying “one-size-fits-all� template was also difficult to ap- the previously separate development and routine ply to specific programs, and was often not fully budgets and explicitly differentiating between consistent with existing performance systems operational and capital expenditures to overcome such as the MSS in key sectors, so interpretation the problem of duplication and overlap; and (iii) and reporting varied. In sum, an external evalu- creating a forward-looking budget based on the ation characterized the regulation as “difficult to use of medium-term economic frameworks. These implement, is unlikely to generate the types of recommendations provided a basis for institution- information required for strategic planning and NOVEMBER 2012 PREMNOTE 3 evaluation and has variable levels of compliance� faces a number of challenges, including the need (Funnel and Djayusman 2008). to resolve how the information will be used in In an effort to further strengthen insti- the annual budget cycle to inform allocation tutional support for M&E, the Deputy of decisions; how to rationalize the information Development Performance Evaluation (DPE) demands made on line ministries; and to clarify was established in BAPPENAS in 2007. How- the role of the MOF with respect to that of other ever, DPE has faced challenges in navigating the institutions monitoring the budgeting and plan- preexisting M&E arrangements and defining ning process. its role among other coordinating bodies. The On the evaluation side, the capacity to external evaluation urged DPE to develop a design, implement, and analyze impact evalua- framework to improve the availability, quality, tions should continue to be expanded because and use of relevant M&E information. Their there is no systematic program coordinated at study recommended that DPE establish its the central level. This is an area where the GOI profile and credibility as a change agent across could build on the support by external develop- the government. The evaluation identified the ment partners. For example, the World Bank need to build capacity and share information, has supported several major impact evaluations particularly with the MOF. It also identified of key development programs since 2005 and the potential risks involved in bringing forward has an active program of evaluations and an the M&E agenda if the process is not managed analytic agenda linked to its engagements.3 As a properly. Such risks include low commitment result of good collaboration, associated recom- and understanding of what is expected, token mendations are often incorporated (though not compliance, distortion of information in the ab- without critical review) into both policy coordi- sence of quality assurance, and the misdirected nators’ and implementers’ planning and budget- use of incentives and sanctions resulting in or ing. This is clearly visible in, for example, the link reinforcing poor practices. between results from the impact evaluation of M&E during this first phase lacked the incen- Indonesia’s largest national community-based tives and institutional arrangements to ensure the poverty alleviation program (PNPM) and the use of the monitoring data for program assess- policy and planning stances that their respec- ments. Although approximately 40 percent of tive implementing and coordinating agencies line ministries regularly report on the indicators have taken. However, such impact evaluations, under PP 39, these data are not analyzed, nor while useful, are expensive and technically dif- are they used for decision making in planning or ficult and therefore require careful planning and budgeting. The lack of analysis can be explained should be used selectively. by the huge burden of the monitoring effort. The performance indicators, based on a logical frame- work methodology, total over 6,000, with a lim- ited focus on outcomes and overall low relevance Table 1. Comparison of M&E Indicators by for evaluation (World Bank [2011b] provides a Country discussion of social assistance indicators). Table Number of M&E 1 compares the large number of M&E indicators Country (year) Indicators in Indonesia to the number of indicators collected in other countries. Canada (2003) 400 Part of the recent impetus to improve France (2005) 1,178 M&E systems in Indonesia has come with the Indonesia (2010) 6,440 strengthening of performance-based budgeting (PBB; Castro 2007). To support this reform, Korea, Rep. of (2007) 2,037 the MOF established an internal M&E unit to Netherlands (2002) 454 enhance the systems for budget monitoring. The unit has developed a new regulation that United Kingdom (1998) 153 collects information on budget implementation Thailand (2010) 3,000 and assesses the outcome of program interven- Source: World Bank’s Public Sector Performance Global tions. The system is in its infancy, however, and Expert Team (2011a), and authors’ compilation. 4 PREMNOTE NOVEMBER 2012 The M&E Terrain and Presidential Working Unit for Development Its Challenges Monitoring and Control (UKP4) and National Team for Poverty Reduction (TNP2K) have made One of the most significant challenges in reforming an extensive investment in technical capacity. It the M&E systems in Indonesia is to create a strong is unclear, however, what the long-term outlook incentive for M&E to be conducted, and for the is for these offices, since they are a product of the findings to be used to inform the budget as well as current political administration and could be program managers’ decisions. M&E information vulnerable under a new administration. As a re- has value only if it is reliable and if it is used—and sult, transmitting and institutionalizing the M&E these attributes are reinforcing. Without strong capacity of these offices should be a high priority. incentives, it is challenging to sustain the intensive effort needed to build and operate an M&E system. While in many other countries an incentive has Monitoring Burden at been provided by a fiscal crisis, this has not been the Ministerial Level the case in Indonesia over the past decade. High The over-reporting burden is particularly evident commodity prices and growing domestic con- when we look at the reporting requirements of sumption coupled with prudent macroeconomic some of the largest ministries (for example, health, management have allowed Indonesia to weather education, agriculture, and public works). These the global economic crisis without a significant ministries are legally required to provide at least fiscal deterioration. At a microlevel, the M&E six regular monitoring reports to coordinating systems have often been an end in themselves ministries or government offices (SMERU 2011, rather than a tool to support a critical need of the 6). In addition to fulfilling these requirements, government. Nonetheless, there appears to be a the individual ministries have internal monitoring strong demand emerging from policy makers and systems to collect information related to budget other stakeholders to improve the quality of public spending and program implementation. Each services in areas such as health, education, social ministry has unique reporting formats based on protection, and infrastructure. a specific program’s internal monitoring system. A further complication is the large number The cost of the reporting burden is clear: the more of actors involved in program planning, imple- resources spent on routine monitoring, the fewer mentation, and M&E. The coordination of the resources there are available for analyzing data and planning and budgeting process is split between for managing the change they might imply. two central finance ministries: BAPPENAS and the MOF, while other central agencies have active Using Existing Data Sets for roles. These actors have developed their own M&E processes to fulfill their need for performance Evaluating Program Outcomes information. The processes have overlapping Indonesia is unusual in having a number of robust regulations for collecting M&E information from data sets that can be used to evaluate development ministries and smaller offices. These regulations outcomes. While some coordinating ministries have raised concerns about the quality of informa- and government offices already make use of the tion reported, the reporting burden, redundant data, in order for the data to become widely used information requests, information gaps, access across the government, a transparent system of to information, and overall M&E policy coordi- data-sharing protocols needs to be established as nation. Moreover, where the processes operate at well as the technical capacity developed in the different points of a single planning and budget- ministries.4 ing cycle, they can work in different directions. The large number of data sets has the po- As of yet, there is insufficient communication, tential to improve the quality of M&E informa- coordination, and data sharing among the dif- tion. For instance, one of the most well-known ferent coordinating ministries, agencies, and data sets is the National Socioeconomic Survey offices. Table 2 describes these M&E practices at (SUSENAS) collected by the National Bureau of the national level. Statistics, which provides socioeconomic indica- There are concerns about the sustainabil- tors representative at the district level. Beginning ity of some of these practices. For example, the in 1963, and fielded every year or every other year NOVEMBER 2012 PREMNOTE 5 Table 2. M&E Practices of Coordinating Ministries, Agencies, and Offices Ministry/agency/office Description of M&E practices Quarterly reporting of national programs aggregated through individual National Development ministries (PP 39). In addition, the current five-year plan includes a matrix Planning Agency (BAP- where each program and its activities have corresponding output and PENAS) outcome indicators. It is unclear how this information will be collected, analyzed, and used to support national planning and budgeting. UKP4 gathers basic information about short-term progress related to the Presidential Working Unit president’s highest national development priorities. The focus is on tangible for Development Monitor- program milestones, with an emphasis on easily quantifiable indicators. ing and Control (UKP4) Strengths of this system are the simplicity of the measures and the ease of reporting. Collects information annually regarding program rationale, budget imple- Ministry of Finance (MOF) mentation, and outcomes from the line ministries. It is unclear how this information will be used in the annual budget cycle. The focus is on performance and accountability of government institutions in terms of planning, performance measurement, performance reporting, Ministry of State Appara- performance evaluation, and performance achievement. Each aspect is tus and Reform (MEN- weighted and the total score is used to rank institutions. The ministry uses PAN) a combination of interviews, direct observation, and secondary data. The results are reported annually to the president. The ministry conducts three types of assessments that focus on the subna- tional level: (i) performance of local policy makers and executors, (ii) local Ministry of Home Affairs capacity to achieve the goals of decentralization, and (iii) performance of (MOHA) geographic areas that recently received autonomy. These assessments are collaborations between a national team and local technical teams. The results are reported at different intervals to the president. This office is a policy think tank headed by the vice president. The agency monitors and assesses the implementation of the country’s major poverty National Team for Poverty alleviation programs. It also aims to ensure that sufficient M&E capacity Reduction (TNP2K) exists within relevant agencies. The office’s strengths include that M&E findings directly inform policy, minimal bureaucratic obstacles, and strong leadership by the vice president. Source: Authors’ compilation. since, SUSENAS uses a nationally representative the local economy. A National Labor Force Survey sample composed of 200,000 households (RAND (SAKERNAS) collects national labor market 2012). Each survey contains a core questionnaire characteristics based on working-age individuals with a household roster yielding the sex, age, within sampled households. The Indonesian Fam- marital status, and educational attainment of all ily Life Survey (IFLS) is an ongoing panel survey household members. This core is supplemented whose data are available in the public domain. The with modules covering about 60,000 households IFLS collects data at the individual, household, that are rotated over time and collect additional and community level on a multitude of topics, information on topics such as health care and nu- including household structure, health, education, trition, household income and expenditure, and employment, and the local environment. Table 3 labor force experience (RAND 2012). A village provides a quick comparison of these data sets. or subdistrict level survey, the Village Potential To date, the full potential of these data sets Statistics (PODES) survey, gathers information on has not been realized for either M&E or conse- local infrastructure and services, population, and quently for performance-based planning and 6 PREMNOTE NOVEMBER 2012 Table 3. Examples of Indonesian Data Sets Data set Substantive focus Sample size and coverage Frequency National Multipurpose household survey 200,000 households Implemented ev- Socioeconomic (basic household characteristics with a rotating module at ery year or two Survey and rotating module) 60,000 households since 1963 (SUSENAS) Village Potential Village/subdistrict characteristics All Indonesian vil- Periodic Statistics lages/subdistricts, with a (PODES) sample of +/- 65,000 National Labor National labor market character- Approximately 50,000 Yearly Force Survey istics of working-age individuals households, depending (SAKERNAS) on the year Indonesian Multipurpose household and Approximately 30,000 Approximately Family Life community survey (for example, individuals represent- every three to Survey (IFLS) demographic, health, and educa- ing about 83% of the five years tion) population Source: Authors’ compilation. budgeting. The timing and availability of these overlapping systems and the need for coordi- data do not make them an appropriate substitute nation, it subsequently lost momentum. One for routine program monitoring; however, these reason is that the different agencies involved did data could potentially provide information about not see how the effort to create a comprehensive longer-term program outcomes relevant to plan- system was addressing their immediate needs. ning. Future cooperation among the National As a result, they have continued to develop new Bureau of Statistics and the ministries as well M&E regulations and activities independently. as local think tanks, universities, and NGOs to The World Bank has more recently focused on analyze program outcomes should be further ex- supporting better integration of the efforts of plored and supported. However, for this to occur, the different agencies—specifically the work a number of limitations, including data access and of BAPPENAS and MOF linked to the annual adequate statistical and analytical capacity in the budget cycle. The demand from these agencies for ministries, will need to be resolved. better performance information is an important incentive to make the system effective; however, Recent Efforts the lesson of the past is that the initiatives will A series of roundtable meetings began in early only be successful if the efforts of the two agen- 2011 in an effort to chart a future course for cies can be coordinated. M&E in Indonesia. Participants included repre- Other development partners are also continu- sentatives from BAPPENAS, the MOF, UKP4, ing to support the GOI’s efforts to strengthen and the World Bank. A major issue discussed in M&E capacity. Policy dialogue with external these meetings was the need to streamline the technical assistance is being provided by differ- various M&E mandates, regulations, and report- ent institutions, including AusAID, the Japanese ing requirements. There was consensus both on International Cooperation Agency (JICA), and the need to share data and on the need to reduce the European Commission (EC). For example, in the number of indicators and thus the reporting terms of building M&E capacity, the World Bank burden for agencies at all levels of the government. has piloted a series of capacity-building workshops A national M&E roadmap, a manual for selecting for staff from the ministries of health, education, indicators, and a technical M&E working group public works, agriculture, planning, and finance. were developed. These workshops featured international and While the roundtable process was effective domestic M&E experts and included hands-on at raising awareness of the burden created by activities for participants. NOVEMBER 2012 PREMNOTE 7 Future Actions up� strategy there remains a need to better har- The development of M&E systems has been monize existing and proposed M&E processes. underway for some time, but in many respects is Stakeholders have legal mandates to collect M&E still just in its beginning phase. An important con- information, some mandates require collecting sideration guiding current efforts is the need for the same or similar information. Certain types strong incentives to develop and sustain effective of information are needed by multiple users; for systems. While the fiscal position continues to be example, budget disbursement, activity outputs, relatively benign, government recognizes the need and program outcomes in the short, medium, and for better M&E systems to support higher quality long term. Mindful of the lessons of the early years government spending, strengthen accountability, of M&E reform, a number of actions are proposed and increase the ability of the budget to respond to monitor the coherence of the overall system, to potential future fiscal challenges. As mentioned improve coordination, and reduce the costs of above, this has led to separate demand from BAP- M&E processes: PENAS and MOF for M&E information that sup- • Maintain a forum to discuss and coordinate ports the annual planning, budget, and reporting changes to existing M&E arrangements: Be- cycle. The short-term focus is on harnessing and cause of the relative newness of many of the supporting these demands for M&E information M&E regulations and the performance-based by developing more integrated processes that meet planning and budgeting systems, additional the needs of policy makers in the two agencies. M&E regulations are being considered to help The initial work by BAPPENAS and MOF is improve the process. Stakeholders should part of a broader focus on how M&E information meet to discuss and coordinate proposed is used for budgeting, planning, policy making, changes—possibly under the auspices of the management, and accountability. The annual roundtable group. planning and budgeting cycle is a key process to • Develop M&E information-sharing proto- effect policy reform and therefore has a need for cols: One of the most significant challenges in good M&E information. However, thought needs developing a more coordinated M&E system to be given to what M&E information would be is establishing protocols for information shar- most useful. In particular, it is likely that monitor- ing—based on information users agreeing on ing information and in-depth evaluations would the specific forms for information collection. play quite separate roles. With the former, it is The resulting capacity to share information possible for BAPPENAS and MOF to manage a will reduce the reporting burden across all review process with wide coverage of government sectors and at all levels of government. spending as part of the regular budget prepara- • Make use of the pending implementation of a tion process. Because Indonesia has introduced a new governmentwide financial management Medium-Term Expenditure Framework (MTEF), information system: While the principle it would be suitable to link this monitoring infor- focus of the FMIS will be on financial transac- mation to the update of institutions “baselines� at tions, there is capacity to collect and report the start of the budget preparation cycle. However, both financial and nonfinancial information. in-depth evaluations are much more resource Agreeing on the nature and frequency of in- intensive and a number of questions would need formation to be collected and shared could be to be considered. In particular: which programs a catalyst to better coordinate M&E systems. should be prioritized for in-depth evaluations? • Develop communities of practice to review What is the technical capacity within the govern- and promote improvements in the quality ment to conduct, analyze, and use the information of performance information. As in other from evaluations for decision making? When countries, there are many stakeholders with in the budget cycle should the evaluations be an interest in the performance information conducted? used in the budget, within government this The proposed approach envisages building includes the central agencies mentioned on areas where there are specific demands as a above, line ministries, the statistics and audit mechanism to move toward a comprehensive offices, and others. Ensuring that they have an M&E framework. However, with this “bottom- input and stake in the process of improving 8 PREMNOTE NOVEMBER 2012 and refining the M&E framework could help authors are also grateful to Gladys Lopez-Acevedo, to enhance its quality, relevance, and use. Enda E. Ginting, Jon Jellema, Jody Zall Kusek, Keith Mackay, Jose Carlos Rodriguez, and Susan Conclusion Wong for providing critical information and com- mentary. The team also expresses its gratitude When considering the Indonesian experience to the European Union, the Royal Netherlands with M&E reform, two important issues domi- Embassy, the Swiss government, and the U.S. nate. The first is the need to foster strong demand Agency for International Development for their for the information, to encourage the provision assistance in cofunding this work through the of relevant (and interesting), timely, and good Public Financial Management Multi-Donor Trust quality information so that the intensive effort Fund (PFM MDTF). needed to build and operate an M&E system is sustained. M&E information that is not linked to the decision-making processes of the government Notes tends to lose relevance and reliability. The second 1. Badan Pusat Statistik, 2012, July 31, retrieved is the need for ongoing policy coordination among from http://bps.go.id. the central agencies to avoid duplicating and con- 2. Excluding subsidies and debt repayment, the flicting requests for information that overload line central government and subnational governments’ ministries. For the M&E system to be effective, expenditures are at the about the same level. both issues need to be addressed, and various ap- 3. Since 2006, the World Bank has also published proaches have been attempted. The most recent Public Expenditure Reviews for certain sectors efforts have focused on encouraging individual (agriculture, health, social assistance, infrastruc- champions who want to use M&E information ture and education). in particular parts of the budget cycle to work 4. For example, the National Team for Poverty collaboratively with the other central agencies. Reduction (TNP2K) uses SUSENAS for targeting and analyzing the beneficiaries of social assistance; About the Authors SAKERNAS to examine the informal sector and minimum wage issues; and PODES to analyze the Mark Ahern is a Lead Public Sector Management availability and quality of village infrastructure. Specialist of the World Bank. He leads the Bank’s engagement on M&E reform and Performance Based Budgeting. Victoria A. Beard is an Associate References Professor of City and Regional Planning at Cornell Arizti, Pedro, James Brumby, Nick Manning, Roberto University. In 2009–10, she collaborated with the Senderowitsch, and Theo Thomas. 2010. Results, Performance Budgeting and Trust in Government. World Bank in Jakarta and advised on M&E reform Washington DC: World Bank. and capacity building. Anna I. Gueorguieva is an Aspinall, Edward, and Marcus Mietzner, eds. 2010. Economist with Poverty Reduction and Economic “Problems of Democratisation in Indonesia: Elec- Management of the East Asia and Pacific Region tions, Institutions and Society.� Institute of South- of the World Bank. While based in Indonesia for east Asian Studies, Singapore. the last three years, she led reviews of public ex- Boadway, Robin, and Anwar Shah, eds. 2007. Intergov- ernmental Fiscal Transfers: Principles and Practice. penditures and the M&E reform agenda. Retno Sri World Bank, Washington, DC. Handini is an M&E Specialist for the World Bank. Castro, Manuel Fernando. 2007. Towards the Insti- For the last three years, she has been working on tutionalization of Evaluation Activities and Tools the M&E reform engagement in Indonesia. in Planning and Budgeting Processes. World Bank, Jakarta, Indonesia Crouch, Harold. 2010. “Political Reform in Indonesia Acknowledgments after Soeharto.� Institute of Southeast Asian Stud- This note is the result of a team effort by the World ies, Singapore. Bank staff in Jakarta working on M&E and perfor- Funnel, Sue, and Rita Djayusman. 2008. “Evaluation Framework for Deputy of Development Perfor- mance-based budgeting. The authors are indebted mance Evaluation.� BAPPENAS, Jakarta, Indonesia. to Theo Thomas and Soekarno Wirokartono for Huage, Arild. 2004. Monitoring and Evaluation for their guidance and leadership in the development Results within the Government of Indonesia. Report of the note and in the field of M&E reform. The of a Rapid Assessment, World Bank, Jakarta Office. NOVEMBER 2012 PREMNOTE 9 Lopez-Acevedo, Gladys, Philipp Krause, and Keith SMERU. 2011. Monitoring and Evaluation of Mackay. 2012. Building Better Policies: The Nuts and Development Programs in Five Ministries: A Bolts of Monitoring and Evaluation Systems. World Study on the System of Implementation. Jakarta, Bank, Washington DC. Indonesia. Ministry of Finance. 2002. “Reform of Public Financial World Bank. 2008. Investing in Indonesia’s Health: Management System in Indonesia: Principles and Challenges and Opportunities for Future Public Strategy.� White Paper Publication Series, 2002/ Spending, Health Public Expenditure Review. Jakarta, KPMK/VII/MK/003. Jakarta, Indonesia. Indonesia. ———. 2012. RKA-K/L Compilation. Jakarta, Indonesia. ———. 2011a. Final Report: Results Based Management RAND. 2012. Indonesia Data Core, http://www.rand. in Thailand: Implementing Results Based Manage- ment in Thailand. World Bank Public Sector org/labor/bps.html. Performance Global Expert Team. Bangkok, Shah, Anwar. 1998. “Fostering Fiscally Responsive and Thailand. Accountable Governance: Lessons from Decen- ———. 2011b. Indonesia Social Assistance Program and tralization.� In Evaluation and Development: The Public Expenditure Review. Jakarta, Indonesia. Institutional Dimension, ed. Robert Picciotto and ———. 2012. Spending More or Spending Better: Improving Eduardo Wiesner, 83–96. New Brunswick, USA, Education Financing in Indonesia, Education Public and London: Transaction Publishers. Expenditure Review. Jakarta, Indonesia. This note series is intended to summarize good practices and key policy findings on PREM-related topics. The views expressed in the notes are those of the authors and do not necessarily reflect those of the World Bank. PREMnotes are widely distributed to Bank staff and are also available on the PREM Web site (http://www. worldbank.org/prem). If you are interested in writing a PREMnote, email your idea to Madjiguene Seck at mseck@worldbank.org. For additional copies of this PREMnote please contact the PREM Advisory Service at x87736. This series is for both external and internal dissemination 10 PREMNOTE NOVEMBER 2012