52820 the World bank notes November PREM Public Sector 2009 NUMBER 145 Implementing Public Expenditure Tracking Surveys for Results: Lessons from a Decade of Global Experience Asli Gurkan, Kai Kaiser, and Doris Voorbraak, PRMPS * Public Expenditure Tracking Surveys (PETS) can serve as a powerful tool to inform prevailing public financial management (PFM) practices and the extent to which government budgets link to execution and desired service delivery objectives and beneficiaries. Since the first PETS in Uganda in 1996, tracking exercises have now been conducted in over two dozen other countries, often as part of core analytical and advisory work related to PFM. This note synthesizes the findings and lessons from a number of recent PETS stocktaking exercises and indicates their potential benefits for enriching PFM and sectoral policy dialogues in a variety of country settings. Key findings include: (i) PETS have proven to be useful as part of a broader policy strategy aimed at improving service delivery results; (ii) PETS has become a brand name for very different instruments, but at its core there is a survey methodology that requires skilled technical expertise and a solid knowledge of budget execution processes; (iii) policy impact in a variety of PETS experiences could be further strengthened by stronger country ownership and effective follow-up; and (iv) the Bank could enhance PETS results through strategic partnering, and greater emphasis on dissemination and communication strategies aimed at involving actors who can foster actions on the ground. A. What is a PETS? provide a measure of leakage of resources. In A PETS tracks, locates, and quantifies the seeking to derive representative and cred- flow of public resources across various ible quantitative information on whether administrative levels of government. It and how much funds actually reach points aims to determine, on a sample basis, how at the end of the public expenditure much of the original allocations (financial, chain, a full-fledged PETS will typically salaries, in-kind items) reach the next level need to collect information concerning of government and, ultimately, service the expected and actual receipt of public delivery units such as clinics and schools. funds for several dozen, if not hundreds, Unaccounted resources between levels of front-line service delivery points. * This note synthesizes findings presented at a 2009 PREM Learning Week half-day course. Special thanks goes to Zafar Ahmed for additional contributions to the note. The review draws on an inventory of PETS, interviews with 18 World Bank task team leaders (TTLs), a desk review of both existing and newly commissioned papers, a field-based review of five country case studies (Cambodia, Ghana, Tanzania, Uganda, and Zambia) undertaken by ECORYS (2008), as well as an assessment of civil society engagement around PETS (Ramkumar 2008). FROM THE POVERTY REDUCTION AND ECONOMIC MANAGEMENT NETWORK A PETS helps focus on the links be- but are also pertinent to higher-capacity tween effective PFM and actual service settings elsewhere.2 delivery. Consequently, it is potentially More compelling and accessible in- valuable for the cross-cutting dialogue on formation concerning budget execution PFM, often conducted among stakeholders for service delivery beneficiaries can most in ministries of finance and the Financial importantly help draw in enhanced popu- Management (FM) and Poverty Reduc- lar demand across service delivery ben- tion and Economic Management (PREM) eficiaries and civil society organizations families in the World Bank. This work is (CSOs). While this note considers PETS particularly relevant to sectoral dialogues from mostly a PFM perspective, there are in education and health (notably the Hu- other approaches, such as viewing PETS man Development sector, HD), as well as through the lenses of sector analysis, and other infrastructural and administrative civil society advocacy. service delivery sectors (see Table 1). The Successful PETS can expect to achieve implementation of a PETS has itself be- a range of results. Whether or not sur- come part of a benchmark for achieving vey results are able to quantify leakages, basic performance scores within the Public PETS can shed significant light on the Expenditure and Financial Accountability actual functioning of public expenditure (PEFA) indicator concerned with frontline systems, including planning and manage- service delivery financing (see Kaiser and ment capacities of ministries; delays in Steinhilper, 2008).1 The PETS can speak disbursements and volatility of transfers; most directly to "implementation deficit" equity in allocation; and accountability issues in PFM. This includes the signifi- mechanisms. By focusing on the opera- cant, albeit challenging, reform agenda tional impact of budgets, PETS studies can in Africa, and the observation that many help reveal whether spending from higher "African countries tend to make budgets levels of government meets its intended better than they execute them" (see An- budget allocations at the point of frontline drews, 2008:22). These observations are service delivery units. certainly not limited to the Africa region, Table 1: Examples of PETS objectives Country/Study Objective Uganda: Education PETS 1996 Assess why increased public expenditures in the social sectors are not leading to improved social indicators. Brazil: Health PETS 2006 Assess whether the resources transferred to states and municipalities are used for the intended purposes. Madagascar: Health PETS 2006 Explore the potential of premium schemes. Mongolia: Education PETS 2006 Assess equity and regional disparities in quality. Democratic Republic of Congo: Examine spending processes that precede the payroll Education PETS 2007 expenditure phase at the provincial level. Source: PREM Public Sector Group (PRMPS) PETS Database. 2 PREMNOTE NOVEMBER 2009 Central to PETS, in a sense, is a mea- to randomly selected units through sci- sure of budget credibility at a micro level. entific sampling methods. By choosing a In this regard, the findings of a PETS particular flow of resources that can be narrowly defined can only be as good as customized to a sector situation or a spe- the intended budget allocations and their cific program, estimations of the extent timing that are used as milestones. The ex- of leakages can be determined. In recent istence of an education capitation grant in years, a number of nongovernmental or- Uganda provided clarity for benchmark- ganizations (NGOs) have used methods ing this expenditure flow, and hence the combining elements of case-specific track- headline result that less than 13 percent ing and survey-based exercises under the of these expenditures reached their in- PETS banner (Ramkumar, 2008). The tended beneficiaries. One tangible impact name "PETS" has also been used inter- of a PETS process may, therefore, be to changeably with other survey methods institute greater clarity and transparency including surveys of absenteeism and as to what frontline facilities are actually ghost workers. entitled to in the public budget, and in The best known PETS, particularly turn, an entry point for citizens and civil for its strong policy impact, remains the society to hold government to account. Uganda study of 1996. The study was able The Mongolia PETS, for example, encour- to trigger a concrete set of policy actions aged the adoption of allocation rules with such as publishing intergovernmental greater equity for rural areas. transfers of public funds in major news- But PETS has become a broad brand papers, and replacing the central supply name for different types of products, even of in-kind materials with school-based including lighter expenditure flow-scop- procurement. The smart information and ing exercises that do not encompass core communication campaign on the PETS aspects of the PETS survey methodology results, combined with strong political application.3 The PETS method refers interest in reforms, contributed signifi- Seven Elements for an Effective PETS 1. DefinetheobjectivesandscopeofthePETS,includingwhethertolookataspecificexpenditure programorprogramcomponent,orspecifictransferssuchascapitationorbursaryschemes. 2. GiventhescopeofaPETS(forexample,thefacilitiesandflowsproposedforsurvey),ensure adequate time and resources for completion. 3. EngagekeycounterpartsupstreamofthePETStodevelopcooperationandownership. 4. Conductaninstitutionalmappingofkeyresourceflowstofront-linefacilities(includingcashand in-kind)priortosurveyimplementationandprioritizetheseresourcesbasedonpolicyrelevanceand measurability. 5.ConductaRapidDataAssessment(RDA)toverifyifthedatarequiredareavailable,andensure adequate testing of the survey instruments and close monitoring of data collection. 6. Considerthepolicypositioninginrelationtokeycross-cuttingandsectoralministries,includingfor within the overall PFM/public expenditure dialogue. 7. Identifyclearpolicyrecommendationsandactionablereforms,anddedicatesufficienttimeand resourcesforeffectivedisseminationandfollow-up. NOVEMBER 2009 PREMNOTE cantly to make real change possible. The · Public Expenditure Reviews (PERs), which work in Uganda was also complemented analyze the country's fiscal position, its by a series of robust policy research impact expenditure policies (in particular, the evaluations that demonstrated the impact extent to which they are pro-poor), and that information can have on service de- its expenditure management systems; livery outcomes (Reinikka and Svensson, · Country Financial Accountability As- 2004). sessments (CFAAs), which evaluate the Feedback by task team leaders (TTLs) strengths and weaknesses of account- and clients suggests that well-designed ability arrangements for managing and implemented PETS can make very public resources in areas like budget- powerful contributions to the policy ing, accounting and audits, and also dialogue, including how expenditures are identify the risks these may pose to the monitored and various program compo- use of World Bank funds; nents are held accountable on a day-to-day · Public Expenditure and Financial Ac- basis. Note that leakages as measured by countability (PEFA) indicators, which PETS do not necessarily imply corruption, assess a country's PFM system against a but in some cases simply reflect realloca- list of indicators in the areas of budget tion of funds for alternative priorities at credibility, comprehensiveness, trans- intermediate levels of the bureaucracy. parency, and the stages of the budget PETS are most appreciated for provid- process (that is, planning, execution, ing additional information to the PFM reporting, and audit); and sectoral dialogue, and in many cases · Country Procurement Assessment Reviews documenting and systematizing weakness- (CPARs), which examine public pro- es already sensed by the various counter- curement institutions and practices in parts. However, there are some concerns borrower countries; about the impact and actual follow-up · Public Investment Management/Efficiency effectiveness of PETS. Weak country own- Reviews (PIM/ER), which examine the ership and political will sometimes limit efficiency of the public investment the achievement of expected results. Few management function; PETS are positioned as instruments for · Reports on the Observance of Standards and creating more accountability in deficient Codes (ROSCs), which help countries service delivery systems. Survey results strengthen their financial systems (in- often are not actively disseminated to civil cluding accounting and auditing) by im- society. Nevertheless, the larger question proving compliance with internationally that emerges is how this information is fed recognized standards and codes; into wider PFM reform action plans. · Institutional and Governance Reviews (IGRs), which evaluate the quality of B. Positioning of PETS in the accountability, policy-making, and ser- World Bank: A Snapshot vice-delivery institutions in a country There are a number of diagnostic eco- from a broad governance perspective; nomic and sector work (ESW) products · Governance and Anticorruption (GAC) that the World Bank often uses to analyze diagnostics, which examine shortcom- a country's PFM system (World Bank, ings of formal PFM systems; and 2008a). These include: · Public Expenditure Tracking Surveys (PETS). 4 PREMNOTE NOVEMBER 2009 A Bank Independent Evaluation the context of Poverty Reduction Strat- Group (IEG) review notes that the PETS egy Paper (PRSP) exercises and Heavily has proven to be a powerful addition to the Indebted Poor Country (HIPC) initia- PFM toolkit (World Bank, 2008b). It has tives. PETS have been seen as valuable in been helpful in identifying problems with post-conflict settings such as Sierra Leone expenditure and financial management, and Rwanda to establish service delivery including corruption in these areas. The baselines in contexts where no such infor- development of this tool by the Bank has mation was available. also helped to improve transparency and The foremost motivation of Bank societal accountability. The review also staff has been to identify problem areas in notes that the costs and time demands public expenditures and related account- have limited the application of PETS for abilities. A large number of PETS have more universal application. been undertaken by the Bank to examine PETS are often conducted in conjunc- country policies and to assess the impact tion with the broader PFM engagement of specific social programs (see Table 2). with clients by the Bank. Often they are PETS can be useful for upstream dialogue undertaken as part of PERs or in parallel, and can be a tool for joint analysis and and also have direct links to the bench- collaboration between a sector ministry marking of key PEFA indicators (PI-1 to and the Bank team. PI-21). PETS have also frequently used The majority of PETS have been to speak to concerns around the quality implemented in the Africa Region and in and targeting of public expenditures in the HD sector. Most of the follow-up stud- Table 2: Selected Examples of PETS Assessing Country Reforms/Policies Country Program/Policy Description Peru GlassofMilk(VasodeLeche) The study serves as an impact evaluation for the program largest social transfer and the second largest component of transfers from the central government to municipalities. Mongolia Education Funding Formula The main question examined is the impact of the government'sfundingformulaonefficiencyandequity across regions. Cambodia Priority Action Program (PAP) in ThestudyexaminesfinancingofPAP. primary education Primary healthcare reform This study collected data on the interaction between differentlevelsofthehealthsystem,inparticular,in relationtofinancing,allocation,distribution,anduseof healthcare resources. Ghana Decentralization in service Thestudywasundertakentomeasuretheimpact delivery of the government's budget increases for health andeducationinlinewiththelegalframeworkof decentralization. Source: PRMPS PETS Database NOVEMBER 2009 PREMNOTE 5 Figure 1: Distribution of PETS by Sector/Theme 50 45% 45 40% 40 35 30 Percent 25 20 15 10 4% 4% 5 2% 2% 2% 0 Health Education Poverty Agriculture Water and Roads Justice Reduction Sanitation Sector Source:PETSDatabase(N=47Country,Sector,YearPETS).. ies have also been carried out in Africa, Delivery Surveys (QSDS). The QSDS including Uganda, Tanzania, Madagascar, emerged from the PETS experience and and Zambia. Education and health have go beyond the PETS focus of tracking been the most common sectors in which funds. It examines the efficacy of spend- PETS were implemented, although there ing, as well as incentives oversight, and the are examples of the water & sanitation and relationship between those who contract agriculture sectors (Figure 1). In Burundi, for a service and those who deliver it. a PETS was also completed in the justice sector, as well in health and education. Figure 2: PETS TTL Affiliations However, there is scope for wider ap- plicability of the PETS instrument both regionally and sectorally, perhaps even in middle-income countries with more sound External budgeting and accounting systems. agency, PREM, The initial batch of PETS at the Bank 21% 28% were primarily led by Development Eco- nomics (DEC), followed by HD and PREM. PETS TTLs have been mostly affiliated DEC, 36% with these three networks (Figure 2). The HD, 15% different professional affiliations of TTLs have manifested themselves in a diversity of PETS survey designs and objectives. PETS have frequently been used in combination with Quantitative Service Source:PETSDatabase(N=39Country/YearPETS). PREMNOTE NOVEMBER 2009 The frontline facility or service provider up to the US$400,000 range.4 The TTLs is typically the main unit of observation in indicated that a large part of the costs was QSDS (just as the household is the unit of often for hiring expensive survey firms, or observation in household surveys, or the senior international experts. firm in surveys of the investment climate). Sufficient time and resources are nec- In some cases, beneficiaries are also sur- essary to plan, design, and implement a veyed to allow for cross-validation of in- survey. Normally, it takes about one year to formation. Absenteeism and ghost worker complete a PETS. The length of the survey studies use the QSDS approach. These depends on the time needed for designing combined studies allow the examination of the suitable methodology, finalizing the how resources are used and services offered Terms of Reference (TOR), and hiring to the population in addition to identifying consultants. For several PETS conducted leakages. Examples of combined PETS/ in West Africa, the process of identifying QSDS include the Chad, Nigeria, and key survey objectives and designing survey Zambia surveys (Gauthier, 2006). instruments took an average of three to six The key national partners for PETS months, and the hiring of local consultants are very context-specific but largely de- took about six months. In some cases, termined by incentives and the power dy- survey lengths were longer. (For example, namics among ministries. To implement the Mozambique study, originally planned the surveys, the World Bank teams have for six months, ultimately took 24 months traditionally worked in cooperation with to complete.) In some cases, the collection central ministries, namely, the ministries (and "cleaning") of data took considerable of finance and sectoral ministries. Minis- effort despite the development and use of tries of finance have been supportive of expensive survey forms and data collection PETS initiatives as they have a clear incen- methods. tive in seeing sector-based resources being Dissemination activities have varied used more efficiently. On the other hand, depending on the level of involvement there are cases where sectoral ministries by the national actors (see Table 3). Most have been on the defensive with respect Bank teams have organized dissemination to the studies; this is because the PETS activities after the completion of the sur- are perceived as part of anticorruption veys. In most cases, the activity consisted crusades and as a way to cut public spend- of a one-time meeting with the relevant ing in their respective sectors. (This was ministries. Often, smaller dialogue rounds the case, for example, with the Nigeria are more effective than large scale plenary Health PETS.) presentations. The cost of the survey depends on a A recent trend has been to engage number of factors, including its scope, its demand-side actors for collecting quanti- combined PETS/QSDS nature, the sample tative and qualitative information and for size, geography, and labor costs in the disseminating PETS findings. A number country. Costs of surveys undertaken in of studies tapped into the incentives and a single sector range from US$75,000 to existing capacities of nongovernmental US$200,000. When two or more sectors actors including NGOs, CSOs, commu- are considered jointly involving a larger nity-based groups, PTAs, and unions (see sample and two surveys, the cost might go Table 4). NOVEMBER 2009 PREMNOTE Table 3: Selected Examples of Dissemination Activities Country Dissemination Activities Mali RepresentativesfromtheIMF,MinistryofEducation,MinistryofFinance,aswellas sectoralunionsparticipatedinthehalf-dayworkshopanddiscussedthesurveyresults. Mozambique Regionalworkshopswereheldtodiscussthefindings,whichincludedMinistryof FinanceandMinistryofHealthofficials,andhospitaldirectors. Ghana LocalFMradiosranaprogramaboutthePETSfindings. Uganda Thefindingsofthestudieswerepostedonthedoorsoffacilitiestomakethemavailable toallstaff,clientsassociatedwiththefacilities,andtoencouragedemand-side accountability/consumer empowerment. Source:Ramkumar(2008). C. Some Challenges culations in the books. This has, in some in Conducting and situations, resulted in a flawed perception Disseminating PETS by the PETS teams that leakage or even Measuring resource leakages has been corruption has occurred. Collection of difficult in some countries due to the com- data on "total resource flows" can be dif- plexity of resource flows, and lack of reli- ficult sometimes. In general, collection able and available data.5 In Sub-Saharan of data, including budget data, can pose Africa, countries rarely use hard allocation significant challenges in many countries. rules and often it is up to administrations National counterparts, even when at district levels to decide how resources supportive of the study at inception, can are allocated. Measuring leakage is par- sometimes dispute the validity of findings. ticularly challenging for in-kind transfers, In Rwanda, the Ministry of Finance ques- because often the cost of these transfers tioned the claim that there were significant is unknown at the facility level, signaling delays in delivery of resources. In Mali, the problem of information asymmetry. the government officials disputed the Lack of clear allocation rules particularly methodology that led to the findings of creates problems if a PETS aims to track corruption (school books were being sold resources for several years.6 in the market). The results of this study Even when the funds do reach the in- subsequently were not released in the tended beneficiaries, the lack of detailed public domain and apparently received accounting might lead to erroneous cal- no follow-up. Table 4: Examples of PETS Process Led by NGOs Country Name of NGO Tanzania REPOA,theEconomicandSocialResearchFoundation(ESRF) Malawi The Civil Society Coalition for Quality Basic Education (CSCQBE) TransparencyInternational(TI);CatholicCenterforJustice,Developmentand Zambia Peace (CCJDP); and Zambia National Education Coalition (ZANEC) Mongolia The Open Society Forum Source:Ramkumar(2008). PREMNOTE NOVEMBER 2009 Some of these tensions with the na- and (ii) bottlenecks and policy ques- tional counterparts can be avoided if Bank tions were thrashed out and taken into staff adequately address political economy consideration in the PETS design. And, considerations upfront and make them an on the counterpart side: (i) the unit to integral part of the PETS process. A num- be tracked was clearly identified; (ii) ber of TTLs noted that PETS can be often lower levels of administrations did not seen as an instrument imposed from the have independent funding sources; outside, and consequently civil servants (iii) lower levels of administrations did may not necessarily take part in the pro- not have discretionary powers in the cess. Care must be taken that the PETS is use of funds; and (iv) funds were not not perceived as a punitive or audit type disbursed by the central government instrument of one agency or government directly to service facilities (Savedoff, level in relation to another, but as a con- 2008). If these preconditions are met tribution to broader reform efforts. Poorly from the start, a PETS may be the most timed completion and dissemination of a appropriate tool for identifying certain PETS with respect to the electoral cycle specific sectoral challenges. In cases may also be unconstructive, and therefore where the country context is not suitable, needs to be carefully assessed for a given alternative approaches towards tracking country context. budgets could be more appropriate op- The institutional structures at the tions in terms of time and resources. Bank can sometimes create a challenge in · Invest enough time and thought in the the follow-up of PETS findings and similar planning stage, typically 3­6 months products. Bank incentive structures can depending on the country context be focused on deliverables and fiscal year and previous diagnostic work: TTLs in budgets; thus, the Bank teams (often led particular should consult with other at by a TTL based at Headquarters) may be the country level and think through the less concerned with what actually happens usefulness of doing PETS before initi- with such documents once the initial work ating fieldwork. Successful planning is completed. Once the survey is done, for PETS would include: mapping the peer reviewed, and published, team mem- respective resource flows, determining bers usually move on to other assignments. the scope for leakage, and establishing This may also result in lack of clarity for how the various public expenditure the division of roles and responsibilities flows are being recorded. for the dissemination stages. · Set appropriate and attainable objec- tives based on the country context and D. Lessons Learned: available information: Quantifying Suggestions for PETS Teams leakage should not be always considered a main objective of PETS. The objec- Planning tive of the study could be to identify · Determine whether PETS is the right problems in the deployment of human tool: PETS have been most effectively and in-kind resources, such as staff, text- implemented and used in cases where, books, and drugs; or, to investigate spe- on the Bank's side: (i) the time, human cific issues such as timeliness of resource resources, and funding allocations in delivery by paying further attention to the planning phase were sufficient; lags in service delivery (see Table 5). NOVEMBER 2009 PREMNOTE 9 Table 5: Selected Examples of Qualitative Findings of PETS in Education Country Year Findings Congo,DemRep.of 2007 Answerabilityoffinancialofficersisnotwelldefined;cashleakages for salary distribution occur at the payment level. Albania 2004 Three different and uncoordinated allocation methods exist for school supplies and teaching materials. Municipalities are very unequal in their funding capacity. Azerbaijan 2006 District Education Departments are not accountable to schools and thushavenoincentivetodeliverschool-basedfinancialdocuments (smetas). Madagascar 2003 Teachers hired by Parents Associations (FRAM) earning less than theofficialminimumwageisakeyfactorforlowqualityofeduca- tion in rural areas. Source: PRMPS PETS Database. · Have a fact-finding mission for an insti- circulate survey instruments for com- tutional assessment, mapping of roles ments. and responsibilities of relevant actors, · Evaluate the trade-offs for doing a and review of stakeholders: A detailed joint PETS: A number of teams have institutional understanding would be conducted joint PETS in two or more greatly enhanced by being in the field sectors, most commonly as joint health and seeing the way public resource al- and education PETS (for example, location actually works on the ground. Ghana and Rwanda). Doing joint PETS If the TTL is new to the country, it is could certainly be beneficial in terms always useful to treat the first visit as of economies of scale; it is, overall, a a fact-finding mission and have the more cost-effective way of completing flexibility to adjust the objective, scope, the survey work. Yet, a number of TTLs and the survey design accordingly. have raised concerns about the quality · Promote country ownership from the because each sector has its own dynam- planning stages. The message should ics, and needs its own data collection be clear from the start that it is not the methods and survey questions. intention of the World Bank to embar- · Evaluate the trade-offs of covering rass and undermine the government by single versus multiple types of expen- documenting misdemeanors, exposing ditures or facilities: In many country corrupt deeds, and identifying indi- contexts, there is no effective data col- viduals. PETS are likely to have the best lection system; thus, the survey teams results if key client counterparts believe have to start with a broad agenda to the study findings would help govern- identify problem areas. In many cir- ment to do a better job at effectively cumstances, it might be impossible to managing its funds and provide better conduct PETS on the basis of a census services for the people. One way to because of the large number of facili- start building consensus around PETS, ties in the country and resulting time as suggested by Gauthier (2006), is to and cost factors.7 The survey team may 10 PREMNOTE NOVEMBER 2009 decide to have a greater scope and have · To the extent possible, ensure that a larger sample of facilities, but that the mix of individuals in the PETS often results in worse quality of data. team have adequate experience in When the type of expenditures can be similar type of surveys and substantial quantified, then TTLs have suggested country knowledge. An experienced focusing on micro issues.8 and knowledgeable team may create a more realistic timetable and conduct Implementation more suitable activities for this kind · Recognize that a well-managed process of survey. Ideally a PETS team is com- can unleash additional benefits to the posed of people with different skills PETS survey results: Several teams ob- and perspectives, including audit and served that the consultations, dissemi- sector-specific experience. An alterna- nation, and feedback processes created tive approach to hiring experts has a platform for ministries and individ- been to engage government officials to ual authorities, who have traditionally carry out the work, as was the case with been skeptical of each other, to culti- Cameroon and Sierra Leone. However, vate trust and create the incentive for this approach may raise questions on them to work towards a common goal. the capacity and independence of the This is particularly important because teams and the objectivity of the data in many country contexts, mistrust and collected. In the case of Indonesia, secrecy may prevail among and across ministry staff were part of the enumera- ministries. The idea of creating a steer- tor team. While this was useful for local ing committee is a particularly useful capacity building, the enumerators approach for promoting cooperation were not necessarily well trained to among different governmental and elicit correct financial information.9 nongovernmental bodies. Recruiting the right people at the local · Understand that it is ultimately the level is also key.10 Civil society organi- respective sector ministry that needs zations would be valuable resources in to be the main actor for the imple- some contexts for carrying out PETS mentation of policy: It does not always with their local knowledge and orga- yield the optimum results to work with nized structure.11 the ministry with which there exists established relations (oftentimes the Dissemination and Follow-Up ministry of finance for the Bank team) · Strike a practical balance for making and bypass other ministries. The min- findings from a PETS actionable: The istry of finance has its own incentives, nature of a PETS may range from being which may or may not be suitable to the purely diagnostic (for example, survey agenda. Even when sectoral ministries results on leakages), analytical, or even might be harder to work with from the incorporate impact evaluations (for ex- donor's perspective in the short run, ample, assessing the impact of certain it is crucial to engage them for the interventions). The background work longer-term sustainability of effective and survey are likely to provide indica- public expenditure management in a tions of a range of weaknesses in pre- particular sector. vailing systems. This is to be expected, NOVEMBER 2009 PREMNOTE 11 since by its very motivation and nature, ships between the government and PETS are applied in settings where civil society or user groups can sig- monitoring and auditing systems are nificantly enhance the capacity of the weak. An important challenge for government and perform an oversight PETS will therefore be to help define function in ensuring the delivery of key priorities and tractable options for services. However, involving NGOs/ addressing these. The Ghana PETS CSOs in the policy discussions on sec- (ECORYS, 2008) highlighted that the tor specifics may not always be a useful process should be used to encourage contribution. It would be better to have the definition and ownership around a targeted approach, and promote context-specific solutions, rather than their contributions according to their prescriptions with limited promise for specific issues of interest.12 follow-up results. · Establish a PETS monitoring and fol- · Carefully align the timing and dis- low-up mechanism: A number of TTLs semination of the PETS results with the suggested that the first PETS should event schedules of both government be used as a baseline to pave the way counterparts and the wider public. for a more comprehensive follow-up As it takes a considerable amount of survey.13 PETS could be conducted on a time to finalize the study, the timing much smaller scale in a first test-round for releasing the findings needs to be involving a smaller number of facilities planned carefully to align construc- and districts. Although a small sample tively with reform processes. In the might not be representative, this would case of Ghana, the dissemination of still be an illustrative exercise to un- the results coincided with the 2000 derstand the underlying problems. elections, which created challenges in Some TTLs have argued that PETS implementation. should be a repetitive mechanism. A · Determine the share of responsibilities PETS could serve as a reference when- among Bank units for dissemination ever there is new work taking place on activities: Country Management Units, that particular sector in the country. particularly those staff based in the Repeated PETS promise to be an im- field or with active communication with portant instrument for benchmarking the government, should take a share of progress in implementation of reforms the responsibility for initiating and fol- over time. An emerging number of re- lowing up on dissemination activities peat PETS across a range of countries to ensure that there is clear ownership can speak to this issue. In terms of of the process. TTLs should be con- "headline" numbers of fiscal leakages, sidered a part of the country unit--not care must be taken, however, that like left alone to complete the tasks. There is compared with like. The impressive should also be a plan to disseminate declines in fiscal leakages from 87 the findings in local language transla- percent to below 20 percent evidenced tions. by the Ugandan case pertained only · Do not overestimate the capacity and to the capitation grant. Changes in "neutrality" of civil society: Partner- institutional arrangements or types of 12 PREMNOTE NOVEMBER 2009 financial flows (notably from budgets the World Bank: Rotterdam, The Neth- based on discretionary versus transpar- erlands (December). ent--for example, rules-based--alloca- Engberg-Pedersen, Poul, Kai Kaiser, and oth- tions) may also be important examples ers. 2005. "Public Expenditure Tracking of progress. Teams may thus wish to Surveys: A Stocktaking." Draft: World prioritize flows both in terms of their Bank, Washington, DC. volume as well as their nature as critical Gauthier, Bernard. 2006. "PETS-QSDS inputs for a frontline service to func- in Sub-Saharan Africa: A Stocktaking tion well. In focusing on specific public Study." World Bank, Washington, DC expenditure resource flows, the PETS (September). Gauthier, Bernard, and Ritva Reinikka. provide some of the most tangible 2007. "Methodological Approaches to forms of evidence of PFM performance the Study of Institutions and Service down to the actual frontlines where ser- Delivery: A Review of PETS, QSDS and vices actually intersect with intended CRCS." World Bank, Washington, DC. beneficiaries. Hubbard, Paul. 2007. "Putting the Power of Transparency in Context: Informa- In sum, successfully designing, imple- tion's Role in Reducing Corruption in menting, and leveraging a PETS is not Uganda's Education Sector." Working without its challenges. But the PETS re- Paper Number 136. Center for Global mains one of the most powerful tools to Development, Washington, DC. systematically drill down and ultimately Kaiser, Kai, and David Steinhilper. 2008. help strengthen actual public expenditure "Intergovernmental Public Financial processes for frontline service delivery Management for Decentralization: impacts. Evidence from the Public Expenditure & Financial Accountability (PEFA) Bench- References marking Indicators." World Bank and Amin, Samia, Jishnu Das, and Markus Gold- GTZ: Paper presented at National Tax stein, eds. 2007. Are You Being Served? Association Annual Meetings, Philadel- New Tools for Measuring Service Delivery. phia, PA. Washington, DC: World Bank. Ramkumar, Vivek. 2008. "Civil Society Ex- Andrews, Matthew. 2008. "Managing Money periences in Monitoring Government for (a) Change: Themes and Challenges Budgets." Draft prepared for the World for African Public Financial Manage- Bank (PRMPS). International Budget ment." Paper for the Brookings Institu- Partnership, Washington, DC. tion--Transparency and Accountability Reinikka, Ritva, and Emmanuel Ablo. 1998. Project, and the World Bank--Africa "Do Budgets Really Matter? Evidence Region. Harvard University, Kennedy from Public Spending on Education School of Government, Cambridge, MA and Health in Uganda." Policy Research (August). Working Paper Number 1926. World ECORYS Nederland BV. 2008. "How to Bank, Washington, DC (June). Conduct a Successful PETS? Lessons Reinikka, Ritva, and Jakob Svensson. 2004. from Five Countries on How to Shape "Local Capture: Evidence from a Cen- the PETS Process." Draft prepared for tral Government Transfer program in NOVEMBER 2009 PREMNOTE 1 Uganda." The Quarterly Journal of Econom- Implementation and Uses." Report sub- ics 119(2) (May): 678­704. mitted to the World Bank. Social Insight, ------. 2006. "Using Micro Surveys to Portland, ME. Measure and Explain Corruption." World World Bank. 2008a. "Enhancing Capabili- Development 34(2) (February): 359­370. ties of Central Finance Agencies: From Reinikka, Ritva, and Nathanael Smith. Diagnosis to Action." PRMPS Draft 2004. Public Expenditure Tracking Surveys Framework Paper. World Bank, Wash- in Education. International Institute for ington, DC. Educational Planning. Paris, France: ------. 2008b. "Public Sector Reform: What UNESCO. Works and Why? An IEG Evaluation Savedoff, William D. 2008. "Public Ex- of World Bank Support." World Bank, penditure Tracking Surveys: Planning, Washington, DC. 14 PREMNOTE NOVEMBER 2009 Annex A: Selected PETS/QSDS Region/ Initial Country Year Type Sectors Sponsor AFR Burundi 2007 PETS Health/Education/Justice WB-PREM Cameroon 2004 PETS Health/Education WB-PREM Chad 2004 QSDS Health WB-DEC Congo,Dem. 2007 PETS Education (teacher payroll WB-PREM Rep. of expenditure) Ghana 2000 PETS Health/ Education WB-HD Kenya 2004 PETS Preliminary Report Health/ Education DFID Madagascar 2003,2005, PETS-QSDS Health WB-DEC 2006 Mali 2005,2007 PETS-QSDS Education,Health WB-HD Mozambique 2001,2004 PETS-QSDS,PETS Health DFID Namibia 2004 PETS-QSDS Health,Education IFPRI Niger 2008 PETS Health,Education WB-PREM Nigeria 2004 PETS,QSDS Health WB-DEC Rwanda 2003,2004 PETS Education,Health/ WB-PREM Education Senegal 2002 PETS Health WB-DEC SierraLeone 2000/01, PETS1and2,PETS- Agriculture/Health/ DFID 2003 review Education/Water and Sanitation Tanzania 1999,2001, PETS and PETS review Health,Education, USAID, 2003,2004 Agriculture and Roads REPOA Uganda 1996,2001, PETSandPETS-QSDS Health,Education WB-DEC 2003,2004 Zambia 2001,2002, PETS-QSDS(3) Education WB-DEC 2004 EAP Cambodia 2005 PETS (2) Health,Education WB-HD LaoPDR 2008 PETS Health,Education WB-PREM Mongolia 2006 PETS Education WB-HD Papua New 2002 PESD Health WB-DEC Guinea ECA Albania 2004 PETS (2) Health,Education WB-HD Azerbaijan 2006 PETS Education WB-PREM Tajikistan 2008 PETS (programmatic PER) Health WB-PREM LCR Honduras 2001 PETS +QSDS Poverty Reduction WB-PREM Peru 2002 PETS +QSDS PovertyReduction, WB-PREM Education SAR Bangladesh 2003 Absenteeism Survey Health WB-HD MNA Yemen 2006 PETS Education WB-PREM Source:PublicSectorGroupPETSinventory(beginning2009).Thelistattemptstobecomprehensive,butomissionsof particularlymorerecentPETStasksarelikely. NOVEMBER 2009 PREMNOTE 15 Endnotes expenditure allocation: the capitation grant 1. The PEFA indicators have now been system, which was based on clear rules, thus applied in almost 100 country settings, with trackable rather easily. an increasing number of repeat assessments 9. The United Nations Educational, Scien- coming on stream. tific and Cultural Organization's (UNESCO) 2. Andrews (2008) suggests that PFM International Institute for Education Plan- performance, as captured by the various ning (IIEP) has developed a training course sub-dimensions of PEFA scores, becomes on PETS in education. The Mongolia and more challenging when they need to involve Cambodia PETS teams, among others, ben- multiple/deconcentrated actors, rather than efited from this training. concentrated functions. 10. A positive example is Nigeria, where 3. Detailed PETS methodology, guidance, the lead local consultant was a very qualified and good practice notes are under prepara- doctor and who was instrumental in designing tion for use by Bank TTLs. the survey and finding qualified professionals 4. For example, the costs for implementing for data collection. PETS in Chad was US$150,000­US$200,000; 11. The advantage of using CSOs is not Mali, US$140,000; Cameroon, US$300,000 in the survey design and data collection (larger sample and two surveys); and Tajiki- stages, but rather in disseminating PETS stan, US$75,000. findings and overseeing service delivery 5. Chad, Madagascar, Tajikistan, and Mo- performance. zambique are examples with no rules-based 12. Ramkumar (2008) notes that the big- allocation systems. gest challenge CSOs face in implementing 6. Furthermore, finding an average per- PETS (and, indeed, in all budget monitoring centage of leakage may not accurately reflect work) is access to information. In addition to large variations in leakage across schools, as this challenge, CSOs also struggle with the observed in Reinikka and Svensson (2004). significant scale of coverage, resources, and 7. Interview with Waly Wane (TTL for technical skills required to implement PETS Chad PETS). comprehensively. 8. One of the reasons why the 1996 Uganda 13. This was the case with the Uganda study successfully captured leakage was be- survey. cause it was able to focus on a specific type of This note series is intended to summarize good practices and key policy findings on PREM-related topics. The views expressed in the notes are those of the authors and do not necessarily reflect those of the World Bank. PREMnotes are widely distributed to Bank staff and are also available on the PREM Web site (http://www.worldbank.org/prem). If you are interested in writing a PREMnote, email your idea to Madjiguene Seck at mseck@worldbank.org. For additional copies of this PREMnote please contact the PREM Advisory Service at x87736. PREMnotes are edited and laid out by Grammarians, Inc. Prepared for World Bank staff