64633 S P D I S C U S S I O N PA P E R NO. 1119 Measuring Governance and Service Delivery in Safety Net Programs Gloria M. Rubio September 2011 Measuring Governance and Service  Delivery in Safety Net Programs  Gloria M. Rubio1 September 2011 Human Development Network The World Bank                                                              1  This paper was jointly commissioned by the Knowledge and Learning Program on Measuring Governance  and Service Delivery at the HD Chief Economist’s Office and the Social Protection Strategy and Results  Group. Dena Ringold and Laura Rawlings provided valuable guidance for this work. Irvin Rojas (COLMEX)  provided important research assistance by collecting and reviewing information on governance assessment  tools. Emil Tesliuc and Lucian Pop provided helpful comments and Rodrigo Quintana helped edit and format  and gave useful inputs.  1    Abstract This paper develops a framework to assess organizational performance in the delivery of social safety nets. Specifically, it provides guidance to task teams and program managers for identifying indicators of governance and service quality in targeted cash transfer programs. The paper identifies governance issues along the results chain of service delivery and suggests policy and performance indicators for assessing program inputs ─ human resources, financing and resource management; and program activities ─ operational procedures, MIS and control. It also suggests indicators of organizational performance and the quality of outputs, including demand-side accountability mechanisms. JEL Classification: H11, H83, I38 Keywords: governance, service delivery, safety nets, cash transfers, service quality indicator, performance monitoring 2    Table of Contents    Introduction .......................................................................................................................... 4  1.  Governance and social protection .................................................................................. 5  2.1 Governance in service delivery ................................................................................................. 5  2.2 Tackling error, fraud and corruption ......................................................................................... 6  2.3 International development approach to governance measurement ....................................... 7  2.  Framework for Measuring Governance and Service Delivery in Cash�based Safety Nets . 8  3.1 Results Chain in Cash Transfer Programs .................................................................................. 8  3.2 Where Does Governance Come into Play? ............................................................................. 10  3.3 Measuring Governance and Quality of Service Delivery in Safety Nets Programs ................. 11  3.4 How is governance different in cash transfers than health and education? .......................... 12  3.  Identifying Governance and Service Quality Indicators for Safety Nets Programs ......... 13  4.1 Human resources .................................................................................................................... 13  4.2 Financing and Resource Management .................................................................................... 16  4.3 Operational procedures, information management and control mechanisms ...................... 19  4.4 Organizational Performance of the Delivery of Cash Transfers .............................................. 23  4.5 Quality of Social Assistance Service Delivery .......................................................................... 24  4.6 Demand�side accountability in service delivery ...................................................................... 27  4.7 Data Collection ........................................................................................................................ 31  4.  Conclusions .................................................................................................................. 32  5.  References ................................................................................................................... 33  Annexes .............................................................................................................................. 39  Annex 1. Overview of governance and social protection assessment tools ................................. 39  Annex 2. List of indicators related to social protection issues ...................................................... 51    3    Introduction    Making services work for poor people involves a number of challenges including strengthening the governance of public sector institutions. Specifically, this requires setting the right set of incentives and accountabilities to affect the way provider organizations, their managers, and staff behave, as well as the quality and efficiency with which they deliver services. Thus, how providers are selected, paid, monitored, and held accountable for their performance has gained increased relevance in the service delivery reform debate. In recent years the international development community has made efforts to define and measure governance in public service provision. It has sought to generate and systematize empirical evidence on the links between service delivery quality and governance. Although this work has focused mainly on health and education, governance and service quality are also relevant in social protection. For example, efforts to strengthen social assistance targeting, administration and service quality can benefit from a conceptual framework to help diagnose, monitor and evaluate the performance of institutions in charge of social assistance services, including their incentives and accountabilities, as well as service quality and efficiency. As a first step towards measuring governance in social protection, this paper develops a framework to assess organizational performance in the delivery of social protection services with a focus on cash-based social safety nets (social safety nets). In particular, this paper aims to identify a set of governance and service quality indicators that can be applied to targeted safety net programs. The paper is structured as follows. The first section summarizes existing literature on governance and service delivery relevant for social protection. The second section develops a conceptual framework to measure governance and service quality in the delivery of cash benefits. The next section discusses a potential list of governance and service quality indicators for social safety net programs as well as data collection options. Finally, the last section presents the conclusions.   4    1. Governance and social protection    Governance has been the subject of multiple definitions and applications. There is a vast literature on governance applied to a variety of fields including corporate management, public sector management and international development. More recently, the governance dimension has entered the analytical work and policy debate on human development, particularly in relation to education and health care service delivery and to a lesser extent, social protection. As a result, the literature on assessing the performance of service delivery in social protection is still limited. This section aims to start filling this gap by drawing from lessons learned more broadly on human development and public sector management, and apply them to the delivery of social safety nets. This section briefly touches upon three areas of governance relevant for social protection in general: (i) governance in service delivery; (ii) tackling error, fraud and corruption; and (iii) governance measurement in international development. 2.1 Governance in service delivery  Governance can be defined as “the set of incentives and accountabilities that affect the way provider organizations, their managers and staffs behave, as well as the quality and efficiency with which they deliver services� (Fiszbein, et al., 2011). Recognizing that poor governance and service quality in the delivery of social services is as relevant to achieve human development outcomes as funding has led policymakers and academics to better understand and seek to influence the performance of public service providers (World Bank, 2003a; Brinkerhoff and Bossert, 2008, Savedoff, 2009; Lewis and Pettersson, 2009a and 2009b; Fiszbein, et. al., 2011). In particular, Fiszbein, et al. propose a framework for the selection and measurement of governance and service delivery indicators for health and education. This framework distinguishes between two measurable aspects of governance: policies and performance. First, the set of policies, or rules of the game, that influence the ways that service providers function. These are the incentive and accountability arrangements that frame service provision. Second, governance materializes as performance – the actual practices and behaviors of providers resulting from policies. Governance performance is measured by the extent to which the governance policies work in practice. Governance policies and performance in turn influence the quality of service delivery. 5    The framework includes five areas in which governance issues play a critical role in health and education: human resources; financing and resource management; procurement and management of critical inputs (textbooks and pharmaceuticals); information; and provider entry. 2.2 Tackling error, fraud and corruption  Governance in social protection has been addressed primarily from the operational and institutional perspective of tackling error, fraud and corruption (World Bank, 2007, and Stolk and Tesiluc, 2010). This approach categorizes governance risks related to the misdirection of public funds into three types of problems: error, fraud and corruption (EFC). Error refers to unintentional mistakes of various natures, while fraud and corruption refer to intentional attempts to exploit the system by benefit claimants and program staff respectively. Safety net programs have inherent design features such as complex eligibility requirements and changing eligibility over time which pose a particular challenge for controlling error, fraud and corruption (EFC). Eligibility is often based on the economic position of the household, which is difficult to assess and hard to verify. Moreover, eligibility may change over time as households’ poverty status varies. Evidence from developed countries suggests a four-pronged strategy to control EFC: prevention, detection, deterrence and measurement (Stolk and Tesliuc, 2009). A number of actions considered in the strategy can be mapped to the governance framework discussed above. In particular, the EFC strategy emphasizes the need for developing appropriate administrative procedures or rules, including a clear delineation of procedures in the processing and verification of benefit claims and clarity in the allocation of institutional responsibilities across the different levels of government or between government and civil society. Additionally, it suggests the adequate use of IT systems and staff training to help in reducing human error and improve performance in following procedures. Moreover, the generation of intelligence to detect cases of EFC through data matching and telephone or online grievances and complaints mechanisms, as well as, established review processes of benefit claims, and sanctions are part of the incentive arrangements to deter fraud and corruption. Finally, information campaigns aimed at raising awareness of the rights and obligations under the rules of the social safety nets programs can create social control and accountability mechanism over program implementation. These recommendations coincide with the findings of a review of conditional cash transfer (CCT) programs in Latin American countries which pointed to the need for a better definition of institutional responsibilities; adequate implementation of management information systems; and thorough control and accountability systems (World Bank, 2007). 6    2.3 International development approach to governance measurement   Governance has gained importance in the international arena and has evolved from a narrow focus on public sector management to a broader concern for interconnected issues, including the role of formal and informal institutions, security, human rights, and corruption. As a result, bilateral and multilateral development agencies, among other organizations, have developed a number of governance assessment tools (UNDP, 2007 and OECD, 2008 and 2009). These tools broadly serve three sets of objectives: (i) strategic decisions of donors regarding aid allocation and risk management; (ii) donor-client policy dialogue and joint decisions on aid allocation, performance assessment and operational set-up; and (iii) the promotion and design of domestic governance reform processes. Most of these instruments focus on political systems and public administration. Some address social governance issues including pro-poor spending and access to, and effectiveness of, service delivery. Annex 1 presents an overview of governance assessment tools by focus areas. It identifies the tools addressing social protection issues, and distinguishes those dealing with broader public sector management topics such as financial management, human resource management (HRM), transparency, accountability and corruption that can be adapted to analyze social protection performance. Similarly, there are a few initiatives that systematize information on the performance of social protection systems2. Although these are not explicitly used as governance assessment tools, they include some indicators that may be useful for measuring service quality. A number of these initiatives are discussed below. Annex 2 lists all of the indicators in the governance assessment tools related to social protection.  International Labor Organization Social Security Database offers information on compulsory cash benefit programs (e.g. sickness, maternity, old-age, disability, family allowance, work injury and unemployment) for 124 countries. It generates a statistical database on coverage, contribution rates, expenditures, legal framework, means-tested conditions for benefits, benefit size and duration, and waiting period.  Social Protection Expenditure and Performance Reviews (SPERs) are country- level analysis of social protection systems, expenditures, financing and performance in terms of coverage, distributional aspects and administrative efficiency.  European Commission, Mutual Information System on Social Protection provides comparable and regularly updated information on social protection systems legislation in 31 European countries. It includes charts and descriptions of                                                              2  Some of these initiatives originate in Europe´s standard�setting and coordination efforts of social security  systems.   7    social protection systems organization, as well as comparative tables of statutory regulations of core areas (health care, sickness, maternity/paternity, invalidity, old- age, survivors, work injuries and occupational diseases, family allowances, unemployment benefits, minimum protection, long-term care and financing)3.  Eurostat – European System of Integrated Social Protection Statistics (ESSPROS) is a database using a common framework which enables international comparison of the administrative national data on social protection benefits and financing across European countries. In sum, governance lenses have not been systematically applied to social protection. Nonetheless, there are valuable inputs and lessons from different areas in the literature4. The next section aims at contributing to the knowledge of social protection governance by proposing a measurement framework for cash-based social safety net programs. 2. A Framework for Measuring Governance and Service Delivery  in Cash­based Safety Nets  Social protection encompasses a range of different types of policies and programs, including social insurance, labor markets, and social safety nets. Although they share a number of common features in their administration, differences in benefits and eligibility criteria exist. Hence, social protection policy instruments involve different operational procedures and corresponding organizational challenges and require tailored governance approaches. As a starting point, the framework developed in this section will focus on cash benefits. In the future, this framework could be further adapted to other types of social safety nets (e.g. social services) and social protection programs. 3.1 Results Chain in Cash Transfer Programs  Results chains depict the theory of change behind a program. They describe a logical, plausible pathway of how a sequence of input, activities and outputs ─ which an implementing agency is directly responsible for─ interacts with behavioral processes and external factors to achieve a particular outcome. Results chains help associate what is being done with intended and actual outcomes. They therefore help trace and track program                                                              3  A similar initiative by the Council of Europe is the Mutual Information System on Social Protection of the  Council of Europe (MISSCEO). Following the same structure as MISSOC, it produces comparatives tables on  social protection systems legislation in 15 states: Albania, Armenia, Azerbaijan, Bosnia and Herzegovina,  Croatia, Georgia, Moldova, the Russian Federation, Serbia, “the former Yugoslav Republic of Macedonia�,  Turkey and Ukraine as well as Australia, Canada and New Zealand.  4  Other areas in the literature worth exploring in future reviews include research in management – performance management and organizational effectiveness, and services marketing.   8    implementation, link specific program outputs to specific inputs and evaluate whether the program is achieving its intended results. Figure 1 illustrates a simplified traditional results chain for a cash assistance program. In this case, the links of the results chain involve the following: - Inputs: include staff from the ministry, institution and/or local government responsible for operating the program; the budget allocated to the program; and the central and local facilities where the program operates. - Activities: comprise all operational procedures required for program implementation, including outreach and application, beneficiaries’ selection and enrollment, payment delivery; and periodic eligibility reassessment. - Outputs: are the cash transfers delivered. - Outcomes: are the ultimate program objectives. For social safety nets programs, the main outcomes are improved household consumption in the short term, and poverty reduction in the long term. They are influenced not only by program outputs but also by other factors like the economy and individual behaviors (foregone income, crowding-out of other transfers, etc.). Figure 1: Traditional Results Chain for a Cash Delivery Social Assistance Program   Source: Author 9    3.2 Where Does Governance Come into Play?  Governance enters the results chain as an enabling cross-cutting factor first through the availability and performance of inputs, as well as their transformation into outputs through program activities (see Figure 2). Secondly, governance policies and performance influence the quality of service delivery – defined here as program accessibility and the quality of the client and service provider interaction and transactions. This explicit emphasis on the service quality dimension of outputs represents a “missing middle� in the traditional results chain. Service delivery is a function of supply and demand side factors5. On the supply side, governance policies and performance shape the way organizational providers6 materialize their mandate and manage program inputs and activities to generate outputs and reach the target population through frontline staff.7 Service delivery is also influenced by demand- side factors. Through direct influence on service providers and indirect pressure on politicians and policymakers8, citizens and beneficiaries can demand changes in the delivery of a program. Governance policies and performance on access to program information, choice, participation, and independent assessment also have an effect on the demand-side accountability of service delivery.                                                              5  There are also external factors such as technology that influences service quality.  6  Organizational providers are public, private nonprofit, and private for�profit entities that actually provide  services. In the case of social assistance services, most provider organizations will be within the public  sector, for example, a ministry, department, agency or bureau.  7  Following the service provision framework of the 2004 World Development Report (WDR), on the supply  side there are two accountability relationships: the compact relationship between organizational providers  and the state, and the management relationship between frontline professionals and the organizational  provider. Policymakers make resources available and delegate powers and responsibility for collective  objectives to service providers. In turn, organizational providers assign frontline workers with tasks and  delineated areas of responsibility, furnishing them with the resources to act.  8  “Clients may seek to hold service providers accountable for performance in two ways. Client power  connecting clients and providers is the direct, “short� route of accountability. When such client power is  weak or not possible to use, clients must use voice and politics in their role as citizens to hold politicians  accountable—and politician/policymakers must in turn use the compact to do the same with providers. The  combination of the two is the roundabout, “long� route of accountability.� (World Bank, 2003a)  10    Figure 2: Governance in Social Assistance Programs Source: Author 3.3 Measuring Governance and Quality of Service Delivery in Safety Nets  Programs  Governance can be measured along two dimensions: policies and performance. Governance policies are the rules of the game – both formal/de jure and informal/de facto – that set the incentive and accountability arrangements under which program stakeholders operate. These can involve national or local level policies, and laws and regulations applicable to the public sector at large, such as budget or civil-service laws. They can also entail program specific rules such as operational manuals, administrative procedures or job descriptions. The distinction is important as they imply different strategies and stakeholders for the policy dialogue. Governance performance refers to how the governance policies work in practice - that is, the organizational behavior and practices resulting from the existence and application of a set of rules. Staff absenteeism is an example. It reflects the application of governance policies in human resource management involving a meritocratic civil service, adequate compensation scheme, performance evaluation, and sanctions and rewards system. 11    3.4 How is governance different in cash transfers than health and  education?  Cash transfers are transaction-intensive programs that require frequent client-provider contact. In contrast to classroom teaching or curative health, they involve overall better defined and comparatively easier to systematize activities. Examples of transactions include, but are not exclusive to: (i) beneficiary application and selection to the program; (ii) benefit delivery; (iii) grievance and redress mechanisms. Figure 3 takes a closer look at program activities or operational processes involved in delivering cash transfers. In addition to the implementation steps, there are two crucial cross-cutting components for program operation: management information system (MIS) and control mechanisms. Responsibilities along the implementation process are often shared across organizations and government levels. The program results chain and the more detailed operational procedures can help identify critical areas for assessing governance in safety nets programs. They refer to program inputs ─ human resources, financing and resource management, and program activities ─ operational procedures, MIS and control mechanisms. These elements should be considered in relation to the organizational performance (intended versus actual) in cash transfer delivery –outputs– and the quality of social assistance services, including demand-side accountability mechanisms in safety nets. Figure 3: Social Assistance Programs Activities IMPLEMENTATION PROCESSESES  COVERAGE  BENEFICIARY  STRATEGY AND  APPLICANTS’  SELECTION &  BENEFICIARY  PAYMENT  RECERTIFICATION  PROMOTION  REGISTRATION  VERIFICATION  ENROLLMENT  MANAGEMENT INFORMATION SYSTEMS AND CONTROL MECHANISMS  Source: Author   12    3. Identifying Governance and Service Quality Indicators for  Safety Nets Programs   The selection of any particular set of indicators has underlying normative assumptions. There is some evidence from OECD countries on which governance interventions are cost- effective measures for reducing EFC (Stolk and Tesiluc, 2010). However, not all governance interventions have a proven impact on organizational performance including service quality, or program outcomes. These indicators should be treated as hypothesis to be tested rather than evidence to follow (Savedoff, 2011). Existing tools for assessing governance in social protection systems include a number of indicators to draw from. In particular, human resources and financial management governance indicators have been extensively used in the past. Available sources could be used to identify relevant indicators and evaluation criteria. However, most tools are geared towards general public sector performance measurement. Thus, specific data collection for indicators tailored for safety nets will be required. As explained above, governance and service quality assessment in safety nets programs involves measuring indicators in six areas: a. Human resources b. Financing and resource management c. Operational procedures, information management and control mechanisms d. Organizational performance in cash transfers delivery e. Service quality9 f. Demand-side dimension of service delivery 4.1 Human resources  Social assistance programs involve a number of interactions between frontline staff and beneficiaries, including program outreach and application process, eligibility assessment, applicants’ information verification, beneficiary enrollment, and benefits payment. The rules of the game for hiring, compensating, training, evaluating and rewarding program staff will hence shape their incentives, which in turn, may influence their work performance in providing efficient and client-oriented services10.                                                              9  Although service quality is one of the dimensions of organizational performance, it will be addressed  separately to highlight its importance.  10  The impact of human resources management on organizational performance has been studied mostly in  the private firms’ context. There is no consensus on the robustness of the relationship between HR systems  and firms’ performance, and more work is needed on measuring organizational performance (Rogers and  Wright, 1998). The links between HRM governance policies and public organizations performance are still  13    Table 1 shows the main issues in assessing governance in human resource management (HRM) and the corresponding policy and performance indicators. Performance measurements may be associated to more than one governance policy in human resources or in other areas. For example, staff absenteeism may be associated with employees’ motivation and accountability framework determined by retention and compensation policies, as well as with performance evaluation and control mechanisms. As indicated next to each indicator, some are included in existing assessment tools such as the WB Actionable Governance Indicator Initiative (AGI), the Country Performance and Institutional Assessment (CPIA), or the US Anti-Corruption Assessment Framework (ACF). These indicators usually refer to the overall civil service, but can be adapted to gauge human resources performance in a particular sector or program. Annex 1 presents detailed information on the assessment tools, including references to consult the rationale and evaluation criteria for each indicator. Interestingly, the main focus of available indicators is on hiring, promotion, and compensation practices, and less so on training and delineation of responsibilities. EFC literature, however, recognizes the role of the latter as part of the prevention initiatives. Thus, for social safety nets programs is important to complement existing HR indicators with measurements of staff knowledge and application of procedures and responsibilities.                                                                                                                                                                                       being studied, thus indicators discussed in this section should be considered as part of hypotheses to be  tested.  14    Table 1. Human Resources Management Governance Indicators in Safety Nets Programs Issue Policy Indicators Performance Indicators Hiring, retention and - Average number of qualified (long-listed) candidates per advertised - Quarterly staff turnover rates [AGI] promotion program openings [AGI] - Monthly average staff absentee rate [AGI] - Average 3-year turnover rates among recent recruits by types of position (e.g. Titles, urban vs. urban locations) [AGI] - % of program vacancies filled through advertised, competitive procedures - Ratio of average total remuneration for staff in the highest rank to [AGI] average total remuneration for staff in entry level position [AGI] - Degree in which hiring and promotion are based on merit and - Perception of political influence in staff promotion, transfer and/or performance, and ethical standards prevail [CPIA] appointment [GAC] - Existence of a legislative framework for the civil service regulating recruitment, job security and retention [ACF] - % of staff for whom annual performance evaluations were completed on - Existence of clear rules regarding tenure [ACF] time [AGI] Duties/Responsibilities - Existence and clarity of job descriptions/responsibilities and administrative procedures - % of staff performance evaluations falling in each rating category [AGI] - Existence of conduct code or equivalent [ACF] - % of staff receiving the lowest performance rating in two successive - Existence job performance standards Compensation -Average staff remuneration as a % of average economic sector wages years who have left the program within the following year [AGI] [AGI] - Degree of correspondence between program staff skills and job profiles - Ratios of average staff to private sector total remuneration by title [AGI, ROSC] (e.g. % of staff whose skills match their job profile) - Degree of flexibility in paying more attractive wages in hard to fill - Level of staff proficiency and compliance with administrative procedures positions (e.g. rural areas, technical specialists) [CPIA] - Extent to which pay and benefit levels do not deter talented people - Degree of correspondence between standard and actual staff workloads from entering the public sector [CPIA] - Perceived effectiveness of the training opportunities offered by - Level of adequacy of civil service compensation to sustain an appropriate livelihood [ACF] institutions at improving work performance [GAC] Training -Regularity of training for civil servants on rules and procedures - Degree of understaffing to perform required tasks [SGACA] governing recruitment, hiring and promotion [ACF] - Existence, relevance and participation on training aimed at improving - Share of “ghost� employees (those who receive a payment and are not work performance legitimate program personnel) Evaluation -Existence, type and application of performance evaluation systems Rewards/Sanctions -Existence, type and application of rewards and sanctions tied to job - Extent of public employment use to diffuse social tensions at the expense performance of efficiency and effectiveness [SAF] Note: AGI-WB Actionable Governance Indicators; CPIA-WB Country Policy and Institutional Assessment; ACF- USA Anti-Corruption Assessment Framework, ROSC- IMF Fiscal Module Report on Observance of Standards and Codes; GAC- WBI Governance and Anti-Corruption Country Survey; SGACA - Netherlands, Strategic Governance and Corruption Analysis; SAF- Netherlands, Stability Assessment Framework. 15    4.2 Financing and Resource Management   As in any other development intervention, financing and resource management is an enabling factor in program implementation. The rules concerning budget planning and execution, resource allocation across agencies and government levels involved in program implementation, and budget reporting and monitoring matter for organizational performance. In particular, policy and performance indicators in this area are very relevant as social safety nets may not have the same legal status as health, education or pension systems. As a result, their funding availability may be subject to changes in government. In this regard it is central to identify the legal and administrative mechanisms that ensure budget availability for the program life cycle. In many countries sub-national governments have a significant role in financing social safety nets benefits. Incentives and accountability mechanisms for local governments to deliver these programs are critical since these tend to lack influential constituencies who can pressure governments to pay benefits. As shown in Annex 1, financial management is included in a number of existing governance assessment tools. In particular, PEFA Public Financial Management Performance Framework is one of the most comprehensive instruments. It includes a comprehensive list of high level indicators, disaggregated assessment dimensions and a set of evaluation criteria for appraising the operational performance of key elements of financial systems. Although their measurement focuses on central governments, legislatures and external audit institutions, a number of PEFA indicators can be adapted to assess sectoral or program level financial management. Data collection and additional indicators specific for social safety nets can be obtained from other tools such as Public Expenditure Reviews (PERs) and Public Expenditure Tracking Services (PETS). Table 2 presents the main indicators for assessing financing and resources management governance. Many policies and rules in financial management are central government-wide regulations, although their application (policy performance) may vary across institutions. In a few cases, rules and procedures may be sub-national government or program specific. 16    Table 2. Financing and Resource Management Governance Indicators in Social Assistance Programs Issue Policy dimension Performance dimension Budget - Existence and adherence to a fixed budget calendar and approval preparation process process - Nature of involvement in the budget preparation by the ministry, - Executed social assistance budget as a percentage of department or agency in charge of social assistance approved budget (aggregate and by budget line) at the - Existence of a clear and comprehensive guidance for the preparation of central and sub-national government levels 11 budget submissions (budget circular or equivalent) - Resources received as a percentage of approved transfers - Timely budget approval by the legislature from the central to sub-national governments for social Policy-based - Existence of social protection sector strategy with multi-year costing of assistance programs* budgeting recurrent and investment expenditure - Timely availability of approved resources in cash and in - Availability and monitoring of social assistance program multi-year kind transferred to frontline offices coverage targets* - Share of local governments that publish quarterly reports Predictability - Existence and use of rules-based systems for the distribution of of budget execution* and control in transfers from the central to sub-national governments for social - Frequency and extent of wage (and travel reimbursement) budget assistance programs arrears for frontline office staff (those with direct contact execution - Degree of timeliness and reliability of information on transfers from the with clients) central to sub-national governments - Frequency and extent of benefit payments arrears - Regularity of tracking of resources received in cash and in kind by - Effectiveness of payroll controls frontline offices across the country, irrespective of which level of - Effectiveness of internal controls for non-salary government is responsible for the operation and funding of those units expenditures - Extent to which cash flows for social assistance programs are - Frequency and scope/nature of audits performed forecasted and updated - Evidence of effective and timely follow up on internal and - Frequency and transparency of adjustments to social assistance external audit recommendations program budget allocations - Timely publication on the internet of key social assistance - Existence of protection mechanisms of social assistance expenditure in expenditure information at the national and sub-national case of in-year adjustments to budget allocations government levels * -Existence of payroll controls (reconciliation of personnel records and - Availability on internet of a summary of program budget payroll; internal controls and audits) written in clear, simple language [ROSC] - Quality and transparency of procurement regulatory framework and application (for equipment purchasing, selection of payment providers, etc) - Existence of internal controls for non-salary expenditures (expenditure                                                              11  The ministries, departments and agencies concerned for the purpose of this indicator are those which are directly charged with responsibility for  implementing the budget in line with sector policies and which directly receive funds or authorization to spend from the Ministry of Finance. Department and  agencies that report and receive budgetary funds through a parent ministry should not be considered in the assessment.  17    Table 2. Financing and Resource Management Governance Indicators in Social Assistance Programs Issue Policy dimension Performance dimension commitment controls and rules for processing and recording transactions) - Coverage, independence and quality of internal audit function in social assistance program - Existence of systematic procedures for program management response to internal audit findings Accounting, - Existence of routine data collection or accounting systems that provide recording and reliable information on all types of resources received in cash and in kind reporting by frontline offices across the country, irrespective of which level of government is responsible for the operation and funding of those units. - Regularity of report compilation on resources received in cash and in kind by frontline offices across the country - Regularity of bank reconciliations (or with other payment agents – e.g. post office, etc.) - Regularity of reconciliation and clearance of suspense accounts and advances - Quality and frequency standards of regular budget reports and annual financial statements External - Existence and adequacy of auditing policies scrutiny and - Existence of systematic procedures to follow up on audit audit recommendations Transparency - Existence and use of a robust classification for monitoring and tracking social assistance spending - Existence and comparability of social assistance expenditure tracking at central and sub-national levels of government - Existence and thoroughness of policies for public access to key social assistance expenditure information (including transfer distribution rules from the central to sub-national governments) at the national and sub- national government levels Note: ROSC- IMF Fiscal Module Report on Observance of Standards and Codes; *Indicators not included in reviewed existing assessment tools. 18    4.3 Operational procedures, information management and control  mechanisms  Social assistance programs require well-defined operational procedures, management information systems and control mechanisms to ensure that the right people get the right benefit at the right time. As depicted in Figure 3, a series of operational processes must be in place to identify and enroll beneficiaries, deliver program benefits to them, eventually reassess their eligibility as well as manage their complaints in case any of the mentioned- transactions went wrong. The rules concerning the definition of clear-cut operating procedures and institutional responsibilities could make a substantial difference in reducing discretionary decisions on eligibility and benefit levels, giving certainty to applicants as to what to expect, and in case things go wrong, how to fix them. Table 3 includes the main indicators in assessing governance in program operation, information management and control. On one hand, policy indicators in this area assess the existence and pertinence of standard procedures and control mechanisms along operational processes. On the other hand, performance indicators include compliance and efficiency measures associated with established procedures and controls. Most policies and rules are likely to be program-specific, as opposed to government-wide regulations. As discussed in HRM or financial management indicators, performance measurements may be associated to more than one governance policy in program operation. Only a few governance assessment tools address social protection issues and they mainly focus on pensions. Hence, most indicators in Table 3 are not included in existing assessment tools. Nonetheless, many can be found among results matrices of safety nets projects. Program operations are often sequenced in a systematic manner starting off with beneficiary outreach, application, selection and enrollment, passing through benefit delivery until exit and graduation. Outreach activities inform potential clients of program benefits, eligibility criteria and application process. These activities should be aligned with a program coverage strategy based on the characteristics of the target population and the intended number of beneficiaries for specific regions in a given period. For example, the target population for a program may be female-headed households in selected rural communities. Alternatively, the target population for a larger urban program may be poor households nationwide. The latter case may warrant a mass media promotion campaign and a program coverage expansion plan spanning a few years. The former would require community outreach activities in selected rural areas and coverage targets may be met in a shorter period. Thus, a governance focus of this operational step would analyze the existence of an annual and medium term program coverage strategy and the extent to which this strategy and outreach activities are adequate to reach the target population given budget and implementation constraints. 19    Application, beneficiary selection and enrollment follow. In particular, the application process, the choice of beneficiary selection methods and their implementation matter for program targeting results, and presumably, for poverty outcomes. To what extent targeting methods are appropriate given country circumstances and are applied according to well- defined and sound administrative procedures are relevant governance issues. Moreover, it is important to consider applicant´s transaction costs in terms of number of visits, time spent and paperwork required for program application and enrollment. The more time and resources these spend doing paperwork, the higher the opportunity costs (i.e. work absence), and the less likely are to participate in program activities. Targeting indicators such as the share of transfers received by the poor are also discussed in Table 4 . Payment of benefits should be accurate, secure, reliable, transparent, affordable and hassle-free, and most importantly, timely. Rules defining payment sites and recipient lists, calendar and frequency, process and amounts as well as payment agents and instruments are important aspects of program governance. EFC prevention recommendations include ensuring that benefit payment is as secure as possible, ideally by making payment directly to recipient accounts – the fewer hands physical money travel through, the better (Stolk and Tesiluc, 2010). Another aspect worth considering are the rules regarding the adjustment of benefit levels over time, whether it is constant or inflation-indexed to maintain its real value. Exit and graduation considerations may set rules on benefit duration or eligibility recertification. If beneficiary selection includes proxy-means testing, recertification may be an opportunity to update the scoring system. Recertification may thus improve program targeting performance, but it also involves additional costs (Coady et. al., 2004). Governance analysis would consider to what extent clear and transparent rules regarding duration of benefits, recertification frequency and processes exist. Information management and control mechanisms are cross-cutting elements vital for the effective and efficient processing of benefit claims. Adequate MIS allow a systematic registry of beneficiaries and benefits payment record-keeping, both indispensable for program management and minimizing human error in the processing of benefits. Moreover, in some OECD countries data submitted by claimants is cross-checked against other government databases, as a cost-effective way to detect EFC (Stolk and Tesiluc, 2010). The rules regarding which information is collected, who is responsible for collection, how it is systematized, kept and used can influence organizational performance. Finally, control mechanisms along the operational processes are also instrumental. These include a range of activities such as household information verification, database cross- checks, performance audits, and monitoring and evaluation. Some control tools involving independent assessments enable demand-side accountability and are included both in Tables 3 and Table 6. 20    Table 3. Governance Indicators in Program Operation and Control Mechanisms Issue Policy dimension Performance dimension Coverage - Existence and tracking frequency of annual and medium term strategy and coverage strategy promotion - Degree of alignment between promotion activities and - Consistency between coverage strategy and number and type of coverage strategy (including adequacy of dissemination beneficiaries (rural/urban, male/female, etc.) channels and materials) -Level of staff compliance with operational procedures and processing - Adequacy of public awareness activities on program rights and times obligations -Program application and enrollment transaction costs (e.g. number of Application, - Existence of sound poverty diagnosis to identify poor, visits, time spent, etc.) selection and vulnerable groups and those lacking services [CPIA] -Program recertification transaction costs enrollment - Type and adequacy (cost-effectiveness) of targeting method - Compliance with payment calendar (e.g. % of cash transfer payments choice (simple means test, proxy means test, community-based delivered according to payment calendar) targeting, categorical targeting, self-selection) - Compliance with eligibility reassessment calendar (e.g. % - Degree of objectivity and transparency of eligibility criteria recertification of program beneficiaries due to be recertified after - Degree of clarity and soundness (i.e. separation between established years of enrolment) assessment and payment functions) of operational procedures - Completeness and reliability of beneficiary registry and payment for program application, beneficiary selection and enrollment registry Payment - Existence of a benefit payment calendar - Average time taken to update in the MIS changes beneficiary - Type of payment method (bank deposit, debit card, cash, etc.) information (e.g. household composition (newborns or deaths) or - Level of reliability of payment method and process (direct address changes) payment to beneficiary through secure channel) -Percentage of cash transfers made automatically through MIS - Evolution of program benefits (constant, indexed, - Number of progress reports produced by the MIS, according to discretionary evolution, etc.) [SSD] agreed reporting standards Recertification - Clarity and transparency of rules concerning duration of - Degree of systematic use of control mechanisms (household benefits and frequency of eligibility reassessments information verification, database cross-checks, telephone hotlines, - Duration of benefits [SSD] etc.) - Frequency of eligibility reassessments - Degree of use of risk-based investigations - Degree of clarity and soundness of operational procedures for - Frequency of EFC monitoring program eligibility reassessment - Relevance, independence and quality of program evaluation Information - Existence of adequate beneficiary registry and payment - Frequency and scope/nature of performance/operational audits Management records performed - Extent of use of IT system to register and process benefit - Administration costs of social assistance program [SPEPR] claims, and cross-check data - Clarity of roles, responsibilities and procedures for data collection, systematization, quality control and reporting 21    Table 3. Governance Indicators in Program Operation and Control Mechanisms Issue Policy dimension Performance dimension Control - Type and adequacy of verification mechanisms of information mechanisms submitted in program application (database crosschecks, random house visits, etc.) - Adequacy of procedures to refer cases for investigation - Existence and type of monitoring mechanisms (spot checks, operational audits, third-party monitoring) of staff compliance with operational procedures - Adequacy of monitoring mechanisms for third party implementation (local government, payment agency, NGOs) - Existence and type of systematic activities for program M&E and management response to M&E findings - Existence of policies to ensure public access to M&E information at the national and sub-national government levels - Existence of systematic procedures to follow up on audit recommendations - Existence of policies to ensure public access to program beneficiary list Note: CPIA-WB, Country Policy and Institutional Assessment; SSD - ILO, Social Security Database; SPEPR – ILO, Social Protection Expenditure and Performance Reviews 22    4.4 Organizational Performance of the Delivery of Cash Transfers   Organizational performance in social safety nets can be measured along five dimensions: program relevance, coverage, poverty targeting, efficiency and service quality ─discussed separately in the next section. Table 4 includes summary measures of organizational performance in these areas. Most indicators belong to the output category, except for poverty targeting measures that may be considered a grey area between outputs and outcomes. Clear-cut outcomes such as increased consumption levels are not included here as the main focus is on the supply side. One way to assess program relevance is through the adequacy of benefit levels compared to a benchmark. The conceptual benchmark considered in Social Protection Public Expenditure Reviews for social safety nets is the gap between a household’s income and the poverty line. Coverage is a key performance dimension that gauge to what extent program magnitude is appropriate given the target population size. If coverage is low, the program is ineffective in reaching a meaningful proportion of vulnerable population. Coverage information could be presented by pertinent breakdowns such as rural-urban, regional breakdowns, gender, and so forth. It should be measured as a rate where the denominator is the total population deemed as vulnerable, for example, the total number of poor households. Commonly used poverty targeting measures include the share of total transfers received by the population falling within the bottom 40, 20, or 10 percent of the national income distribution, the share normalized, and the inclusion and exclusion errors. However, to what extent better targeting as measured by these indicators imply a large impact on poverty or a more cost-effective intervention is an empirical question yet to be settled12. Finally, a starting point to assess program efficiency may be administrative costs. They should be high enough to allow adequate program administration but low enough to be efficient. Cost information should be compared to international or regional benchmarks. Another useful measure of program efficiency is the proportion of program funds that are lost due to EFC13. Some of the indicators of organizational performance are included in social protection assessment tools such as the ILO Social Security Database (SSD) or the World Bank Social Protection Public Expenditure Reviews (social protection-PER).                                                              12  See Ravallion 2007 for a critical review of different targeting measures.  13  The following definitions are taken from Stolk and Tesiluc, 2010. Targeting errors may be due to program  design (when social assistance programs use imperfect poverty proxies to identify poor beneficiaries) and to  program implementation (when eligibility decisions and payments diverge from program rules). The later  component is due to EFC. Error refers to unintentional mistakes on behalf of benefit claimants or staff in the  benefit office. Fraud refers to intentional behavior on the part of the benefit claimant to defraud the benefit  system. Corruption is an intentional attempt by staff to exploit the social protection system.  23    Table 4. Organizational Performance Indicators Relevance - Adequacy of benefit levels with respect to poverty line [social protection-PER] Coverage - Coverage rate (number of program beneficiaries with respect to total vulnerable or target population disaggregated by rural-urban distribution, region, gender, formal/informal sector, and so on) [social protection-PER, SSD, SPEPR] � Horizontal distribution of benefits (proportion of cash benefits distributed by gender, formal/informal sector, and distribution to other groups identified as the most vulnerable) [SPEPR] Poverty Targeting14 - Targeting Differential (difference between program participation rate of the poor and the participation rate for the non-poor) Efficiency - % of transfers delivered on time and to the right recipient - Error, fraud and corruption rates (proportion of total transfers that are lost due to error, fraud or corruption) [social protection-PER] - Administration costs [social protection-PER, SPEPR] Note: social protection-PER- WB Social Protection Public Expenditure Review; SSD- ILO, Social Security Database; SPEPR- ILO, Social Protection Expenditure and Performance Reviews.   4.5 Quality of Social Assistance Service Delivery  Accessibility and quality in social services provision are generally recognized as important factors influencing social inclusion15. A number of OECD countries have developed service quality standards for organizations delivering public services. A few examples include the Charter Mark Standard in the UK, the National Standards for Disability Services in Australia or the Customer Service Standards of Rockville City Government in the US. Some of the quality dimensions emphasized in these initiatives that may be relevant for social assistance are the following. Table 5 includes alternative indicators in each dimension.  Service quality standard-setting and performance: existence and compliance with service and performance standards.  Service accessibility: extent to which social assistance services are physically, spatially and socio-culturally accessible to all individuals in the target population, regardless of their age, gender, ethnicity, language, disability condition, location, and so on.  Privacy, dignity and confidentiality: extent to which each beneficiary is treated with the same level of privacy, dignity and confidentiality.                                                              14  See Ravallion 2007 for a broader list and discussion on different targeting measures.  15  Measurement of service quality has been extensively researched in the private sector management and  marketing literature. Nonetheless, the conceptualization and operationalization of service quality as well as  the link between service quality, purchase intentions and business performance continue to be subject of  debate (Seth et al. 2005). In the public sector, research on the impact of service quality on human  development outcomes is more recent and has been focused primarily on health care and education  services.  24     Cultural awareness: delivery of non-discriminatory services which are sensitive to social and cultural values of the individuals, their families and community.  Complaints and disputes: existence of publicized and easy-to-use complaints procedure, including a commitment to deal and solve complaints within a time limit.  Exit and reentry: extent to which individuals are assisted to plan for the exit from social safety nets and assured that reentry is available, if required.  Service integration: development of links with other social assistance service providers at local, state and national level to ensure access to complementary services. Table 5. Service Quality Indicators Service quality standard - Existence of precise and measurable service standards setting and performance - Regular performance monitoring and benchmarking with similar services - Clients and staff participation in setting and reviewing service standards - Dissemination to clients and potential clients of service standards and performance - Assessment, recording and analysis of clients’ satisfaction levels and action taken to deal with any problems Service accessibility Accessibility policies - Existence and application of equal opportunities and disability policies, procedures and training for service delivery - Extent to which target population is defined, their needs regularly identified and services are planned and delivered to meet those needs Affordability - Availability and use of different means to reduce the cost of program information and application (e.g. mobile offices, toll-free phone numbers, etc.) - Travel time and transportation costs to facilities - Application and enrollment costs - Average time required for program application and enrollment - Individual’s perception of ease of service accessibility and paperwork required Physical, spatial and functional accessibility - Convenience of location and access (% of facilities in close proximity to public transport, with clear signage, level access, that are compliant with accessibility regulations for people with physical disabilities) -Convenience of facilities (e.g. % of facilities with adequate waiting areas which accommodate children, are properly maintained, sanitary, completely operational, fully stocked and supplied ) - Extent of use of technology to provide information and access to services (if appropriate) - Posting and observance of operation hours (e.g. % of facilities with posted operation hours, % of facilities with staffed information/registration counters during business hours at random visits) - Convenience of operation hours (% of beneficiaries who consider operation hours convenient) Socio-cultural accessibility - Availability of information materials in different languages and formats -Ratio of the number of beneficiaries who have difficulties communicating in the official language and the number of trained interpreters or bilingual staff - Availability of information and tailored service access for individuals with special needs (people with visual or hearing difficulties, those who have difficulties understanding or reading the official language, and people with 25    learning difficulties) - Individual’s perception of service provision in a manner sensitive to the age, sex, ethnic, and linguistic background of each person - Social distance Privacy, dignity and - Existence, implementation and monitoring of policies and procedures on confidentiality protecting individuals’ privacy, dignity and confidentiality - Existence, compliance and monitoring of ‘customer care’ policies - % of staff – client interaction in which staff performed satisfactorily (i.e. listen to client’s request/question and provide complete, knowledgeable, accurate, precise information regarding the inquiry) - Individual’s perception of respectful, honest and professional treatment - Individuals’ perception of discrimination - Extent to which complaints of abusive treatment are monitored and addressed - Number of complaints of abusive treatment (trend over time) Cultural awareness - Extent to which staff have knowledge of the social and cultural groups represented in the target population and an understanding of those social and historical factors relevant to their current circumstances - Extent to which service delivery considers the needs and unique characteristics of social and cultural groups represented in the target population and involves these groups in the planning and implementation of services - Extent to which specific cross-cultural staff training, and involvement of representatives of relevant cultural groups in service delivery planning is promoted - Extent to which social assistance employs staff or develops links with other service providers/organizations with relevant experience in the provision of services to the specific social and cultural groups represented in the target population - Extent to which issues related to social and cultural prejudice among staff are monitored and addressed - Extent to which representation of social and cultural groups amongst caseloads are monitored and reviewed - Individuals’ perception of service delivery in a manner which is sensitive to their social and cultural beliefs, values and cultural practices Complaints and disputes - Existence of staff guidance and training to handle complaints and empowerment to put things right - Extent to which the number and type of formal and informal complaints, compliments and suggestions received, and how quickly they are dealt with are recorded, analyzed and posted in the internet - % of local offices that publicized regularly information on the number and type of complaints, compliments and suggestions received, along with the improvements made as a result - Clients and staff assessment of complaints procedure - Regular review and improvement of complaints procedure, taking into account the views of clients and staff - Number and type of formal and informal complaints, compliments and suggestions received - Average time to address a complaint or dispute - Percentage of unresolved complaints or dispute Exit and reentry - Existence of documented individual exit plan which is begun during entry to social assistance to ensure ongoing continuity of other relevant social services once the client has exited from social assistance - Extent to which the exit plan is reviewed in collaboration with the client - Extent to which clients and other service providers and agencies involved in follow-up (if applicable) are aware of how to gain entry to social assistance at a later date, if needed - Monitoring of reasons for exiting social assistance services 26    - Percentage of clients who exit social assistance services by reason - Percentage of clients who exit social assistance services and apply for reentry within a certain period of time - Percentage of clients who exit and reentry social assistance Service integration - Existence of arrangements to provide coordinated services and formal processes to promote inter-agency collaboration - Extent to which separately funded programs work collaboratively to achieve a single integrated social assistance system - Percentage of clients who enter other social services through social assistance - Percentage of clients who benefit from coordinated services In addition, there are two additional dimensions that are gaining significance in service quality assessment and may be relevant for some social assistance programs. First, there is the focus on individual needs that shifts away from the traditional ‘top-down’ service provision to service delivery designed to meet individual needs and personal goals. Second, there is an emphasis on decision making and choice that provides individuals a range of service alternatives and information to assist in the selection of the most appropriate option in the setting most empowering for the client. 4.6 Demand­side accountability in service delivery  Demand-side accountability refers to the influence that individuals can have in their dual role as direct service clients –i.e. client power - as well as citizens that participate in political processes that define collective objectives –i.e. political voice (World Bank, 2003a). Client power may be enhanced by choice in service delivery (e.g. vouchers in education), better information and direct participation. Political voice is exercised through voting, advocacy, and representation in political bodies, often facilitated by public disclosure of key policy information. Governance policies and performance shape the mechanisms, both formal and informal, through which client power, and to some extent, voice, are fostered and exercised. Following Ringold, et. al. (2010), Table 6 includes demand-side accountability measures for social assistance programs in the following areas.  Transparency: access to key policy information (budgetary, rules of operation, performance, and so on).  Participation: client involvement in program decision-making, implementation and monitoring, and capacity building activities for stakeholder participation.  Grievance redress mechanisms: existence of procedures to handle complaints and mechanisms for their follow up and resolution. 27     Independent assessments16: tools and arrangements for independent verification of access to, quality and effectiveness of services through audits, service delivery and other surveys, monitoring and evaluation. Some of the governance policies and performance in these areas have been considered in preceding sections and can be discussed interchangeably. For example, the existence of independent assessments was considered as part of control mechanisms. Governance measures of transparency were included in the areas of financial management and operational procedures.. In addition, grievance redress mechanisms were addressed as one of the service quality dimensions. For the sake of a comprehensive discussion on demand- side indicators, a number of examples mentioned previously will be repeated here. Since evidence on how to foster client power and to what extent it has an impact on development outcomes is work in progress, listed indicators should be treated as hypothesis to be tested.                                                              16  This category combines the third party monitoring and independent assessment areas.  28    Table 6: Demand-side Service Quality Indicators in Social Assistance Programs Issue Policy dimension Performance dimension Transparency - Existence and thoroughness of policies for public access to - Type and adequacy of activities and mechanisms for information key social assistance information (expenditure, rules of dissemination (general information, awareness and sensitization operation, beneficiary list, procurement, M&E and other campaigns, mass media, written materials, community meetings, etc.) performance information, etc.) at the national and sub-national - Share of local governments that publish quarterly reports of budget government levels execution, procurement, performance information, etc. - Share of local offices that post key program information (budget and expenditure, service standards, performance measures, complaints procedures, etc.) in accessible places - Timely publication on the internet of key social assistance information (expenditure, rules of operation, performance assessment, etc.) at the national and sub-national government levels - Availability on internet of a summary of program budget written in clear, simple language - Public awareness of program rights and obligations (e.g. % of target population who correctly identify the eligibility criteria for social assistance benefits) - Perceptions of transparency in social assistance services (among clients, journalists, etc.) Participation - Existence of client participation principles and mechanisms - - Level of client participation in setting and reviewing service standards Existence of staff guidance and training to promote client - Level and type of client participation in service implementation and participation monitoring (consultations, planning, service contracting, oversight, - Existence of capacity building activities for stakeholder participatory M&E) participation - Type (social audits, local committees, etc.) and functioning of client participation mechanisms (i.e. high level of involvement of a well- represented group of beneficiaries) - Clients awareness of participation mechanisms (% of clients who are able to mention at least one participation option) - % of clients that report an active participation in service planning, implementation or monitoring - Clients perception of the value of their participation to influence service quality 29    Table 6: Demand-side Service Quality Indicators in Social Assistance Programs Issue Policy dimension Performance dimension Grievance - Existence of grievance redress policies and procedures - Client awareness, potential and actual use of grievance redress Redress (procurement, fraud, corruption, service quality, etc.) mechanisms (% of clients who are able to mention at least one complaints Mechanisms - Existence of grievance redress institutional arrangements mechanism, % of clients who would complain in case of poor service (appeals committee, ombudsman, etc.) quality, % of clients that considered service quality was bad and used the - Type of channels for individuals to complain and refer cases established procedures to complained about it, % of clients who have (e.g. telephone hotlines, on-line, town-hall meetings) provided feedback) - Existence of staff guidance and training to handle complaints - Number and type of formal and informal complaints, compliments and and empowerment to put things right suggestions received - Existence of procedures to receive, record and address - Average time to address a complaint or dispute complaints, compliments and suggestions - Percentage of grievance redress claims settled within agreed time - Extent to which there is public access to information on the - Clients and staff assessment of complaints procedure and its number and type of complaints, compliments and suggestions effectiveness received, along with the improvements made as a result - Percentage of unresolved complaints or disputes - Extent to which there is a regular review and improvement of - % of local offices that publicized regularly information on the number complaints procedure, taking account of the views of clients and and type of complaints, compliments and suggestions received, along with staff the improvements made as a result - % of local offices that have made during the last semester at least one change in service policies or procedures as a result of client feedback. Independent - Existence and adequacy of auditing policies (procurement, - Frequency and scope/nature of audits performed Assessment financial, operational and performance) - Type, relevance, independence and quality of program M&E tools (spot - Existence of systematic procedures to follow up on audit checks, facilities survey, scorecards, beneficiary assessments, impact recommendations evaluations, etc.) - Existence and content of M&E policies (i.e. existence of - Client awareness of independent assessment activities, results and systematic activities, adequate institutional arrangements and program management response to them corresponding budget allocation for M&E) - Existence of systematic procedures for management response to M&E findings 30    4.7 Data Collection   Indicators for measuring governance and service quality are diverse and require a variety of data collection instruments. For example, policy indicators tend to focus on assessing the existence and adequacy of rules, norms and regulations and may be more amenable to measurement through expert surveys. In contrast, governance performance indicators are more likely to be quantitative measures that can be obtained from administrative data for regular monitoring. This information can also be measured or independently assessed through facilities surveys. Measures of organizational performance on targeting, service quality and demand-side accountability may require household or beneficiary surveys, as well as qualitative data. Expert surveys: This tool elicits expert assessment of key performance dimensions using a set of evaluation criteria. It is useful to assess indicators that are better measured through a combination of quantitative and/or qualitative variables. For example, measuring the existence and clarity of job descriptions and responsibilities would consider the proportion of staff positions with a documented job profile, as well as a qualitative assessment of their adequacy. PEFA Public Financial Management Performance Framework discussed earlier is an example of a widely used expert survey for assessing governance policies and performance on financial management issues. Administrative or monitoring data: Existing government databases can be a rich data source, particularly for governance performance information on human resources, financial and resource management, operational procedures, information management and control mechanisms. For example, indicators such as staff turnover rates, budget execution, the percentage of cash transfers made directly to the recipient’s account, administration costs and the coverage rate can be measured through administrative data. The quality and objective measurement of these data may be heterogeneous, thus it is important to cross- check with other data sources such as facility or beneficiary surveys. For instance, records may show that a certain amount of transfers have been delivered but they may not capture whether a payment agent received an amount from the recipient to disburse the cash. Facility surveys (PETS/QSDS): This instrument collects data on the quality and quantity of public services delivered and the resources used to deliver them at the facility level. Public Expenditure Tracking Surveys (PETS) measure the flow of funds and other resources through the system and assess whether they result in service delivery. PETS can be linked to Quantitative Service Delivery Surveys (QSDS) that focus more on the quantity and quality of services and the behavior of providers. Although most PETS have focused on health and education, they can be adapted to social assistance services. Beneficiary or household surveys: A number of indicators are aimed at measuring providers’ performance through clients experience with service delivery. Beneficiary or 31    household surveys collect data on individuals’ behaviors, knowledge, perceptions and practices, particularly relevant for assessing service quality and demand-side indicators. Examples of indicators measured through household and beneficiary surveys include transaction costs (in terms of money and time spent) involved in program participation, amount and frequency of social assistance transfers received, service accessibility, awareness of rights and obligations and program engagement. National household surveys are also useful for targeting measures and independent measurement of service coverage. Qualitative data: Surveys and quantitative data sources can be complemented with qualitative data that helps understanding the gap between governance policies on paper (de jure) and their actual implementation (de facto). For example, while human resource policies may specify provisions for competitive hiring processes, in practice, positions may be sold or procedures circumvented in many ways. Moreover, there are some areas that are more amenable to qualitative assessment. For example, socio-cultural barriers to service access could be better assessed using qualitative methods since socio-cultural issues can be differently defined and understood by groups or individuals. 4. Conclusions  Governance lenses have not been applied yet to social protection in a comprehensive and systematic way. This paper aimed at contributing to this process by developing a framework for measuring governance and quality of service delivery in social safety nets. The proposed framework assesses governance policies and performance in human resources, financing and resource management, operational procedures and control mechanisms. The framework also incorporates measures of organizational performance in cash transfer delivery, emphasizing service quality, and demand-side accountability. Finally, based on the proposed framework, a list of governance and service quality indicators for social assistance programs is identified. The selection of any particular set of indicators has underlying normative assumptions. There is some evidence from OECD countries on which governance interventions are cost-effective measures for reducing EFC. However, not all governance interventions have proved their impact on organizational performance, including service quality, or program outcomes. Therefore, the list of indicators should be treated as hypothesis to be tested rather than evidence to be followed. Furthermore, the selection of specific governance indicators will differ by country and program context. For example, governance issues will vary between programs that are operated by a central bureaucracy and those operated by an agent, such as local governments. Similarly, different governance aspects such as controlling error, fraud or corruption may be more or less relevant for low, medium and high income countries. 32    5. References  Adema, Willem and Maxime Ladaique (2009), “How Expensive is the Welfare State?: Gross and Net Indicators in the OECD Social Expenditure Database (SOCX)�, OECD Social, Employment and Migration Working Papers, No. 92, OECD Publishing. African Development Bank (2009), “Federal Republic of Nigeria. Country Governance Profile�, African Development Group. Bitner, Mary Jo, et. al. (1997), “Customer Contributions and Roles in Service Delivery�, International Journal of Service Industry Management, Vol. 8. No. 3 Bjuremalm, Helena (2006), “Power Analysis – Experiences and Challenges�, Department for Democracy and Social Development. Boulding, William, et. al. (1993), “A Dynamic Process Model of Service Quality: From Expectations to Behavioral Intentions� Journal of Marketing Research, Vol. XXX Brinkerhoff, Derick W. and Thomas J. Bossert (2008), “Health Governance: Concepts, Experience and Programming Options� Health Systems 20/20, February 2008, USAID. Clingendael Institute (2005), “Problem Driven Governance and Political Economy Analysis�, The Netherlands Ministry of Foreign Affairs. Clingendael Institute (2008), “Framework for Strategic Governance and Corruption Analysis. Designing Strategic Responses towards Good Governance�, The Netherlands Ministry of Foreign Affairs. Coady, David, Margart Grosh ,and John Hoddinott (2004), “Targeting of Transfers in Developing Countries: Review of Lessons and Experience�. The World Bank: Washington, DC. De Crombrugghe, Denis, et al. (2009), “Institutional Profiles Database III�, Documents de Travail de la DGTPE, Document 2009/14, Direction Générale du Trésor et de la Politique Économique. Drivers of Change Team (2003), “What does Drivers of Change mean for DFID?� Department for International Development Policy Division, A draft approach paper. United Kingdom. 33    European Centre for Development Policy Management (2008), “Analyzing and Addressing Governance in Sector Operations�, Tools and Methods Series, Reference Document No. 4, European Commission. European System of Integrated Social Protection Statistics (2008), “ESSPROS Manual�, Eurostat Methodology and Working Papers, European Commission. Federal Ministry for Economic Cooperation and Development (2007), “Development- Oriented Transformation in Conditions of Fragile Statehood and poor Government Performance�, Federal Ministry for Economic Cooperation and Development: Germany. Fiszbein, Ariel, Dena Ringold, and Halsey Rogers (2011). "Making Services Work: Indicators, Assessments, and Benchmarking of the Quality and Governance of Public Service Delivery in the Human Development Sectors." Policy Research Working Paper 5690. The World Bank: Washington, DC. Fritz, Verena et al. (2009), “Problem Driven Governance and Political Economy Analysis. Good Practice Framework�, World Bank. Hagemejer, Krzysztof, “Social protection expenditure and performance reviews. Methodological note on definitions, classifications and performance indicators�, Financial, Actuarial and Statistical Services, International Labour Office. Mimeo. Halloran, John, and Kerstin Calderon-Vera, “Access to Quality Social Services. A Strategy Paper�. European Social Network. Mimeo. http://www.peer-review-social- inclusion.eu/key-themes/quality-and-accessibility-of-social-services-1 Inter American Development Bank, “Governance Indicators Database�, accessed on June 25, 2010. Available online at [http://www.iadb.org/datagob/index.html]. International Labour Organization, “The Social Security Database: User Guide Book�, accessed on June 26, 2010. Available online at [http://www.ilo.org/dyn/sesame/SESHELP.SSDBCodeBookIndex] International Monetary Fund (2009), “Safeguards Assessments�, International Monetary Fund. Finance Department. ________ (2007), “Questionnaire on Fiscal Institutions�, International Monetary Fund. ________ (2006), “Reference Guide to Anti – Money Laundering and Combating the Financing of Terrorism�, Second Edition and Supplement on Special Recommendation IX, 34    The International Bank for Reconstruction and Development/The World Bank/The International Monetary Fund. Kaufmann, Daniel, et al. (2009), “Governance Matters VIII. Aggregate and Individual Governance Indicators 1996–2008�, World Bank Policy Research Working Paper 4978. Washington, DC: World Bank. Kaufmann, Daniel and Aart Kraay (2008), “Governance Indicators: Where Are We, Where Should We Be Going?� The World Bank Research Observer, 23(1):1-30. Kaufmann Daniel, Aart Kraay and Massimo Mastruzzi (2007), “Governance Matters VI: Aggregate and Individual Governance Indicators for 1996–2006.� World Bank Policy Research Working Paper 4280. July. Washington, DC: World Bank. Lewis, Maureen and Gunilla Pettersson (2009), “Governance in Health Care Delivery: Raising Performance,� Washington, DC: World Bank. Mimeo. Lewis, Maureen and Gunilla Pettersson (2009), “Governance in Education: Raising Performance,� Washington, DC: World Bank. Mimeo. Mathauer; Inker (2004), “Institutional Analysis Toolkit for Safety Net Interventions,� Social Protection Discussion Paper Series No. 0418. Social Protection Unit, Human Development Network. Washington, DC: World Bank Mardan, Mohan (2009), “Sector Level Risk Assessment and Risk Management Plan to Assess Public Financial Management, Procurement, and Corruption Risks in the Urban Development Sector�, Asian Development Bank Consultant’s Report. Millennium Challenge Corporation (2009), “Report on the Criteria and Methodology for Determining the Eligibility of Candidate Countries for Millennium Challenge Account Assistance in Fiscal Year 2010�, Millennium Challenge Corporation. Mutual Information System on Social Protection (2007), “Organisation of Social Protection. Charts and Descriptions�, European Commission, Directorate-General for Employment, Social Affairs and Equal Opportunities. Mutual Information System on Social Protection (2008), “Comparative Tables of Social Protection Systems: Correspondent’s Manual 2008�, Council of Europe. OECD (2009), “Donor Approaches to Governance Assessment. 2009 Sourcebook,� DAC Network on Governance 35    ______ (2008), “Survey of Donor Approaches to Governance Assessment,� DAC Network on Governance ______ (2006), “Methodology for Assessment of National Procurement Systems�, Version 4, OECD. OECD, “Benefits and Wages 2007�, accessed on June 29, 2010. Available online at [http://www.oecd.org/document/33/0,3343,en_2649_34637_39619553_1_1_1_1,00.html#r elated OECD, “OECD Family Database�, accessed on June 29, 2010. Available online at [http://www.oecd.org/document/4/0,3343,en_2649_34819_37836996_1_1_1_1,00.html] OECD, “Pension Database�, accessed on June 29, 2010. Available online at [http://www.oecd.org/document/16/0,3343,en_2649_34757_45558288_1_1_1_1,00.html] OECD, “The OECD Sickness, Disability and Work Project�, accessed on June 29, 2010. Available online at [http://www.oecd.org/document/20/0,3343,en_2649_34747_38887124_1_1_1_1,00.html] Office of Conflict Management and Mitigation (2005), “Conflict Assessment: A Framework for Strategy and Program Development�, United States Agency for International Development. Operations Policy and Country Services (2009), “Country Policy and Institutional Assessments. 2009 Assessment Questionnaire�, The World Bank: Washington, DC. PEFA Secretariat (2005), “Public Financial Management. Performance Measurement Framework�, The World Bank: Washington, DC. Ravallion, Martin (2007), “How Relevant is Targeting to the Success of an Antipoverty Program�, Policy Research Working Paper 4385, November 2007, Office of the Director, Development Research Group, Washington, DC: World Bank. Reid, Gary J. (2008), “Actionable Governance Indicators – Concepts and Measurement,� AGI Initiative. The World Bank: Washington, DC. Ringold, Dena, et. al. (2010), “The Demand-side of Governance in HD: Results of a Stocktaking of FY08 Projects� The World Bank: Washington, DC. Rogers, Edward W. and Patrick M. Wright (1998), “Measuring Organizational Performance in Strategic Human Resource Management: Problems and Prospects� Center 36    for Advanced Human Resource Studies, Working Paper 98-09, School of Industrial and Labor Relations, Cornell University, Ithaca, NY Savedoff, William D (2011), “Governance in Health Sector: A Strategy for Measuring Determinants and Performance,� Portland, ME: Social Insight. Policy Research Working Paper 5655. The World Bank: Washington, DC. Seth, Nitin, S.G. Deshmukh, and Prem Vrat (2005), “Service Quality Models: a Review� International Journal of Quality and Reliability Management. Vol. 22 No. 9 Spector, Bertram et al. (2009), “Anticorruption Assessment Handbook�, U.S. Agency for International Development. Swiss Agency for Development and Cooperation (2007), “Governance as a Transversal Theme: an Implementation Guide�, Swiss Federal Department of Foreign Affairs. U.K. Department for International Development (2002), “Conducting Conflict Assessments: Guidance Notes�, Conflict and Humanitarian Affairs Department. U.K. Department for International Development (2007), “Country Governance Analysis�, How to note, Department for International Development. United Nations Development Program UNDP (2007), “Governance Indicators: A Users’ Guide,� Second edition, Bureau for Development Policy, Democratic Governance Group, New York _______ (2009), “A Users’ Guide to Measuring Public Administration Performance,� Oslo Governance Center, Democratic Governance Group, Bureau for Development Policy, Oslo, Norway. United Nations Office on Drugs and Crime, “Comprehensive Self-Assessment Checklist on the Implementation of the United Nations Convention Against Corruption�, accessed on June 26, 2010. Available online at [http://www.unodc.org/unodc/en/treaties/CAC/self-assessment.html]. United States Agency for International Development (2000), “Conducting a DG Assessment: a Framework for Strategy Development�, United States Agency for International Development Technical Publications Series. World Bank (2009), “Core Guidance: Preparing PERs for Human Development�, The World Bank: Washington, DC. 37    ________ (2008), “Country Procurement Assessment Report. Philippines�, Document 47602, The World Bank: Washington, DC. ________ (2007), “Control and Accountability Mechanisms in Conditional Cash Transfer Programs: A Review of Programs in Latin America and the Caribbean� Operational Innovations in Latin America and the Caribbean, Vol. 1. No. 1. ________ (2003a), “World Development Report 2004. Making Services Work for Poor People,� Washington DC: The World Bank and Oxford University Press ________ (2003b), “Country Financial Accountability Assessment Guidelines to Staff�, World Bank. Financial Management Sector Board. World Bank, “Actionable Governance Indicators Data Portal�, accessed on June 29, 2010. Available online at [https://www.agidata.org/main/Home.ashx] World Bank, “Governance Diagnostic Surveys. Survey Instruments - Sample Questionnaires�, accessed on June 25, 2010. Available online at [http://web.worldbank.org/WBSITE/EXTERNAL/WBI/EXTWBIGOVANTCOR/0,,conten tMDK:20726143~menuPK:1740556~pagePK:64168445~piPK:64168309~theSitePK:1740 530,00.html]. World Health Organization (2008), “Toolkit on monitoring health systems strengthening: Health Systems Governance,� Geneva: World Health Organization 38    Annexes    Annex 1. Overview of governance and social protection assessment tools     Tool Approach to data Focus areas and analysis Financial Public Human Transparency Corruption Service Other Relevant Areas Social Protection Management adminis- Resource and delivery tration Manage Accountability ment General governance assessment tools 1 France, Set of quantitative -Decentralization Institutional indicators drawing -Micro-lending Profiles (IP) on expert opinion and factual data · · · · 2 United States, Set of quantitative MCC indicators drawing Scorecard on third-party indicators · · · 3 World Bank, Set of quantitative Gender Equality -Equity of public Country indicators drawing resource use Policy and on third-party -Old-age/Pension Institutional Assessment indicators · · · · · · benefit (CPIA) 4 World Bank Set of quantitative Voice and Press Institute, WGI indicators drawing Freedom on third-party indicators · · · · · 5 EC, Incentive No detailed Tranche information Methodology available (ITM) United Qualitative expert -Payment exemption for Kingdom, perceptions basic services for the CGA drawing on secondary sources · · · · · poor -Pro-poor policy 39    Tool Approach to data Focus areas and analysis Financial Public Human Transparency Corruption Service Other Relevant Areas Social Protection Management adminis- Resource and delivery tration Manage Accountability ment and on a -Perceptions of policy mandatory set of effectiveness third-party indicators 7 Switzerland, Qualitative expert MERV perceptions drawing on secondary sources and third-party · · · indicators 8 Inter- Qualitative expert - E-government American perceptions -Information Development drawing on technologies Bank, DGIA secondary sources and third-party · · · · · indicators 9 World Bank Quantitative data -Reforms agreement -Additional expenses Institute, generated through by government staff due to services not Governance representative received and Anti- surveys -Complaints at social Corruption Country · · · · · · security institutions Survey (GAC) 10 African Qualitative expert Development perceptions Bank, CGP drawing on secondary sources · · · · 11 Germany, Qualitative expert -Cooperative stance Criteria perceptions within the Catalogue (CC) drawing on secondary sources · · · · international community 12 ADB, Qualitative expert Governance perceptions Risk Assessment drawing on secondary sources · · · · · 40    Tool Approach to data Focus areas and analysis Financial Public Human Transparency Corruption Service Other Relevant Areas Social Protection Management adminis- Resource and delivery tration Manage Accountability ment and Risk and primary Management research for Plans updating (GRARMP) 13 Switzerland, Qualitative expert Key perceptions Questions (KQ) drawing on secondary sources · 14 Switzerland, Qualitative expert -Beneficiaries Governance analysis drawing participation in policy as transversal (GT) on secondary sources · · · · design 15 United States, Qualitative expert -Donors’ interests DGSAF perceptions assessment drawing on secondary sources · · · · 16 Netherlands, Qualitative expert -Historical and Strategic perceptions geographical factors Governance drawing on -Donors’ interests and Corruption secondary sources · · · · · · assessment Analysis (SGACA) 17 Sweden, Qualitative expert -Influence of poor Power perceptions people and Analysis drawing on organizations of poor (PA) secondary sources people in the decision – and a series of making process closed/open workshops with · · · · key stakeholders at various levels of society 18 United Qualitative expert -Historical factors -Promotion of pro – Kingdom, Drivers of perceptions drawing on · · · · -Role of aid poor growth 41    Tool Approach to data Focus areas and analysis Financial Public Human Transparency Corruption Service Other Relevant Areas Social Protection Management adminis- Resource and delivery tration Manage Accountability ment Change secondary sources (DoCh) Specific/thematic governance assessment tools 19 IMF, Qualitative expert Assessment of perceptions AML/CFT drawing on secondary sources 20 IMF, Central Questionnaire and Bank qualitative expert Safeguards perceptions 21 IMF, Fiscal Qualitative expert Reports on perceptions Observance of drawing on Standards and secondary sources Codes · · · · (ROSC) 22 PEFA Set of indicators Programme, drawing on expert PEFA opinion and factual data · · · 23 World Bank, Analysis of -Budget allocation for Public fiscal/budgetary social protection, trends Expenditure data; interviews and processes Review -Adequacy of social (SP-PER) protection programs compared with selected · · · · benchmarks -Equity of social protection programs -Old-age/Pension benefit -Unemployment benefit -Risk management 24 World Bank, Qualitative expert CFAA analysis drawing on primary · · · · 42    Tool Approach to data Focus areas and analysis Financial Public Human Transparency Corruption Service Other Relevant Areas Social Protection Management adminis- Resource and delivery tration Manage Accountability ment documents and secondary sources and interviews 25 OECD/DAC Set of indicators -Procurement systems JV on drawing on expert assessment Procurement opinion and factual data · · · 26 UNODC Quantitative and UN qualitative Convention information against Corruption obtained from official · · Checklist governmental sources 27 Germany, Qualitative expert Conflict perceptions Prevention/Ea drawing on rly Warning secondary sources 28 Germany, No detailed Security information Sector Reform available Assessment 29 World Bank, Qualitative expert -Procurement systems CPAR perceptions assessment drawing on secondary sources · · · 30 World Bank, Variable -Problem – driven Upstream framework adaptable Problem- to any program or Driven Governance · · · · · · institution Diagnostic (UPGD) 31 EC, Human Qualitative expert Rights Fact perceptions 43    Tool Approach to data Focus areas and analysis Financial Public Human Transparency Corruption Service Other Relevant Areas Social Protection Management adminis- Resource and delivery tration Manage Accountability ment Sheet drawing on secondary sources 32 Netherlands, Expert -Deterioration of state -Pensions payment Stability perceptions functions (including failures Assessment drawing on basic services) Framework, (SAF) secondary sources and a set of · · · indicators for trend analysis 33 United Qualitative expert Kingdom, perceptions Strategic drawing on Conflict secondary sources Assessment 34 United States, Qualitative expert Conflict perceptions Assessment drawing on Framework secondary sources 35 United States, Qualitative expert - Corruption Anti- perceptions indicators related to Corruption drawing on healthcare Assessment Framework, secondary sources · · · · · · - Corruption indicators related to (ACF) education 36 EC, Sector Qualitative expert Governance perceptions Analysis Framework drawing on secondary sources · · · 37 World Bank, Quantitative and Actionable qualitative Governance Indicators assessment based on secondary · · · · · (AGI) sources     44    Tool Approach to data Focus areas and analysis Financial Public HRM Transparency Corruption Service Other Relevant Areas Social Protection Management adminis- and delivery tration accountability Social Protection 38 ILO, Social Quantitative data Indicators available for Security collected using a one of each of the Database questionnaire following branches: (SSD) medical care, sickness, maternity, old-age, invalidity, survivors', family, employment injury and unemployment. -Expenditure as percentage of the GDP -Number of programs -Number of laws and antiquity -Coverage -Employee and employer contribution · rate -Period and residence condition for being eligible -Duration and evolution of the benefits It also contains branch - specific indicators. -Maternity benefit -Old-age/Pension benefit -Invalidity benefit -Survivors' benefit -Family benefit -Employment injury benefit -Unemployment benefit. 45    Tool Approach to data Focus areas and analysis Financial Public HRM Transparency Corruption Service Other Relevant Areas Social Protection Management adminis- and delivery tration accountability 39 ILO, Social Quantitative data -Account of receipts and Protection collected form expenditure Expenditure secondary sources - Coverage and Performance · - Distributional performance Reviews (SPEPR) 40 EC, Mutual Qualitative -Contributions rates and Information assessment based ceilings System on on country - Invalidity benefit Social Protection specific legislation · - Old-age/Pension benefit (MISSP) - Unemployment benefit 41 EC, Qualitative - Invalidity benefit ESSPROS assessment based - Old-age/Pension on country benefit Survivor’s specific benefit legislation - Family benefit - Unemployment benefit - Housing benefit 42 CE, Qualitative - Invalidity benefit MISSCEO assessment based -Maternity benefits on country -Old-age/Pension specific legislation · benefit - Employment injury benefit 43 OECD, Qualitative - Unemployment benefit Benefits and assessment based - Effectiveness of social Wages en statistical benefits in supporting models family incomes. 44 OECD, Social Quantitative - Aggregate data on Expenditure assessment based social expenditure Database on official sources - Data on itemized tax rates 45 OECD, Quantitative - Public spending on Family assessment based · family benefits 46    Tool Approach to data Focus areas and analysis Financial Public HRM Transparency Corruption Service Other Relevant Areas Social Protection Management adminis- and delivery tration accountability Database on secondary - Child support sources and (maintenance) system qualitative - Maternity benefit assessment base - Additional leave on country entitlements of working specific parents legislation - Typology of childcare and early education services - Quality of childcare workers 46 OECD, Quantitative -Old-age/Pension Pension assessment based benefit Database on statistical - Aggregate data on models and contribution rates official sources - Aggregate data on pension expenditure - Value of pension funds 47 OECD, Quantitative - Disability benefit Sickness assessment based - Disability employment Disability and on official sources Work (project) Sources: OECD 2009 and 2008 (1) Assessment framework: Institutional Profiles Database III. http://www.cepii.fr/institutions/09-12_DGT_IPD%202009_cahiers-2009-14%20ENG.pdf (2) Assessment framework: Report on the Criteria and Methodology for Determining the Eligibility of Candidate Countries for Millennium Challenge Account Assistance in Fiscal Year 2010. http://www.mcc.gov/mcc/bm.doc/mcc-report-fy09-criteriaandmethodology.pdf (3) Assessment framework: Country Policy and Institutional Assessment. http://siteresources.worldbank.org/IDA/Resources/73153- 1181752621336/CPIA09CriteriaB.pdf (4) Assessment framework. Governance Matters VIII. http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1424591 (5) Assessment tool not available. 47    (6) Assessment framework: Country Governance Analysis. http://www.reliefweb.int/rw/lib.nsf/db900sid/JBRN-737J8E/$file/dfid-humanitarianaction- feb07.pdf?openelement  (7) Assessment tool not available. Indicators taken from Donor Approaches to Governance Assessment Sourcebook 2009.  http://www.oecd.org/dataoecd/25/12/42472200.pdf (8) Assessment tool not available. Indicators taken from the underlying Governance Indicators database. http://www.iadb.org/datagob/ (9) Questionnaire for households: http://siteresources.worldbank.org/INTWBIGOVANTCOR/Resources/sample_user.pdf. Questionnaire for public officials: http://siteresources.worldbank.org/INTWBIGOVANTCOR/Resources/sample_publicofficial.pdf (10) Assessment tool not available. Indicators taken from country - specific reports (narrative), available when country partners agree. http://www.afdb.org/en/documents/project-operations/country-governance-profiles/ (11) Assessment framework: Development-Oriented Transformation in Conditions of Fragile Statehood and poor Government Performance. http://www.oecd.org/dataoecd/4/38/43480415.pdf (12) Assessment tool not available. Indicators taken from country – specific reports (narrative). See for example the Nepal’s report: Sector Level Risk Assessment and Risk Management Plan to Assess Public Financial Management, Procurement, and Corruption Risks in the Urban Development Sector. http://www.adb.org/Documents/Assessments/Other-Assessments/NEP/Risk-Assessment-Plan-June2009.pdf (13) Assessment tool not available. Indicators taken from Donor Approaches to Governance Assessment Sourcebook 2009.  http://www.oecd.org/dataoecd/25/12/42472200.pdf (14) Assessment framework: Governance as a Transversal Theme: an Implementation Guide. www.deza.admin.ch/ressources/resource_en_156840.pdf  (15) Assessment framework: Conducting a DG Assessment: a Framework for Strategy Development. http://www.usaid.gov/our_work/democracy_and_governance/publications/pdfs/pnach305.pdf  (16) Assessment framework: Framework for Strategic Governance and Corruption Analysis (SGACA). http://www.minbuza.nl/dsresource?objectid=buzabeheer:43185&type=org  (17) Assessment framework: Power Analysis – Experiences and Challenges. http://sidapublications.citat.se/interface/frmoptimaker3.asp?doctype=3&order=createdate%20DESC&departmentid=298&topheight=55&headerheight=23&f otheight=0&leftframewidth=300&width=820&stylesheet=sida.css&frameout=0&language=14&login=True&username=sida2&password=sida2 (18) Assessment framework: What does Drivers of Change mean for DFID? http://webarchive.nationalarchives.gov.uk/+/http://www.dfid.gov.uk/Contracts/files/ojec_5512_doc_approach.pdf  (19) Assessment framework: Reference Guide to Anti – Money Laundering and Combating the Financing of Terrorism. http://siteresources.worldbank.org/EXTAML/Resources/396511-1146581427871/Reference_Guide_AMLCFT_2ndSupplement.pdf. Neither social protection indicators nor adaptable indicators for social protection contained.  (20) Assessment framework: Safeguards Assessments. http://www.imf.org/external/np/pp/eng/2009/121509.pdf. Protecting IMF Resources: Safeguards Assessments of Central Banks. http://www.imf.org/external/np/exr/facts/safe.htm. Neither social protection indicators nor adaptable indicators for social protection contained.  48    (21) Questionnaire on Fiscal Institutions. http://www.imf.org/external/np/fad/trans/question/quest.pdf (22) Assessment framework: Public Financial Management. Performance Measurement Framework. http://www.pefa.org/pfm_performance_file/the_framework_English_1193152901.pdf  (23) Assessment framework: Core Guidance: Preparing PERs for Human Development. http://siteresources.worldbank.org/EXTPERGUIDE/Resources/PER- Complete.pdf. Includes a complete section for social protection assessment.  (24) Assessment framework: Country Financial Accountability Assessment Guidelines to Staff. http://www1.worldbank.org/publicsector/pe/cfaaguidelines.pdf.   (25) Assessment framework: Methodology for Assessment of National Procurement Systems. http://www.oecd.org/dataoecd/1/36/37130136.pdf (26) Assessment made using a computer-based application: Comprehensive self-assessment checklist  http://www.unodc.org/unodc/en/treaties/CAC/self-assessment.html  (27) Assessment tool not available. (28) Assessment tool not available. (29) Assessment tool not available. Indicators taken from country – specific reports. See for example the report for Philippines: Country Procurement Assessment Report. http://www- wds.worldbank.org/external/default/WDSContentServer/WDSP/IB/2009/04/21/000334955_20090421082041/Rendered/PDF/476020REVISED010WB0CP AR0081April020.pdf (30) Variable methodology: Problem Driven Governance and Political Economy Analysis. The problem – based methodology permits to assess the performance of almost every institution, process and program in the country, sector, or project levels. The framework emphasizes a focus on specific problems or vulnerabilities, as well as the need to understand political economy drivers by examining them in a systematic way. http://vle.worldbank.org/bnpp/files/Fritz%20Kaiser%20Levy%202009%20PGPE_book_12-15-09%20FINAL.pdf (31) Assessment tool not available. (32) Assessment framework: The Stability Assessment Framework: Designing Integrated Responses for Security, Governance and Development. http://www.clingendael.nl/publications/2005/20050200_cru_paper_stability.pdf. (33) Variable methodology: Conducting Conflict Assessments: Guidance Notes. The methodology gives particular attention to the development of strategies/options for programming interventions. http://webarchive.nationalarchives.gov.uk/+/http://www.dfid.gov.uk/pubs/files/conflict-assess-guidance.pdf (34) Variable methodology: Conducting a Conflict Assessment: A Framework for Strategy and Program Development. Designed to help missions: 1) identify and prioritize the causes and consequences of violence and instability that are most important in a given country context; 2) understand how existing development programs interact with these factors; and 3) determine where development and humanitarian assistance can most effectively support local efforts to manage conflict and build peace. http://www.usaid.gov/our_work/cross-cutting_programs/conflict/publications/docs/CMM_ConflAssessFrmwrk_May_05.pdf (35) Assessment framework: Anticorruption Assessment Handbook. http://www.usaid.gov/our_work/democracy_and_governance/technical_areas/anticorruption_handbook/index.html 49    (36) Assessment framework: Analyzing and Addressing Governance in Sector Operations. The analysis framework offers guidance on how to address governance in sector operations in a systematic and comprehensive way. It can be applied for example to health, water and sanitation, education, and transport. http://www.nilsboesen.dk/uploads/docs/Sector%20Governance2008.pdf (37) Assessment tool not available. Indicators taken from Actionable Governance Indicators Data Portal: https://www.agidata.org/main/Home.ashx (38) Assessment framework The Social security database: User Guide Book. http://www.ilo.org/dyn/sesame/SESHELP.SSDBCodeBookIndex (39) Assessment framework: Social protection expenditure and performance reviews. Methodological note on definitions, classifications and performance indicators. http://www.ilo.org/public/english/protection/secsoc/downloads/stat/spersmet.pdf (40)  Assessment framework: Organisation of Social Protection. Charts and Descriptions.  http://ec.europa.eu/employment_social/missoc/2007/organisation_en.pdf  (41)  Assessment framework: ESSPROS Manual. http://ec.europa.eu/eurostat/ramon/statmanuals/files/KS�RA�07�027�EN.pdf  (42) Assessment framework: Comparative Tables of Social Protection Systems: Correspondent’s Manual 2008.  http://www.coe.int/t/dg3/socialpolicies/socialsecurity/Source/corrmanual_en.pdf  (43) Assessment framework: Benefits and Wages 2007. OECD Indicators. http://www.oecd.org/document/33/0,3343,en_2649_34637_39619553_1_1_1_1,00.html#related  (44) Assessment framework: How Expensive is the Welfare State? Gross and Net Indicators in the OECD Social Expenditure Database (SOCX) http://www.oecdilibrary.org/docserver/download/fulltext/5ks712h5cg7l.pdf?expires=1277863783&id=0000&accname=freeContent&checksum=892F72B32 012FF1670C2697DFFA1F108 (45) Assessment tool not available. Indicators taken from OECD Family Database. http://www.oecd.org/document/4/0,3343,en_2825_497118_37836996_1_1_1_1,00.html (46) Assessment tool not available. Indicators taken from Pensions Database. http://www.oecd.org/document/16/0,3343,en_2649_34757_45558288_1_1_1_1,00.html (47) Assessment framework: The OECD Sickness, Disability and Work Project. http://www.oecd.org/document/20/0,3343,en_2649_34747_38887124_1_1_1_1,00.html3 50    Annex 2. List of indicators related to social protection issues    Tool Social Protection Issue Indicators/Questions 3 World Bank, Country  Equity of Public Resource  Strong poverty diagnosis is in place that very clearly identifies poor, vulnerable groups, and those Policy and Use lacking services Institutional  There are no egregious regressive revenue sources Assessment (CPIA)  Public expenditures are fully aligned with poverty reduction priorities  Labor market  Labor market regulations and active labor market policies promote broad access to employment in the formal sector and reflects a balance between social protection and job creation objectives in accordance with the economic circumstances and values of the country.  Old-age/Pension benefit  A diversified, well-supervised, and appropriate combination of pension and savings programs (including mandatory, voluntary, public, private, funded, pay-as-you go, contributory and non- contributory programs) provide affordable, adequate, sustainable and robust income security to most of the potentially vulnerable groups with minimal distortions in the operation of labor markets. 6 United Kingdom,  Payment exemption for basic  Extent to which poor people are required to pay for basic services Country Governance services for the poor  Effective of payment exemption arrangements for poor and vulnerable people Analysis (CGA)  Pro-Poor Policy  Extent to which poverty reduction is an explicit policy priority and how effectively is the policy decided, implemented and monitored in practice  Extent to which public goods and services are provided in ways that recognize, address and reduce discrimination and allow all citizens – and including women, disabled people and ethnic minorities – to benefit  Perceptions of the  How much confidence do the poor have in the ability of the government to help solve their problems effectiveness of policy and in their own ability to influence it? 9 World Bank Institute,  Additional expense due to  Additional costs incurred due to failure of the social security institution to provide requested services Governance and services requested not Anti-Corruption received Country Survey  Payment and bribes  Frequency in which public servants request, or people feel obliged to give retributions like tips/gifts/ (GAC) bribes, etc. at Social Security attention.  Complaining at social  Have you filed any complaints for bad service, delay or mishandling received at some Social Security security institutions institution? 51    Tool Social Protection Issue Indicators/Questions 17 Sweden, Power  Influence of poor people and  How can networks and federations of poor people’s organizations (women and men) be heard and Analysis organizations of poor people represented in decision-making that affects their lives at the family, local, and national levels? in the decision – making  How are the central agents and organizations providing a voice to the poor? process 23 World Bank, Social  Budget allocation for social  Public expenditure in social protection by program and sector “total� as a share of GDP and total Protection Public protection, trends and public expenditures - and in absolute values. Expenditure Review processes  Sustainability of social protection budget (SP-PER)  Adequacy of social  Adequacy of benefit level (benchmarks will vary by program, for example, protection programs average pensions could be compared with average wages, the social pension with the poverty line, compared with selected unemployment insurance with average wages, unemployment assistance with both average wages and benchmarks the poverty line, social assistance with the poverty line, wages on public works jobs with the market wage for similar work and so on  Equity of social protection  For pensions, consideration of intergenerational equity and intra-generational equity among programs participants of different schemes or contribution cohorts.  Contrast the incidence of participation and of payments made where benefits are not uniform in social protection programs.  Old-age/Pension benefit  Procedures and timing for collecting a pension  Unemployment benefit  To what degree do contributions rates for pensions discourage employment in the formal sector?  Risk management  Do social protection programs improve the ability of the household to manage risks by reducing the probability of a shock?  Do social protection programs improve the ability of the household to manage risks by providing an income support through insurance payments (risk mitigation) or transfers (risk coping)? 32 Netherlands,  Deterioration of state  Is there a deterioration or disappearance of basic state functions (e.g. a failure to provide essential Stability Assessment functions (including those of services such as health care, education, sanitation, public transportation)? Framework (SAF) social protection)  Old-age/Pension benefit  Has the state failed to pay the salaries of government employees and armed forces or to meet other financial obligations to its citizens, such as pension payments? 38 ILO, Social Security  Expenditure as percentage of  Total expenditure (public and private) on social protection (percentage of GDP) Database (SSD) the GDP  Mandatory private expenditure as percentage of GDP.  Number of programs  Number of social protection programs.  Number of laws and antiquity  Number of current social protection laws  Date of the first social protection law  Coverage  Coverage of the main social protection program -all residents; all working population (self employed, public and private sector employees); all working population with exception; all salaried workers and 52    Tool Social Protection Issue Indicators/Questions assimilated categories (both public and private sector employees); private sector employees; public sector employees.  Employee and employer  Employee and employer contribution to social protection programs (discretionary, whole cost, flat rate contribution rate amount).  Employer contribution to social protection programs (discretionary, whole cost, flat rate amount).  Government contribution for social protection programs (discretionary, whole cost, flat rate amount).  Period and residence  Period and condition of residence in the country to be eligible for benefits of social protection condition for being eligible programs.  Duration and evolution of the  Duration of benefits in social protection programs. benefits  Evolution of benefit in social protection programs (constant, indexed, decreasing, increasing, discretionary evolution (no fixed rules)).  Maternity benefit  Existence of parental leave.  Are fathers and mothers eligible for paid parental leave?  Existence of maternity benefits in kind.  Old-age / Pension benefit  Legal retirement age for male.  Legal retirement age for female.  Does early retirement provision exist?  Is there gender discrimination in calculation of cash benefits (other than retirement age)?  Survivors' benefit  Calculation rate of the pension of a surviving wife.  Calculation rate of the pension of a surviving husband.  Calculation rate of the pension for the first surviving child.  In case of death, what cash benefit (percentage of earnings) does a surviving spouse receive?  Unemployment benefit.  Maximum age to receive unemployment benefit.  Is accumulation of unemployment benefits possible with other earnings?  Are there special unemployment provisions for old-aged unemployed?  Family benefit  Age limit for payment of benefits for a child.  Age limit for payment of benefits for a student.  Is there equal benefit amount for each child?  Is there any variation in the benefit amount according to the age of the child?  Work injury  Is the travel between home and work also covered by work injury programs?  Invalidity benefit  Calculation method of pension (flat rate, according to incapacity level, according to previous earning, according to number of years of insurance, according to the age). 39 ILO, Social  Account of receipts and  Administration costs of social protection schemes. 53    Tool Social Protection Issue Indicators/Questions Protection expenditure Expenditure and  Coverage  Percentage of persons covered (by gender, age, labor market status) within the whole population or Performance the target group. Reviews (SPEPR)  Range of contingencies, risks and needs covered (old-age and survivors, disability, unemployment, sickness and health, unemployment, maternity, family and children, poverty.  Distributional performance  Portion of cash benefits actually distributed to horizontal distribution groups (with a particular attention to the gender distribution, formal/informal sector distribution, and distribution to other groups identified as the most vulnerable) and access of member of these groups to social protection and other basic social services 40 EC, Mutual  Contributions rates and  Contributions rate and ceilings for social protection Information System ceilings on Social Protection  Invalidity benefit  Preferential employment of handicapped persons  Minimum period of affiliation for entitlement  Old-age / Pension benefit  Minimum and maximum pensions  Supplements for dependants: Spouse/Children  Unemployment benefit  Earnings taken as reference and ceiling for unemployment benefits 41 EC, ESSPROS  Invalidity benefit  Early retirement benefit due to reduced capacity to work  Economic integration of the handicapped  Rehabilitation  Assistance in carrying out daily tasks  Old-age / Pension benefit  Anticipated old age pension  Assistance in carrying out daily tasks  Survivor’s benefit  Funeral expenses  Family benefit  Birth grant  Parental leave benefit  Child daycare  Housing benefit  Rent benefit  Social housing  Benefit to owner-occupiers  Unemployment benefit  Vocational training allowance  Placement services and job-search assistance 42 CE, MISSCEO  Invalidity benefit  Are workers obliged to prove that the reason they are not at work is because they are sick?  Are there any other tests besides incapacity for work and any qualifying period that the applicant must satisfy before receiving the benefit? 54    Tool Social Protection Issue Indicators/Questions  Maternity benefits  Are there any rules on how much time should be taken before and after the birth?  Which is the amount employers are obliged to pay (this maybe a percentage of the employee’s previous wage)?  How long are employers obliged to pay?  Old-age / Pension benefit  Special supplements not related to dependants (cold weather bonuses, payments for those who reach a very great age (80 – 90 years), Christmas or holiday bonuses).  How long does someone have to have been paying contributions, working or residing in the country before they are able to make a claim for benefits?  Survivor’s benefit  What happens to the survivor’s pension if the spouse remarries (or cohabits) with a new partner? Which of the following are entitled to a survivor’s pension: divorced spouse, mixed sex cohabitants (couples living together as man and wife outside of wedlock), same sex cohabitants (homosexual couples living together), illegitimate children, foster children, adopted children.  Employment injury benefit  How long, after the appearance of the incapacity does a person have to wait before the benefit is paid?  Is there a list of diseases that have been recognized as occupational in nature?  Is the recipient re-examined if circumstances change i.e. their condition improves or worsens? 43 OECD, Benefits and  Unemployment benefit  Insurance is voluntary or compulsory for employees? Wages  Initial payment rate (% of earnings base)  Permitted employment and disregards  Additions for dependent family members  Effectiveness of social  Net effects of taxes and social benefits in supporting family incomes by type of family (single person, benefits in supporting family lone parent, one-earner married couple with no children, one-earner married couple with two children, incomes. two-earner married couple with no children, two-earner married couple with two children)  Net replacement rates over a five-year period 44 OECD, Social  Aggregate data on social  Gross and net current public social expenditure (% of GDP) Expenditure expenditure  Gross and net mandatory private social expenditure (% of GDP) Database  Data on itemized tax rates  Average itemized tax rates (some items are inexistent in some countries)  Tax brakes for social purposes (amounts) 45 OECD, Family  Public spending on family  Child-related cash transfers to families with children Database benefits  Public spending on services for families with children  Financial support for families provided through the tax system  Child support (maintenance)  Involvement in the determination of child maintenance (parents, court, administrative agency) system  Rules for determining the amount of payments  Do different arrangements for children of unmarried parents exist?  Age at which support ends  Maternity benefit  Maternity leave maximum duration 55    Tool Social Protection Issue Indicators/Questions  Eligibility criteria for payments of maternity leave  Payment during maternal leave (% of earnings)  Paternity leave maximum duration  Level of payment during parental leave  Proportion of employed parents with a child under age 1 on leave  Additional leave entitlements  Do leave entitlements to care for a dependent relative exist? of working parents  Do leave entitlements for personal reasons exist?  Duration of additional leave  Payments conditions during additional leave  Typology of childcare and  Characteristics of the Centre-based day-care early education services  Characteristics of the Family day care  Characteristics of the Pre-school early education programs  Quality of childcare workers  Main type of staff  Initial training requirements  Does continuous training exist?  Child – to – staff ratio 46 OECD, Pension  Old-age/Pension benefit  Gross pension replacement rates by earnings level (earnings level as multiple of the mean). The gross Database replacement rate is defined as gross pension entitlement divided by gross pre-retirement earnings  Net (of personal taxes and social contributions) pension replacement rates by earnings level (earnings level as multiple of the mean).  Gross pension replacement rates from public, mandatory private and voluntary private pension schemes  Gross pension wealth by earnings level. The gross pension wealth shows the size of the lump sum that would be needed to buy a flow of pension payments equivalent to that promised by the mandatory pension system in each country  Net (of personal taxes and social contributions) pension wealth by earnings level  Aggregate data on  Pension contribution rate (% of gross earnings) contribution rates  Pension contribution revenues (% of the GDP, by employee and employer)  Pension contribution revenues (% of total taxes)  Aggregate data on pension  Expenditures on old-age and survivors’ benefits (% of GDP) expenditure  Expenditures on old-age and survivors’ benefits (% of government spending)  Value of pension founds  Assets value in private pension funds (% of GDP)  Assets value in public pension reserves (% of GDP) 47 OECD, Sickness  Disability benefit  Disability benefit recipients in percent of the population aged 20-64 Disability and Work  Disability benefit recipients in percent of the population aged 20-64 56    Tool Social Protection Issue Indicators/Questions (project)  Disability employment  Employment rates of the working-age population with disability 57    Social Protection Discussion Paper Series Titles No. Title 1119 Measuring Governance and Service Delivery in Safety Net Programs by Gloria M. Rubio, September 2011 (online only) 1118 Assessing Safety Net Readiness in Response to Food Price Volatility by Margaret Grosh, Colin Andrews, Rodrigo Quintana, Claudia Rodriguez-Alas, September 2011 1117 Social Safety Nets in Fragile States: A Community-Based School Feeding Program in Togo, August 2011 (also available in French) 1116 Strengthening Governance of Social Safety Nets in East Asia by Sara Giannozzi and Asmeen Khan, August 2011 (online only) 1115 International Portability of Health-Cost Coverage: Concepts and Experience by Martin Werding and Stuart McLennan, July 2011 (online only) 1114 Liberia’s Cash For Work Temporary Employment Project: Responding to Crisis in Low Income, Fragile Countries by Colin Andrews, Prospère Backiny-Yetna, Emily Garin, Emily Weedon, Quentin Wodon and Giuseppe Zampaglione, July 2011 1113 Employability and Productivity among Older Workers: A Policy Framework and Evidence from Latin America by Edmundo Murrugarra, July 2011 (online only) 1112 Cash Transfers, Children and the Crisis: Protecting Current and Future Investments by Ariel Fiszbein, Dena Ringold, Santhosh Srinivasan, June 2011 (online only) 1111 Severance Pay Programs around the World: History, Rationale, Status, and Reforms by Robert Holzmann, Yann Pouget, Milan Vodopivec and Michael Weber, May 2011 (online only) 1110 Portability of Pension, Health, and other Social Benefits: Facts, Concepts, Issues by Robert Holzmann and Johannes Koettl, May 2011 (online only) 1109 Disability and Poverty in Developing Countries: A Snapshot from the World Health Survey by Sophie Mitra, Aleksandra Posarac and Brandon Vick, April 2011 1108 Advancing Adult Learning in Eastern Europe and Central Asia by Christian Bodewig and Sarojini Hirshleifer, April 2011 (online only) 1107 Results Readiness in Social Protection & Labor Operations by Laura Rawlings, Maddalena Honorati, Gloria Rubio and Julie Van Domelen, February 2011 1106 Results Readiness in Social Protection & Labor Operations: Technical Guidance Notes for Social Service Delivery Projects by Julie Van Domelen, February 2011 1105 Results Readiness in Social Protection & Labor Operations: Technical Guidance Notes for Social Safety Nets Task Teams by Gloria Rubio, February 2011 1104 Results Readiness in Social Protection & Labor Operations: Technical Guidance Notes for Social Funds Task Teams by Julie Van Domelen, February 2011 1103 Results Readiness in Social Protection & Labor Operations: Technical Guidance Notes for Labor Markets Task Teams by Maddalena Honorati, February 2011 1102 Natural Disasters: What is the Role for Social Safety Nets? by Larissa Pelham, Edward Clay and Tim Braunholz, February 2011 1101 North-South Knowledge Sharing on Incentive-based Conditional Cash Transfer Programs by Lawrence Aber and Laura B. Rawlings, January 2011 1008 Social Policy, Perceptions and the Press: An Analysis of the Media’s Treatment of Conditional Cash Transfers in Brazil by Kathy Lindert and Vanina Vincensini, December 2010 (online only) 1007 Bringing Financial Literacy and Education to Low and Middle Income Countries: The Need to Review, Adjust, and Extend Current Wisdom by Robert Holzmann, July 2010 (online only) 1006 Key Characteristics of Employment Regulation in the Middle East and North Africa by Diego F. Angel-Urdinola and Arvo Kuddo with support from Kimie Tanabe and May Wazzan, July 2010 (online only) 1005 Non-Public Provision of Active Labor Market Programs in Arab-Mediterranean Countries: An Inventory of Youth Programs by Diego F. Angel-Urdinola, Amina Semlali and Stefanie Brodmann, July 2010 (online only) 1004 The Investment in Job Training: Why Are SMEs Lagging So Much Behind? by Rita K. Almeida and Reyes Aterido, May 2010 (online only) 1003 Disability and International Cooperation and Development: A Review of Policies and Practices by Janet Lord, Aleksandra Posarac, Marco Nicoli, Karen Peffley, Charlotte McClain-Nhlapo and Mary Keogh, May 2010 1002 Toolkit on Tackling Error, Fraud and Corruption in Social Protection Programs by Christian van Stolk and Emil D. Tesliuc, March 2010 (online only) 1001 Labor Market Policy Research for Developing Countries: Recent Examples from the Literature - What do We Know and What should We Know? by Maria Laura Sanchez Puerta, January 2010 (online only) 0931 The Korean Case Study: Past Experience and New Trends in Training Policies by Young-Sun Ra and Kyung Woo Shim, December 2009 (online only) 0930 Migration Pressures and Immigration Policies: New Evidence on the Selection of Migrants by Johanna Avato, December 2009 (online only) 0929 Ex-Ante Methods to Assess the Impact of Social Insurance Policies on Labor Supply with an Application to Brazil by David A. Robalino, Eduardo Zylberstajn, Helio Zylberstajn and Luis Eduardo Afonso, December 2009 (online only) 0928 Rethinking Survivor Benefits by Estelle James, December 2009 (online only) 0927 How Much Do Latin American Pension Programs Promise to Pay Back? by Alvaro Forteza and Guzmán Ourens, December 2009 (online only) 0926 Work Histories and Pension Entitlements in Argentina, Chile and Uruguay by Alvaro Forteza, Ignacio Apella, Eduardo Fajnzylber, Carlos Grushka, Ianina Rossi and Graciela Sanroman, December 2009 (online only) 0925 Indexing Pensions by John Piggott and Renuka Sane, December 2009 (online only) 0924 Towards Comprehensive Training by Jean Fares and Olga Susana Puerto, November 2009 0923 Pre-Employment Skills Development Strategies in the OECD by Yoo Jeung Joy Nam, November 2009 0922 A Review of National Training Funds by Richard Johanson, November 2009 0921 Pre-Employment Vocational Education and Training in Korea by ChangKyun Chae and Jaeho Chung, November 2009 0920 Labor Laws in Eastern European and Central Asian Countries: Minimum Norms and Practices by Arvo Kuddo, November 2009 (online only) 0919 Openness and Technological Innovation in East Asia: Have They Increased the Demand for Skills? by Rita K. Almeida, October 2009 (online only) 0918 Employment Services and Active Labor Market Programs in Eastern European and Central Asian Countries by Arvo Kuddo, October 2009 (online only) 0917 Productivity Increases in SMEs: With Special Emphasis on In-Service Training of Workers in Korea by Kye Woo Lee, October 2009 (online only) 0916 Firing Cost and Firm Size: A Study of Sri Lanka's Severance Pay System by Babatunde Abidoye, Peter F. Orazem and Milan Vodopivec, September 2009 (online only) 0915 Personal Opinions about the Social Security System and Informal Employment: Evidence from Bulgaria by Valeria Perotti and Maria Laura Sánchez Puerta, September 2009 0914 Building a Targeting System for Bangladesh based on Proxy Means Testing by Iffath A. Sharif, August 2009 (online only) 0913 Savings for Unemployment in Good or Bad Times: Options for Developing Countries by David Robalino, Milan Vodopivec and András Bodor, August 2009 (online only) 0912 Social Protection for Migrants from the Pacific Islands in Australia and New Zealand by Geoff Woolford, May 2009 (online only) 0911 Human Trafficking, Modern Day Slavery, and Economic Exploitation by Johannes Koettl, May 2009 0910 Unemployment Insurance Savings Accounts in Latin America: Overview and Assessment by Ana M. Ferrer and W. Craig Riddell, June 2009 (online only) 0909 Definitions, Good Practices, and Global Estimates on the Status of Social Protection for International Migrants by Johanna Avato, Johannes Koettl, and Rachel Sabates-Wheeler, May 2009 (online only) 0908 Regional Overview of Social Protection for Non-Citizens in the Southern African Development Community (SADC) by Marius Olivier, May 2009 (online only) 0907 Introducing Unemployment Insurance to Developing Countries by Milan Vodopivec, May 2009 (online only) 0906 Social Protection for Refugees and Asylum Seekers in the Southern Africa Development Community (SADC) by Mpho Makhema, April 2009 (online only) 0905 How to Make Public Works Work: A Review of the Experiences by Carlo del Ninno, Kalanidhi Subbarao and Annamaria Milazzo, May 2009 (online only) 0904 Slavery and Human Trafficking: International Law and the Role of the World Bank by María Fernanda Perez Solla, April 2009 (online only) 0903 Pension Systems for the Informal Sector in Asia edited by Landis MacKellar, March 2009 (online only) 0902 Structural Educational Reform: Evidence from a Teacher’s Displacement Program in Armenia by Arvo Kuddo, January 2009 (online only) 0901 Non-performance of the Severance Pay Program in Slovenia by Milan Vodopivec, Lilijana Madzar, Primož Dolenc, January 2009 (online only) To view Social Protection Discussion papers published prior to 2009, please visit www.worldbank.org/sp. Abstract This paper develops a framework to assess organizational performance in the delivery of social safety nets. Specifically, it provides guidance to task teams and program managers for identifying indicators of governance and service quality in targeted cash transfer programs. The paper identifies governance issues along the results chain of service delivery and suggests policy and performance indicators for assessing program inputs - human resources, financing and resource management; and program activities - operational procedures, MIS and control. It also suggests indicators of organizational performance and the quality of outputs, including demand-side accountability mechanisms. HUMAN DEVELOPMENT NETWORK About this series... Social Protection Discussion Papers are published to communicate the results of The World Bank’s work to the development community with the least possible delay. The typescript manuscript of this paper therefore has not been prepared in accordance with the procedures appropriate to formally edited texts. The findings, interpretations, and conclusions expressed herein are those of the author(s), and do not necessarily reflect the views of the International Bank for Reconstruction and Development / The World Bank and its affiliated organizations, or those of the Executive Directors of The World Bank or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work. For more information, please contact the Social Protection Advisory Service, The World Bank, 1818 H Street, N.W., MSN G7-703, Washington, D.C. 20433 USA. Telephone: (202) 458-5267, Fax: (202) 614-0471, E-mail: socialprotection@worldbank.org or visit us online at www.worldbank.org/sp.