The Use of Sectoral and Project Performance Indicators in ~ank-~inancedFinancial Sector Operations: Financial Sector Development Department March 1995 PREFACE This paper is part of a bank-wide exercise undertaken by the Central Vice-Presidencies to operationalize the recommendations of the Wapenhans Report. It is addressed to all Bank staff engaged in work in the Financial Sector, and outlines an approach for utilizing sectoral and project performance indicators in Bank-funded financial sector operations. The authors would like to thank Roberto Mosse and Nicolas Mathieu for their advice and support during the development of the paper as well as the members of the Quality Control Group for their comments on previous drafts. EXECUTIVE SUMMARY THE USE OF SECTORAL AND PROJECT PERFORMANCE INDICATORS IN BANK-FINANCED FINANCIAL SECTOR OPERATIONS 1. The Bank's portfolio of financial sector operations under implementation in FY 94 consisted of 98 operations with a share of 5.6 percent of Bank projects. Lending commitments amounted to US$10.23 billion, or 7.2 percent of total project commitments. The 1994 Financial Sector Portfolio Performance review found that projects were encountering increasing implementationproblems. Thepercentage of problemprojects identifiedand classified as such for the "Financial" portfolio has risen over the last three years; the same applies to the incidence of problem projects in the first 1-3 years of project implementation in the financial portfolio. While part of this trend is the result of closer supervision and better detection of problems early on in the implementation process, it also demonstrates the need to address sectoral management and project implementation issues in a more comprehensive way. .. 11. This note sets out performance indicators for financial loans, including financial intermediation loans (FILs), financial sector adjustment loans (FSALs), and other loans that include financial sector conditionality. It establishesa menu of key indicatorsfor easy reference that are of general applicability, and well as other indicators that may be of use in specific operations. This note should be treated as a complement, and not substitute, to OD 8.30, which provides specific guidance for financial sector operations. iii. Definitions. There are no universally accepted classifications of sector and project performance indicators in the financial sector. The indicators useful in financial operations are grouped in this note under three rubrics: Indicators measuring improvements in the financial sector (the basic indicators used in FSALS, EFSALs and Rehabilitation Loans). Indicators measuring improvements in the financial intermediaries through which the Bank on-lends to final borrowers. Measures of the intermediary's performance are the basic indicators for FILs, although some sector indicators likely will be relevant as well. Indicators applicable to the real sectors which are the object of FILs. iv. Financial Sector Data reflect the depth and role of the formal financial system, as well as the level of repression in the financial sector. The data report can be used to inform financial sector decision making and policy formulation, including value judgements. Sector-specificperformanceindicatorscan assist policy makers, task managers and staff carrying out sector work in: rationalizing a lending strategy, selecting project strategies, and providing a framework for monitoring project activities. Sectoral indicators (derived from sector work or available sector data) are especially useful from the project identification stage and appraisal stages to identify project objectives, and for subsequent evaluation. v. Financial Intermediary Data: Financial performance indicators measure the present financial condition and viability of the institutions, which are important in the determination of the incentive structure under which management and the board are operating. In appraising financial intermediaries, its abilities to undertake efficient operations and to make sound lending decisions are assessed. Further, it must be ensured that the financial intermediary complies with the country's relevant laws and regulations. The indicators of financial intermediary performance provide benchmarks against which to measure progress in financial strengthening efforts and are thus important to monitor the progress and evaluate the outcome of financial sector operations. Institutional Indicators reflect the development of the financial institution as a viable business organization. They relate to major issues of management and board responsibility. vi. Notes to staff similar to this paper address the specific real sectors that are the target of Financial Intermediary Loans (FILS), for example in agriculture and housing. This note provides additional indicators for operations that have the character of small and medium enterprise (SME))lending. A recent appraisal of Bank work in the area of small enterprise development over the past five years identified three principal types of operations with essentially three different sets of objectives: Free-standing SME projects with the objective of absorbing labor, contributing to economic growth and countering market imperfections in access to resources; Poverty loans which incorporate almost all of the Bank's work in the microenterprise and job creation area with the objective of poverty alleviation; and Private sector development loans with the objectives of improving the business environment for private investment and contributing to growththrough developing a more robust SME sector. Certain of the following performance indicators are applicable to all small credit and technical assistance programs, regardless of the size of the targeted enterprise. Others are specific to the objectives of the three types of operations listed above. The indicators listed in this note and the annexes are presented in this context. Table of Contents EXECUTIVESUMMARY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . - i - I. INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . -. 1 - Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 - Objectives and Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .- 1 - 11. ISSUES IN COLLECTION AND ANALYSIS OF FINANCIAL SECTOR DATA . . . . . - 2 - General Experience in Indicator use . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 . The Bank's Monitoring and Evaluation Experience . . . . . . . . . . . . . . . . . . . . . . . . . 2 . Enabling and Risk Factors -3- Purpose and Classification of Performance Indicators . . . . . . . . . . . . . . . . . . . . . . . 3 . Definitions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 . Macroeconomic and Financial Sector Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 . Basic Macroeconomic Data Financial Sector Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .4 . Interest Rates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .5 . Directed Credit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5. - Financial Soundness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6. - Regulation and Supervision . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .6 - Financial Intermediary Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7. - Financial Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 . Institutional Indicators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .8 - Real Sector Indicators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .8 - Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 . Institutional Framework. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 . Annex 1: Regulation and Supervision Indicators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . - 10 - Annex 2: Intermediary Institutional Indicators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .- 20 - THE USE OF SECTORAL AND PROJECT PERFORMANCE INDICATORS IN BANK- FINANCED FINANCIAL SECTOR OPERATIONS I. INTRODUCTION Motivation 1. The Bank's portfolio of financial sector operations under implementation in FY 94 consisted of 98 operations with a share of 5.6 percent of Bank projects. Lending commitments amounted to US$10.23 billion, or 7.2 percent of total project commitments. The 1994 Financial Sector Portfolio Performance review found that projects were encountering increasing implementation problems. The percentage of problem projects identified and classified as such for the "Financial" portfolio has risen over the last three years; the same applies to the incidence of problem projects in the first 1-3 years of project implementation in the financial portfolio. While part of this trend is the result of closer supervision and better detection of problems early on in the implementation process, it also demonstrates the need to address sectoral management and project implementationissues in a more comprehensive way. 2. OED findings of a gradually weakening portfolio performance in all Bank-financed projects were highlighted when the PorgolioManagement TaskForce Report was issued in October 1992 (the "Wapenhans Report"), followed in July 1993 by a management response entitled "Por@olio Management, Next Steps - A Program of Action (Next Steps). This paper is part of the Financial Sector Development Department's response to the issues raised by that report, which emphasized the need for monitoring and evaluation benchmarks in all projects to be derived from agreed sector and project objectives. It provides a menu for task managers useful for the now mandated Project Implementation Plan (see Bank OPIBP 10.00 Annex B). Objectives and Scope 3. The paper aims to guide Bank staff on the uses of basic, sectoral and project specificdata in order to better meet financial sector development goals as defined by both the Bank and its Clients. The paper is closely linked to several other FSD efforts to improve the quality of World Bank operations, such as the FSD Suggested Guidelines and Working Papers WPS 340 and 1034. 4. This note sets out performance indicators for financial loans, including financial intermediation loans (FILS), financial sector adjustment loans (FSALs), and other loans that include financial sector conditionality. It establishes key indicators that are of general applicability, and well as other indicators that may be of use in specific operations. This note should be treated as a complement, and not substitute, to OD 8.30, which provides specific guidance for financial sector operations. 11. ISSUES IN COLLECTION AND ANALYSIS OF FINANCIAL SECTOR DATA General Experience in Indicator use 5. Relevant financial sector indicators generally take the form of either quantitative assessments of data (often financial data and ratios) or qualitative assessments of the adequacy of policy actions, laws and regulations, institutional operating procedures and institutional development efforts. Bank staff with the requisite skills and experience in the financial sector are generally successful in making the required qualitative assessments. However, the veracity of financial data is dependent on the quality of both the accounting systems employed in recording and reporting financial transactions, and the control systems employed to ensuring adherence to accounting standards. Substantive weaknesses often exist in accounting and control systems in financial institutions and government agencies. It is not uncommon for weaknesses in financial institutions to be of systemic proportions in some countries. As such, making qualitative assessments can prove challenging, particularly with regard to assessing the financial condition of financial institutions). In part due to the nature of the challenge, Bank staff have not always been successful making valid assessments. Experience shows that it is necessary to identify and flag problem areas early on in identification of an operation. 6. Task managers and Bank management can utilize performance indicators as tools in their efforts to enhance the quality of projects entering the portfolio, and the performance of projects over their life. Performance indicators can be used as a means to illustrate the key project objectives, and can serve as measures of the extent to which objectives are achieved. Where possible, the indicators chosen should be quantitatively measurable, but in other cases, qualitative but testable indicators will be appropriate. Project performance cannot be measured solely in terms of the performance indicators set out in this note. Other indicators may be more appropriate in specific situations. Moreover, the indicators are no substitute for the sound judgement of skilled professionals in project preparation and supervision, and in the evaluation of project accomplishments. The Bank's Monitoring and Evaluation fiperience 7. The framework for M&E in the Bank can be found in Operational Directive 10.70 (September 1989) on "Project Monitoring and Evaluation" and 13.05 (March 1989) on "Project Supervision". Monitoring is defined as "the continuous assessment of project implementation in relation to agreed schedules and of the use of inputs, infrastructure, and services by project beneficiaries" (OD 10.70: paragraph 2) Evaluation is defined as "periodic assessment of the relevance, performance, efficiency, and impact (both expected and unexpected of the project in relation to stated objectives. " (OD 10.70: paragraph 3) and three types of evaluation,-- interim, terminal and impact-- are defined. Supervision reports are expected to present Key Performance Indicators (KPIs) for each project and provide tabular evidence of quantitative measures of project performance. (OD 13.05 Amex D4: paragraph 6). These indicators are not to exceed one page and are to be based on the Staff Appraisal Report. 8. Monitoring and Evaluation indicators aresometimesincorporatedinto legaldocumentation k d are used at mid-term reviews of project to determine their future course. Information on sub-projects and on indicators for data collection may be incorporated into a working document and placed in project files, or may be held by the Client. In either case, such information is necessary for supervision. Enabling and Risk Factors 9. From project identificationonwards, it is important to assess enabling and risk factors for project success. These can include a wide range of issues taken from previous Bank experience in a given country, such as institutional and project management capacity, as well as political and budgetary constraints. Given the close relationship between financial reform and stabilization objectives, the evaluation of enabling and risk factors is especially important to achieve the appropriate mix and sequencing of macroeconomic and financial sector policy reforms, improvements in financial infrastructure, and strengthening of specific financial institutions. The FY 94 Departmental Portfolio Reviews indicate the prevalence of macroeconomic disequilibria as one of the major obstacles for disbursing adjustment operations. One recent example was the long awaited CFA franc devaluation of January 1994, which had a particularly beneficial impact on the release of pending adjustment operations in CFA member countries. However, general macroeconomic instability and the associated fiscal difficulties often have broad-reaching implications for the performance of investment operations. 10. Inadequate sectoral policy frameworks are also a major source of poor portfolio performance. The 1989Financial Sector OperationsTask Force, which set out Operational Directive 8.30 of February, 1992, had among its objectives the: (i) improvement of the coordination between financial sector reforms, the need for restructuring financial institutions and the requirements of macroeconomic adjustment; and (ii) better integration of general sector policy work with financial intermediary operations. While these objectives are usually kept very much in mind in the country assistancestrategies, individual operations must also be in compliance with the basic requirements of the OD 8.30. For example, in the case of FILs, the Bank should, if necessary, "evaluate the operations and financial condition of participating FIs and visit the FIs and the sub-borrowers." Purpose and Classification of Performance Indicators 11. Detiitions. There are no universally accepted classifications of sector and project performance indicators in the financial sector. The indicators useful in financial operations are grouped in this note under three rubrics: o Indicators measuring improvements in the financial sector (the basic indicators used in FSALs, EFSALs and Rehabilitation Loans). o Indicators measuring improvements in the financial intermediaries through which the Bank on- lends to final borrowers. Measures of the intermediary's performance are the basic indicators for FILs, although some sector indicators likely will be relevant as well. o Indicators applicable to the real sectors which are the object of FILS. 12. Applying the classificationof indicators in policy, intermediateand outcome indicators is difficult in financial sector operations. Indicators measuring the improvements in the financial sector can be classified as policy indicators when they are used to monitor specific financial sector policy actions which were defined as the objectives of financial sector operations during appraisal. At the same time, however, financial sector indicators are often used as proxy or leading intermediate indicators to measure the impact of policy reforms on macroeconomic balance and structural change. Measures of the financial intermediaries' performance are more directly linked to the individual operation and can be classified both as policy and intermediate indicators depending on individual project design as well as on the stage of project implementation. Indicators applicable to the real sectors are useful to identify at the appraisal stageto determine the key objectives and as outcome indicators to later monitor the longer term impact of financial sector operations since the speed and efficiency of growth of the real sectors are intimately linked to the efficient development of the financial sector. Because of imprecise definitions and changes in definitions depending on context, these groupings are useful only as aids to the discussion. It is, however important in each project to differentiate between macroeconomic, sector-wide data and data specific to individual financial intermediaries. Ideally international standardizedmeasures should be utilized at national sectoral level, and reflected in the choice of indicators at the project level, to monitor change over time. However, in reality such standardization does not exist. Individual projects also need a wider range of indicators to reflect specific project goals and circumstances. Project specific indicators are closely tailored to project objectives and therefore cannot be standardized. Macroeconomic and Financial Sector Data 13. Basic Macroeconomic Data are the country-specific economic indicators by which system-wide changes are measured. Macro economic stability is essential to an environment in which economic decisions lead to efficient resource allocation. Lacking macro economic stability, price signals are distorted, savers are less willing to hold domestic financial assets, and investment decisions become focused on the protection of the real value of assets. Financial sector reform measures can themselves affect macroeconomic stability, particularly through their impact on the fiscal budget and monetary policies. For example, interest rate reform may reduce fiscal subsidies but increase the cost of servicing the public debt. The restructuring of financial institutions may result in large immediate costs to the budget depending on how recapitalizations are financed, but they may reduce the need for budgetary or central bank support to the financial system in later years. The speed and phasing of financial sector reform must, therefore, be mindful of the macroeconomic and incentive environment. In advance of any financial sector operation, an explicit assessment should be made of the conduciveness of the country's macroeconomic environment in realizing the operation's goals. 14. Key Indicators for the basic macroeconomic analysis are the inflation rate (WPI, CPI), the consolidated government and central budget deficitIGDP, the market-based value of exchange ratelofficial exchange rate, and the current account deficit1GDP. Other Us&l Indicators such as the extent of exchange controls, and the level of direct foreign investment, may provide indicationsof macro economic stability in certain financial sector operations. 15. Financial Sector Data reflect the depth and role of the formal financial system, as well as the level of repression in the financial sector. The data report can be used to inform financial sector decision making and policy formulation, including value judgements. Sector-specific performance indicators can assist policy makers, task managers and staff carrying out sector work in: rationalizing a lending strategy, selecting project strategies, and providing a framework for monitoring project activities. Sectoral indicators (derived from sector work or available sector data) are especially useful from the project identification stage and appraisal stages to identify project objectives, and for subsequent evaluation. 16. Key Indicators characterizing the financial sector are the level of financial assets (MJGDP), the ratio of deposits greater than six months to total deposits, the interest spread between loans and deposits, the ratio of assets in privately owned banks to total bank assets, the level of reserve requirements and interest rates paid on reserves. Other Useful Indicators that may be helpful in certain operations include the depth of the interbank and money markets, the depth of the government paper market, the market capitalizationof stock markets, stock exchange turnover, the extent of the role of non- bank financial institutions (e.g. level of assets in privately owned institutions), the level of financial sector concentration, entry and exit conditions in the banking system, the volume of bank funding derived from the central bank, the independence of the central bank, the efficiency of payments systems, the extent of mandatory portfolio requirements and interest rates paid on such mandatory investments, the level of financial sector taxation, and the level of access by small clients to the formal financial system. 17. Interest Rates. The level and structure of interest rates have crucial implications for the development of the financial sector and for the mobilization and allocation of financial resources throughout the economy. The level of interest rates can influence the extent of resource mobilization and the rationality of investment decisions. A structure of interest rates that does not properly reflect differences in risks, liquidity and transaction costs may reduce total financial savings and encourage their suboptimal use. Interest Rates are thus a central concern of the Bank's economic dialogue with borrowing countries. An analysis of interest rates and, as necessary, a program for their correction, are essential elements in the formulation of financial sector strategies and, hence, the Bank's ability to undertake financial sector operations. A description of the interest rate regime, the relative proportions of the system covered by market and nonmarket regimes, and the resulting level and structure of interest rates in the economy should be included in the documentation for all financial sector operations. 18. Key Indicators are the level of real interest rates (deposit ratelinflation rate), interest rate spreads (prime loan rateldeposit rate for 6 and 12 month terms), the structure of interest rates (short termllong term deposit rates, non-prime lending ratelprime lending rate, term and interest rate on longest commercial instrument, etc.) and the preponderance of market determined interest rates.Measures of lending interest rates should include fees and commissionsto the extent they cannot be attributed to direct origination costs. Other Useful Indicators include other measures of spreads, margins and the shape of the yield curve. 19. Directed Credit. Credit is frequently directed by governments to specific end uses through a variety of techniques. In many developing countries, poor information, unfamiliarity with modern financial instruments, and lack of lender experience may limit market response to existing opportunities and demands made by financial sector reforms. Directed credit can be used to overcome market imperfections, thus opening market access to previously excluded groups and sectors. At the same time, it can inhibit the growth of market finance, and lead to unproductive investment, especially if policy and institutional obstacles exist, such as interest rate controls, inadequate legal protection of claims, and discriminatory real sector policies that reduce borrowers' creditworthiness. Financial sector strategies should evaluate the importance of directed credit programs in the country and their consistency with sound financial sector development. 20. Key Indicarors in this area are the number of government directed credit programs, the flow and volume of credit in directed programs in relation to total credit flow and volume, and the ratio of the typical interest rate on directed credit to prime lending rate.Other Useful Indicators might include the distribution of total credit by sector (government, public enterprise and private enterprise), and the specificity of targeted beneficiaries, where targeting sectors should be viewed as preferable to targeting specific borrowers or classes of borrowers. 21. Financial Soundness: Financial institutions in many countries have been severely weakened by varying combinations of macroeconomic instability, distorted financial sector policies, poor supervision, the impact of adjustment programs on major borrowers, political interference in their operations, and poor management. Financial sector reform and development depends on healthy institutions with the capacity to respond to the changingpolicy framework, evolving economicconditions, and growing competition. Lacking financial soundness, new resource allocation by the financial sector is likely to be devoted toward accommodating borrowers with substantial debt arrears. Such resource misallocation will hamper economic recovery and growth. The potential collapse of sizeable financial institutions can damage the credibility of the financial system and impair resource mobilization. 22. Key Indicators to assess financial soundness include the ratio of total assets in insolvent banks to total bank assets, the ratio of non-performing bank assets to total bank assets, the ratio of total assets or liabilities of other insolvent financial institutions to total assets or liabilities of such institutions, the return on equity of financial sectors and the financial system. Other Useful Indicators might include the level of capital and reserves, the adequacy of loss identification and provisioning mechanisms, and timeliness with which action is taken to resolve failing institutions. 23. Regulation and Supervision: Financial development relies heavily on the confidence of savers and financial institutions in the fundamental quality of financial assets available to them. Competition is important both to innovation in financial services and to overall financial market efficiency. At the same time, however, financial institutions are especially vulnerable to loss of public confidence, to imprudent behavior of their managers, and macroeconomic stability can be jeopardized by failure of major financial institutions. Notwithstanding the importance of competition, financialsector liberalization must take place within a strong prudential regulatory framework. Effective regulation and supervision of financial markets and institutions are important in promoting the stable provision of finance and the sustained growth in resource mobilization. Documentation for financial sector operations should contain information on the qualities of competition, regulation and supervision in the sector as well as on their implications for the design and speed of reforms. 24. Key Indicators in the area of regulation and supervision are the capacity and legal authority to supervise financial institutions, suitable criteria and systems for licensing, prudential regulations consistent with international standards, adequate systems for off-site and on-site supervision, enforcement powers and implementation, and resolution of problems in troubled intermediaries. Since these key indicators are for the most part not quantitatively measurable, qualitative assessment factors are set out in Annex One. Financial Intermediary Data 25. Financial Performance: Financial performance indicators measure the present financial condition and viability of the institutions, which are important in the determination of the incentive structure under which management and the board are operating. In appraising financial intermediaries, its abilities to undertake efficient operations and to make sound lending decisions are assessed. Further, it must be ensured that the financial intermediary complies with the country's relevant laws and regulations. The indicators of financial intermediary performance provide benchmarks against which to measure progress in financial strengthening efforts and are thus important to monitor the progress and evaluate the outcome of financial sector operations. 26. Key Indicators of financial performance are the Performance of Credit L i e Operations: Collection Experience of On- sufficiency of capital (solvency lending Institutions or capital adequacy ratio, as per local prudential regulations), the An important indicator in the design and monitoring of credit line operations interest spread between loans is the loan collection experience of on-lending institutions. This indicator is and deposits, the quantity and particularly useful in assessing the achievement of the development objective of credit line operations. There is no strict one-to-one relationship between quality of earnings (net the profitability and the repayment of a loan as businesses might or might not incornlaverage equity, net repay loans independent of the profit or losses they make on the loan. As an incomeltotal average assets, net empirical matter, however, it appears that loan profitability and repayment operating incomeltotal gross are linked. Borrowers who expect to have to repay their loans seem to be income), the quality of assets more careful in their choice of projects than those who do not expect to (non-performing assetsltotal repay. Not having to repay loans may lead to capital misallocation, since the borrower will make money even from socially unprofitable projects. This assets, restructured assets/total broad empirical association between loan collection and efficient capital assets, charge-offsltotal assets), allocation supports the concern of the Bank in improving bank supervision the sufficiency of liquidity and regulation in its borrowing member countries. (loansldeposits, short term assetslshort term liabilities, The 1994 Financial Sector Portfolio Review, however, showed that in over gross borrowingslcash plus 40 credit line operations, only two reported on whether loans made under the credit were performing. In 22 SME operations, only two quantified the money market assets plus collection experience of the on-lending institution and those were the only two securities with a residual that reported on whether collection performance was good or bad. Of the 23 maturity of less than 12 months) restructuring projects, 12 had credit line components: none reported on the and the Subsidy Dependency performance of these credit line components. Of three privatization projects Indexy. Other Useful Indicators with credit line components, none asked whether the onlent credit lines were performing. Welldesigned and well-supervised lending operations should in assessing the performance of monitor the collection experience of the loan portfolio. intermediaries might include the extent of concentration (geographic, sectoral and client), income sensitivity (levels of interest rate risk, foreign exchange rate risk and position, or market, risk), operating efficiency, and rates of growth. -11For moredetailed instructionsonthe preparationofthe index, see ManagementNote, May 4. 1993: "Useof the Subsidy Dependency Index in Assessing the F i c i a l Performance of Development Finance Institutions"and Bank Discussion Paper 174. 27. Institutional Indicators reflect the development of the financial institution as a viable business organization. They relate to major issues of management and board responsibility. Key Indicators are the independence and capacity of the Board, risk management procedures, budgeting and capital planning, management information systems, use of Internationally Accepted Accounting Principles, annual audits by qualified firms, and programs for human resource development. Since these key indicators for the most part are not quantitativelymeasurable, qualitative assessment factors are set out in Annex Two. Real Sector Indicators 28. Notes to staff similar to this paper address the specific real sectors that are the target of Financial Intermediary Loans (FILS), for example in agriculture and housing. This note provides additionalindicators for operationsthat have the character of small and medium enterprise (SME)lending. A recent appraisal of Bank work in the area of small enterprise development over the past five years identified three principal types of operations with essentially three different sets of objectives: Free-standing SME projects with the objective of absorbing labor, contributing to economic growth and countering market imperfections in access to resources; Poverty loans which incorporate almost all of the Bank's work in the microenterprise and job creation area with the objective of poverty alleviation; and Private sector development loans with the objectives of improving the business environment for private investment and contributing to growth through developing a more robust SME sector. Certain of the followingperformance indicators are applicable to all small credit and technical assistance programs, regardless of the size of the targeted enterprise. Others are specific to the objectives of the three types of operations listed above. The indicators listed below are presented in this context. 29. Key Sectoral Indicators include the number of beneficiaries of creditITA loans, the average loan size, the number of jobs created and the cost per job created, repayment rates, and the level of demonstrated demand to TA programs. Key Institutional Indicators would be the profit or loss on the operation, the share of portfolio accounted for by the target group, increased capacity to lend to and service the target group, the level of concentration of market shares among available lenders. With respect to the three types of operations listed above, the following Key Indicators Specific to Distinct Types of Loam can be identified. For SMEs, indicators would include the share of the sector in national output and employment, the share of the sector in aggregate credit extended, as well as the competitiveness of the sector. In poverty loans, the impact of the operation on national or regional levels of poverty and income must be measured, as well as the impact on national or regional levels of employment. Also, the efficiency of emergency social programs' operationsmust be assessed. Important indicators for loans for the development of the private sector are the level of private sector investment, and the number and depth of reforms implemented. Implementation 30. Institutional Framework. The importance of: (a) monitoring as a management tool for project implementing staff; and (b) the institutional responsibilitiesfor monitoring and evaluation should normally be addressed duringproject preparation. Subsequentlythe appraisal team should weigh: (a) the adequacy of the proposed monitoring and evaluation system; and (b) the appropriateness of separating the entity responsible for evaluation from the implementing agency. While there are no hard and fast rules on this score, the ability of the implementing agency to critically evaluate its own work must be a deciding factor. During the implementation phase, supervision missions should coach Clients in the use of monitoring as a management tool to collect data, track project performance, and analyze and evaluate the results. This task will become more transparent in future projects once the now mandated Project Implementation Plan becomes more commonplace in SARs. (see Bank OPIBP 10.00 -- Annex B). 31. The Financial Sector Portfolio Reviews often show significantdiscrepancies between the achievement of development objectives and the implementation progress. Part of the explanation may be the different speed at which different components can move. Particularly there seems to be limited awareness that some technical assistance components and institutional development efforts (such as strengthening the supervisory framework of Central Banks) are not going to be realized within the often unrealistic time frames initially planned. In other instances (such as institution building or technical assistance loans) the blame for limited progress in implementation is often attributed to the weak project management capacity of governmental institutions, the constant change of plans and personnel, and the lack of familiarity with Bank procedures and processes. 32. Efficient implementation requires that a time-table for monitoring project activities be set up that: prioritizes project activities, identifiesthe monitoring indicators to be used for measurement; sets dates for targets to be achieved; identifiesthe parties responsiblefor monitoring, and reports on the status of actions. Monitoring project progress is a task for implementing agencies and supervision missions, particularly where data collection and MIS capacity building are important components of the project. Project data must be physically available to supervision staff, as well as translated into English where necessary, and simple mechanisms such as the standardization of data reporting forms, where multiple institutions are involved, canensure more effectivesupervision. Incorporationof data collection into legal documents is a means of encouraging Client participation. ANNEX 1 REGULATION AND SUPERVISION INDICATORS 1. Capacitv to Supervise Financial Institutions Supervisor's Autonomy (See FSD Suggested Guideline 1and Working Paper WPS 340) Clear goals (i.e. promote systemicsoundness; promote soundnessof financial institution management; promote sound market practices) Supervisory authority established (e.g. for prior approval of entry, geographic expansion, expansion of activities, etc.; to establish of enforceable practice standards; to conduct inspections; to intervene in management; to remove management; to suspend and withdraw licenses) Sufficient degree of political independence (mandate, goals, budget, decision and policy making) Supervisor's operational budget not politically dependent, and sufficient to fund operations Supervisor's Resources (See FSD Suggested Guideline 1 and Working Paper WPS 340) Salaries sufficient to attract and retain skilled individuals (e.g. salaries equivalent to private sector salaries) Supervision staff of sufficient numbers and skills to assure adequate coverage of supervisory objectives Institutionalized career development program for supervisory staff Institutionalized training program which ensures updated technical skills Communicates clearly the supervisory objectives to financial institution managers and routinely seeks their views. Supervisory Approach (See FSD Suggested Guideline 1and Working Paper WPS 340) Promotes soundnessthrough preventative action by addressing financial and institutional weaknesses prior to crises Follows a top-down approach to assess quality of management and risk management processes Ensures continuous monitoring of financial institutions, with intensity reflective of their risk profile Includes adequate mix of supervisory means (e.g. on-site, off-site, management discussions, auditors' meetings) Mix assures current view of capital adequacy, asset quality, management quality, earnings, and liquidity Acceptable access to and communication with auditors (e.g. review of audit programs and working papers, meetings on core issues) Formal, periodic meetings with management to assess strategy and plans, and review outstanding issues and plans of corrective action Supem'sory Strategy & Goals (See FSD Suggested Guideline 1and Working Paper 340) Explicit annual plan for supel-visory purposes (including annual plan of full scope and targeted inspections) Operational objectives address transparency of financial condition, soundness of risk management process and financial practices, long term viability of financial institutions and sufficiency of operational cash flow and net income, systemic risk considerations, intermediation efficiency, and fairness in competition Financial institutions' risk rating is the key element used to assign supervisory resources and mix of tools 2. Lwal Capacitv of Authoritv to Supervise Financial institutions Supervisory Authority (See FSD Suggested Guideline 1) Legal mandate is broad in scope and provides supervisors with flexible powers and authority Powers include the ability to: Specify what constitutes sound financial practices, Define minimum standards for financial institutions' risk management programs, Establish minimum scope, extension and reporting standards for auditing, Communicate with external auditors and have access their working papers, Specify detailed regulatory accounting practices (RAP), Specify financial institutions' financial reporting requirements, Inspect financial institutions and subsidiaries, Obtain information regarding affiliates, insiders, and their related interests Enforcement Powers (See FSD Suggested Guideline 13) 12 Powers include ability to: Request specialized diagnosticslaudits, Suspend or disqualify auditors, Prescribe preventive time bound action plans Require directors and managers to cease unsound and unsafe practices Restrict activities, limit expansion or growth, and condition them on capitalization plans, Restrict activitiessubjecttomanagement developingappropriate riskmanagementprograms, Constitute necessary provisions, and limitlsuspend dividends, Assess penalties according to the gravity of the offense, Tailor sanctions according to risk posed to financial stability, depositors and creditors, Order related parties to restitute financial gains, Order related parties to compensate financial damage to the financial institution or third parties, Remove or suspend imprudentlunsound directors and management, Intervene financial institutions' management and order closure and liquidation of financial institutions. 3. Suitable Criteria and Svstems for Licensing Licensing Mechanism (See Working Paper WPS 1034) Scope is broad (i.e. new financialinstitutions, change in control, new activities, corporate structure, change in management) Grants power to deny license based on subjective criteria (i.e. suitability) Grants the authority to prescribe authorization criteria, information needs and minimum entry capital requirement (MEC) MEC compatible with type of license and permissible activities Supervisor has flexibility to augment MEC according to projected level of business MEC is constituted entirely in cash Screening process includes test of sources of capital and related debt service requirements Licensing of directors and managers subject to is "fit and proper" test (i.e. an assessment of integrity, experience, and capacity) Screening includes assessment of veracity and reasonableness of stated business plan Screening includes assessmentof ability to establish efficient risk management programs and internal controls Supervisor can impose specific limitations during first years of new institution Transfer of shares of existing institutions is subject to supervisor's scrutiny (e.g. prior approval, especially for banks) at defined levels for significant controlling-interest (20%) and subsequent changes (33930%) Supervisor informed of acquisitions of significant participations and subsequent increases (with prior approval requirements for substantial investments) Shareholders, directors and managers obligedto disclose information relating to ownership interests in financial institutions Supervisor can ultimately deny licensing based on "market needs and competition" test Permissible Activities (See working paper WPS 1034) New activities are conditioned to existence of adequate processes, systems and necessary capital New activities are controllable by directors on a consolidated basis Organization of new activities permits supervision on a consolidated basis Supervisor can order termination of activities if not properly backed by management processes, systems and capital Board Responsibilities (See FSD Suggested Guideline 12) Financial institution laws expand corporate law fiduciary responsibilities of directors, including specifically to: Select, retain, compensate, and dismiss senior management, Establish performance criteria to assess management's effectiveness, Approve basic policies, plans and procedures, including business plans and annual budgets, Establish clear delegation of authority and risk limits, Provide for an effective internal audit function, Monitor management's and financial institution's compliance with laws, regulations, and 14 internal policies, Directors have legal liability for their actions/omissions Organizational structure of the financial institution and of management is conducive to oversight by directors Directors have established committees of non-executive (outside) member to deal with ethics issues, auditing, and conflict resolution Board required to have a majority of outside directors 4. Prudential Regulations Consistent with International Standards C q h l Adequacy (See FSD Suggested Guideline 9) Capital adequacy regulation for banks is compatible with BIS standards in respect to asset and off- balance sheet risk weights, definition of regulatory capital, and minimum ratio (8%) Capital adequacy regulation for investment firms is compatible with EEC capital adequacy directive standards or other recognized major market capital rules Supervisors have discretion to request additional capital based on qualitative assessments Deterioration in capital adequacy triggers automatic enforcement & resolution actions Capital adequacy regulation is applied on a group-wide basis for financial conglomerates (i.e. groups of related financial institutions), including provisions to preclude the double-counting of capital, and the overstatement of capital due to intra-group transactions Lending and Large Exposure Limits (See FSD Suggested Guideline 7) Law or regulation includes comprehensive definition of risk exposures and comprehensive combination rules for groups of related borrowers Individual lending limits are less than or equal to 25% of capital Combined large exposures (those exceeding 10%of capital) are less than 8 times capital Excess over limits automatically attract an additional capital requirement Lending and large exposure limits are applied on a group-wide basis for groups of financial institutions Investment Limits Operational fixed assets cannot exceed 100%of regulatory capital Other real estate repossessed or owned subject to sub-limits and time bound divesture regulations; Holdings of equities in individual non-financial enterprises cannot exceed 15% of capital, and aggregate holdings cannot exceed 6096, unless the enterprise is fully consolidated with the financial institution, or the holding in excess of the limit is deducted for capital adequacy calculations Cross-share participations among financial institutions forbidden unless the institutions are fully consolidated or the investments are deducted for capital adequacy purposes Insider Lending (See FSD Suggested Guidelines 5 and 7) Insiders defined to include owners of 5% or more of the financial institution's shares, directors and senior managers, and the related interest of these parties Related parties defined to include direct & indirect significant interests (10% or more) in corporations, and family members up to second degree on consanguinity Lending to insiders subject to prior board authorization, and is never at concessionary terms Lending collateralized by the financial institution's own shares is forbidden Lending to enterprises where the financial institution's sharesldebentures are significant part of the assets is forbidden Foreign Exchange Limits (See FSD Suggested Guideline 11) Comprehensive definition of positions (to include spot and forward transactions, where forward positions include off-balance sheet contingencies) Differentiation among structural and tradingldealing positions Required reporting Overall net foreign exchange open position (larger of the sum of all short and of all long positions) customarily < 5% to 10% of capital Individual foreign currency position customarily < 5% to 10% of capital Higher limits require specific authorization based on quality of management, policies and procedures Short positions forbidden for unexperienced financial institutions 5. Adeauate Svstems for OfPSite Supervision Regulatory Accounting Practices (See FSD Suggested Guidelines 3, 6 and 8) Supervisor is empowered to set regulatory accounting practices (RAP) RAP and GAAP are consistent with international accounting standards @AS) RAP supports accurate disclosure of financial condition and outlook of business operations RAP includes graduated asset classification system (ACS), a requirement for the establishment of adequate provisions for losses, and rules for the suspension of the accrual of interest on non- performing assets ACS is applied to off-balance sheet risks and contingencies ACS focuses on the actual and prospective capacity of debtors to service debts out of operational cash-flow ACS set minimum specific loan provisions based on severity of classification and probability of loss consistent with FSD Suggested Guidelines ACS sets minimum generic loan provisions for assets not classified (over 1%,according to economic environment) ACS prohibits direct or indirect financing of arrears with additional lending RAP specify valuation standards for fixed assets and requires financial institutions to develop real estate appraisal procedures RAP specify valuation standards for equities based on fair economic value RAP specify valuation standards for portfolios of financial instruments followingIAS 40 RAP mandate translation of foreign currency via profit and loss statement RAP specify troubled debt restructuring and valuation rules for debt-to-equity and debt-todebt swaps; Ertemal Auditing (See FSD Suggested Guideline 4) External audit program has adequate scope and extension of review Supervisor maintains an adequate relationship with auditors and has access to working papers Supervisor has established a complete long audit report with comprehensive coverage Auditor has explicitly to sign-off on the adequacy provisions and reported net income 17 Auditor has explicitly to report on the adequacy of internal controls and reporting Audit is on a consolidated basis for financial conglomerates Auditor has to report adjustments and reclassification over a threshold level Supervisor can request additional review work by auditors Financial institution's relationship with the external auditor is channeled through a board-level audit committee Off-siteMonitoring System (See FSD Suggested Guideline 3) Supervisor is empowered to establish required reporting for supervisory purposes Reporting to supervisor includes periodic information in terms of: On-balance and off-balance sheet activity and income performance Dimensions of credit risk exposure, quality, distribution, and concentration Dimensions of foreign exchange and interest rate risk exposures, and liquidity position Systems able to support peer and trend analysis Peer groups allow for comparative norms of performance Trend analysis provides at least 4 years information System calculates spreads, margins, average cost and income rates, and measures of efficiency System's early warning capabilities include at least a screening filter for detection of outliers System provides for tracking large exposures of most important borrowers System supports supervisor's management information system (MIS), including qualitative assessments and rating assigned 6. Adecluate Systems for On-Site Suvervision Staffing (See working paper 340) Examiners are well trained, skilled and remunerated Teams are properly staffed and have available sufficient time and adequate access to records Assigned examination staff ensures adequate coverage under command of examiner in charge 18 Inspection Process (See working paper 340) Examination teams focus on core risk areas with adequate scope, extension, and testing Examination objectives focus on assessing financial condition and risk management Uses a topdown approach (systems and risk management programs are assessed simultaneously) Examination cycle ensures full scope examination each 18to 24 months Portion of total credit risks reviewed equal or exceed 60% of risk assets and off-balance sheet exposures Generally, all individual loans greater than 0.5% of capital are selected for review Lending to related parties is specifically reviewed and tracked Management processes for financial risks and positions are assessed Management quality is explicitly assessed (e.g. planning, policing, depth, succession, self-dealing, approach to risk, etc.) Examiners request the passing of accounting tickets for adjustments before concluding examination Examiners review quality of audit programs (external and internal) Examiners rate financial institutions' condition and propose preventivelremedial actions and time bound program of correction Examination Reports (See working paper 340) Examination report is well designed and comprehensively focuses on core weaknesses that need to be corrected Examination report is transmittedldiscussed with financial institutions' board, and encourageslelicits corrective action 7. Enforcement Power & Imdementation Supervisory Powers (See FSD Suggested Guideline 13) Ability to imposetime bound action plans for improving in systems and risk management programs Ability to suspend and remove incompetentlincapableldishonest directors and managers Ability to limit growth and expansion Ability to order cessation of riskylimprudentlunsafelunsoundpractices 19 Ability to limit lending and investment, and discontinue activities Ability to restrict dividends, and bonuseslfeeslremunerations paid to directorstmanagers Ability to order recapitalization plans within a time framework Ability to change auditor, and request clarificationslfurther investigation by them Implementation (See FSD Suggested Guideline 13) Written procedures for use of enforcement powers Evidence that enforcement powers are routinely utilized 8. Resolution of Problems in Troubled Intermediaries General (See working paper WPS 340) Problem and insolvent financial institution characteristics are clearly defined in law orland regulation Supervisor can order restructuring, closure, liquidation or merger Restructuringlclosing functions are separated from the supervisory function Rehabilitation only possible when it represents the least costly solution, and only where the ownership proportion of existing shareholders is eliminated and culpable management is replaced Well developed policies, techniques, and capacity to determine viability and liquidation valueslcosts Supervisor agency has funding for actions which ensure that resolution leads to stable solution Transparent process for re-privatization focuses on obtaining stableldurable resolution ANNEX 2 INTERMEDIARY INSTITUTIONAL INDICATORS 1. Independence and Capacity of the Board Portion of board comprised of non-executive directors Existence of audit committee comprised solely of non-executive directors Experience of board members Extent of indebtedness to institution of board members and their related interests 2. Risk Manapement Procedures Existence and quality of policies and procedures addressing credit risk, interest rate risk, foreign exchange rate risk, position or market risk, and liquidity Formally established risk limits Routine and comprehensive measurement of risks and liquidity Reporting and approval processes regarding exceptions to risk limits 3. Budgeting and Capital Planning Formal budget process Periodic reporting of actual results versus plan Formalcapital planning process incorporating growth projections, and current and future risk profile 4. Management Information Systems Addresses all major lines of business Aggregates risk exposures across business units and legal entities Independent testing of accuracy Process of periodic evaluation and enhancement 5. Use of Internationallv Accepted Accounting Princi~les Internal accounting principles that comply with GAAP and RAP 21 Establishment of provisions for losses in a timely manner 6. Annual Audits by Qualified Firm Audit performed by recognized firm Engagement letter agreed by board committee Covers all branches and subsidiaries of the subject institution Results reported directly to board committee 7. Proeram for Human Resource Development Formal policies regarding recruitment, compensation, training, evaluation and counselling Job descriptions Sound evaluation and promotion procedures Management succession program