g s-£V&t - C/v tK U" Document of The World Bank Report No. 12987-RU STAFF APPRAISAL REPORT THE RUSSIAN FEDERATION NANAGEMENT AND FINANCIAL TRAINING PROJECT NOVEMBER 23, 1994 HUMAN RESOURCES DIVISION COUTRY DEPARTMENT III EUROPE AND CENTRUL ASIA REGION CURRENCY EQUIVALENTS Unit of Currency = Ruble December 1991 = 170 December 1992 = 420 December 1993 = 1229 March 1994 1700 November 1994 = 3085 TERMS & ACRONYMS CF - Central Foundation DC - Direct Contracting EBRD - European Bank for Reconstruction and Development EDI - Economic Development Institute EU TA-CIS - European Union Technical Assistance for the Commonwealth of Independent States Project GAAP - Generally Accepted Accounting Principles GAAS - Generally Accepted Accounting Standards GKI - State Committee on the Management of State Property IAB - International Accounting Board IBRD - International Bank for Reconstruction and Development ICB - International Competitive Bidding IEC - Internal Evaluation Committee ILO - Internat;onal Labor Organization IMF - International Monetary Fund IPK - Upgrading Institute IS - International Shopping LCB - Local Competitive Bidding LIB - Limited International Bidding LS - Local Shopping MOF - Ministry of Finance MUV - Index of Unit Value of Manufactured Exports Oblast - Region OECD - Organization for Economic Cooperation and Development PPF - Project Preparation Facility Raion - Sub-division of a region RF - Regional Foundation NTF - National Training Foundation SOE - Statement of Expenditure STF - Systemic Transformation Facility STS - State Tax Service TA - Technical Assistance TOR - Terms of Reference USAID - United States Agency for International Development FISCAL YEAR January 1 - December 31 THE RUSSIAN FEDERATION MANAGEMT AND FINANCIAL TRANNG PRomm' CONTENTh Page No. LOAN AND PROJECT SUMMARY ...UMMARY..... ., i 1. INTODUCTrON 1. H. THE SETRAL SE7tNG . 3 A The Demand for Market-Oriented Training ........................... 3 B. Training Supply Issues ........................... 7 C. Plans and Programs ........................... 13 D. A Strategy for Market-Oriented Trang ........................... 15 m. LEsOm OF ExPERC E AND BANK SRATEGY .......................... 19 A. Lessons of Experience ........................... 19 B. Bank Lending Strategy .................. 21 TV. TrE PtoJEr ................................................. 23 A. Project Objectives .......................................... 23 B. Program Descriptions ........................................ 23 V. INTERMEDARY: THE NATiONAL TRAINNG FOUNDATION ....... .. ............ 27 A. Approach and Justification ................................. ... 27 B. Clients and Customers ..... 28 C. Organization and Staffing of the NTF 28 D. Procedures .............................................. 32 E. Criteria .35 F. Financing and Cost-Recovery Policies .35 G. Legal Framework .37 VI. PRojcr COm, FINANCIG, MAGDmNAGEMENT AD .IMPLEMENTAIG N. .39 A. Project Costs .39 B. Project Financing .42 C. Managemea and Implementation .43 D. Procurement .45 1. Whs report is based on the findings of Task Manger Richard Johanson (1LO training specia) as well as Carolyn Ban ublic admno specialist), Robert Brinkcshof (staff development s hpecialst), Andret D asev (dis e speciast), Ivan Habanec (implementbon specilist), George Knawaty (small business develpnmet pecia), Frank Kenefick (procurement specialist), Irma Kichigia (legd specias), Peter Knight (mass media peci), Poetr Kowar (mangement development speciali), Tatana Kylova (finance and accounting spociaist), Miln Kubr (managemnt development specialist), Michael Maher (managemen deve speiist), Marsha McGmaw Oive (Rusda ad public finae specialist), Olivier de Messiers (roject costing and imlemenaion speciaist), Mohamed Momsa (finane end accoung specias), Gathoni Mungai (i specialist), Barbma Nunberg (public se 4 it), Joao B. Olveia (orgzaonal den and distance education peci), Dmitry Pelrin (Russia spew-A', Robert Polderm (oaizational design pecialist), Nalni Srinivas (EC3HR), and Mary Elizabeth Ward (bankini specislb-. Rlph Habiso (EC2HR), Adriaan Verspoor (MSP) and Harold Wackma (EMTBG) were per reewers for d.iE project The supervisg manger is Robert Liebendhal (BC3HR); the Director is Russell Cheethm EC3). E. Disbursements ........................................... 49 F. Sustainability ........................................... 52 VII. PROjEcT BENmE MR N) SS ..................................... 53 A. Benefits ........................................... 53 B. Risks . ............................................. 55 VII. AGRMENTS REAC iED AN RECOMmENDATiNS ........................ 57 Annexes Annex 1. Action Program For Market-Oriented Traning ......................... 59 Annex 2. Management Development: Action Plan and Programs ................... 63 Annex 3. Accountancy, Audit and Financial Management: Action Plan and Prograns .... ... 73 Annex 4. Banking Training: Action Plan and Programs ......................... 87 Annex 5. Public Finance: Action Plan and Programs ........................... 93 Annex 6. Organzation, Stffing And Procedures of the National Training Foundation .... .. 97 Annex 7. National Training Foundation Evaluation Plan ......................... 113 Annex 8. Monitoring and Impact Indicators ................................. 119 Annex 9. Subproject Appraisal Criteria .................................... 121 Amex 10. Projected Levels of Cost Recovery ............................... 129 Annex I1. NTP Cash Flow Projections .................................... 133 Annex 12. Detailed Baseline Project Costing ................................ 137 Annex 13. Summary of Technical Assistance Inputs ............................. 139 Annex 14. National Training Foundation Staff Development Plans ................... 143 Annex 15. Summary of Programs, Subprograms, and Subprojects ................... 149 Annex 16. Implementation Plan ........................................ 153 Annex 17. Detailed Procurement Plan and Schedule ............................ 161 Annex 18. Supervision Plan ........................................... 165 Annex 19. Documents in Project Files .................................... 167 aim & Diagms Diagram 5.1: The National Training Foundation (NTF) Network .................... 29 Diagram 5.2: Structure of the Central Foundation of the NTF ..................... 30 Chart 6.1: Disbursements Illustrated Per Semester and Cumulatively ................. 50 Chart A6. 1: Organizadon Chart-Positions of the Central Foundation ................. 104 Chart A6.2: Subproject Evaluation Process ................................. 106 Tabkes Table 5.1: Initial NTF Staff Estimates .................................... 31 Table 6.1: Project Costs Summary by Project Component ........................ 39 Table 6.2: Project Costs Summary by Category of Expenditure ..................... 40 Table 6.3: Proposed Technical Assistance .................................. 42 Table 6.4: Financing Plan ............................................ 42 Table 6.5: Types of NTF In-House Training ............. 43 Table 6.6: Summary of Proposed Procurement Arrangements. ............. 45 Table 6.7: Estimated Loan Disbursement Schedule ............. 49 Table 7.1: Financial and EconoLuc Benefits ............. 54 Table A6.1: NTF Structure-Key Functions and Responsibilities ...... ....... 99 Table A6.2: Relationships Between the Central and Regional Foundations of the NTF .. 102 Map IBRD No. 25948 THE RUSSIAN FEDERATION MANAGEMENT AND FINANCIAL TRAING PROJECT LOAN AND PROJECr SUMMARY BORROWER The Russian Federation BENMCIARY The National Training Foundation AMOUN US$40 million, equivalent TERMs 17 years, including 5 years grace, at the Bank's standard variable interest rate ROJECr ORnC'v The proposed project supports the first phase of a long term program for market- oriented training. Its purpose is to increase the quality and supply of skills needed to support the transition to a ma&ket economy in such areas as enterprise restructuring, the financial sector re'orms and privatization. The project has three principal objectives, namely: (i) to train practitioners in three core fields of management, financial sector, and public finance; Qii) to develop an intermediary institution, the National Training Foundation, to mobilize and channel resources for high priority training investments; and (iii) to implement pilot projects in other key market areas and establish the basis for a broader second-phase investment. ftoiwr DESCRIION The project would finance programs within the following eligible fields: (J) Managem Developmet. Trainig managers in privazed eterpriss; strengthening management training capacity in terns of policy, standards, resources (earning materials and trainers); and development of professional supporting infrastructure (management consultants, associations and networks). Qi) 77we 15w5i Secor. Accounting, auditing and financial management: Training of accountants, auditors and financial managers; strengthening training capacities in curricula, standards, and training materials; developing existing faculty; and upgrading training technologies. Banking: Training of bankers (chief executives in sound bank management and mid-level managers and officers in credit risk investment banking management, and international banking); training in bank accounting; training of trainers; and development of teaching programs and material. (iii) PbAc Pace. Training of senior public finance officials in revenue collection and budget processes. (Iv) Phase 2 Pt Pmjecs. Developing and implementing subprojects of limited scale in other important areas, such as small business, public administration and market economics for the general public. Page U Loan and Project Summarv The project features the development of the National Traing Foundation (NTFI, an intermediary, for selecting and funding subprojects in the above fields. The creation of a new training foundation was justified by the massive scale of training requirements, which exceed the scope of any existing agency; the present lack of coordinating mechanisms; the need to involve non-government institutions and employers in the process; the need to address regional requirements flexibly; and the need for objective project selection and strict financial controls. The NTF is an autonomous institution with half government and half private ownership. Its structure includes a central institution with a network initially of three Regional Foundations. The NTF would be the vehicle for defining strategies, identifying priorities and allocating funds on a competitive basis to various locally-generated training projects of high priority. RATIONALE FOR BANK INOLVEMNET The proposed project would make a direct contribution to the Bank's objectives of promoting marke. based institutions and the private sector in Russia. First, it would help develop the human capital necessary for the successful functioning of a market economy and the implementation of economic reforms, particularly financial and enterprise reforms. Second, it would assist in labor market restructuring and employment creation by providing entrepreneurial and professional skills. Third, it would help restructure key areas of the enormous training infrastructure and place it on a more competitive footing. Finally, it would develop an institution, the National Training Foundation, that would help rationalize training investments and mobilize additional official and private assistance for training. BENEFm The project's immediate outputs would be (i) the creation of new and better training resources in management, accounting and finance, banking as well as public finance, including definition of teaching standards, curricula, learning materials, trainers and networks; (ii) the creation of new capacities to deliver training, including the development of a private training infrastructure, local training capacities, distance learning and in-company training; and (iii) the development of an umbrella organization able to define training strategies and to evaluate locally-generated subprojects according to technical standards. The ultimate output would be substantial numbers of people trained to international standards in the core skills required fbr a market economy. The outcome would be increased productivity and more rapid labor redeployment which would contribute to an accelerated transition to a market economy and greater individual and capital earnings. RISKS The project entails the creation of a new institutional structure in a complex and changing environment. The principal potential risks include: (i) the quality and incentives for NTF staff being sufficient to make the project work; (ii) a lack of quality control when selecting subprojects; (iii) an inadequate application of financial controls; and (iv) a possible dispersion of funds over too many programs with a consequent loss of impact. The project design has incorporated structures, procedures and criteria specifically intended to minimize the above risks. However, given the country context, borrower and donor staff must keep such factors under close and constant review. In addition, the project entails basic development work in terms of teaching programs, development of materials and training of trainers. Full realization of the benefits of these investments depends on additional financing for their dissemination and application. Loan and Projea Swnmary Page iii ESTIMATED PROJECr Cos0st (Rubles billions) (USS millions) % % Total Foreign Base Local Foreign ? Local Foreign Exchange Costs Managem.ent Development 20.5 25.1 6.6 8.2 : 55 30 The Financial Sector 35.7 32.2 6 11.6 10.4 47 45 Public Finance 6.9 7.3 4 2.2 2.4 52 9 Phase 2 Pilot Projects 4.2 2.9 1.4 0.9 41 5 Progwa,w Areas SS loJ 63.1 64.6 ' 20.4 21.0 A 51 84 Nalo Tnd niaFowukion 11.7 4.7 . 3.8 1.5 % 28 11 Total Basdle Costs 79.0 72.2 t 25.6 23.4' 4 48 100 ndoeConoing es 3.8 3.5 1.2 1.1 48 5 TOTAL PROJECT COSTS 82.8 75.7 26.9 24.5 ' l 48 t Figures may not total due to rounding. PRoJECr FINANCING PLANt Total Foreign Exchange Local Costs Project Costs US$M % US$M % Poject Gnvenrment (Federal/Regional) 0 0 3.0 11 6 National Training Foundation 0 0 1.0 4 &A 2 Subprqect Beneficiaries 0 0 6.3 23 4 12 Govemment of Switzerland 0.6 2 0.5 2 44 2 World Bank 23.9 98 16.1 60 " 78 m02., = 2 = TOTAL 24.5 100 26.9 100 100 tFigures may not total due to ounding. Page lv Loan and Project S&unay EhTED LOAN DisBuSRsNr SCHEDuu D_bunement Over 48 Months Bank Fuia Year 95 96 97 98 99 Semester (6 mondhs) 1 2 3 4 5 6 7 8 Per Semester (%) 1.0 2.5 5.0 8.0 16.0 24.5 28.8 14.2 Per Semster (USS millions) 0.6 0.8 2.0 3.2 6.4 9.8 11.5 5.7 Cumulative (%) 1.0 3.5 8.5 16.5 32.5 57.0 85.8 . Cumulative (USS millions) 0.6 1.4 3.4 6.6 13.3 22.8 34.3 Figum may not total due to roundg. TIRE RUSSIAN FEDERATION MANAGEMENT AND FINANCIAL TRAINLNG PROJECT I. INTRODUCrION 1.1 The Russian government is in the process of making a massive, unprecedented shift from a command to a market economy. The requirements for such a change include restructuring most productive sectors completely, creating more service-oriented functions, establishing a labor market, and wholesale restructuring of the government's role. Experience elsewhere, however, has shown that economic reforms cannot be implemented adequately without trained human resources. 1.2 In recognition of the importance ox trained human capital for the transition, the Bank started analyzing Russia's training requirements and supply ,onstraints in mid-1992. These analyses led to the definition of a possible market-oriented2 training project in which the government showed interest. With the assistance of a grant from Japan's Policy and Human Resources Development Fund, Bank staff undertook comprehensive studies and prepared a project proposal in close collaboration with Russian specialists. A notable aspect of this collaboration was a World Bank-Russian government workshop held near Moscow in May 1993 during the preparation process, in which seventy-five Russian experts participated in working out specific program proposals. Further collaboration took place in connection with the institutional underpinnings for the project, including a Founders' Meeting on the proposed instiutional intermediary. These preparatory studies and actions formed the basis for pre-appraisal in November 1993 and appraisal in March 1994. i.3 The proposed project recommends IBRD financing of US$40 million to establish the National Training Foundation, an intermediary through which to finance high priority subprograms in market-oriented fields. The proposed loan is recommended for Bank financing for the following reasons: * it would address critical issues in the demand for and supply of market skills ftrough a detailed strategy and action plan (chapter 2); * it incorporates lessons learned from experience and would support the Bank's sectoral and country strategy (chapter 3); * it would finance high priority training programs needed for the transition to a market economy (chapter 4); * it would create an innovative Russian intermediary to select, finance, and supervise high priority investments in training (chapter 5); * its costs are reasonable and implementation has been planned in detail (chapter 6); * it would confer important financial and economic benefits, and steps have been taken to minimize the risks involved (chapter 7). The following chapters of the report address each of these topics in turn. 2. "Markt*oriented as used here refers to fields and occupadons critial to a market economy; specificaly: enteprise management; acounacy, finance, and audiding; bankng; small and medium-ie business; and new civil service and public adminisaton concepts. wTraining" is used in this report to mean preparation for a specific occupation, as differentiated from education, which refers to acquiring knowledge wihout reference to a specific job or occ ation. ^ReWaining" means preparing someone who alreay has an occupation for a new one. "Upgrading means improving one's knowledge and sldlls widhin an existng occupation. In-servicc training" means upgrding during employment, either on or off die job. The levels of trainig covered in the report are for professionals nd technicians. This firmework explicitly excludes from consideration technical subjects, such as engineering, in which Russian standards are reasonably high, and also manual or vocational skills and occpatons. TIE RUSSIAN FEDERATION MANAGEMeNT AND 1ANANCIAL TRA1ING PROJECT R. THE SECrORAL SETnNG A. THE DEMAND FOR MARI-ORIETE TRAING 2.1 Russia has begun major market-based reforms and has made an impressive, but uneven, start in its transition to a market economy. Liberalizing the economy further will require deeper structura reforms and a firm legal basis for private sector activity, which the government is beginning to undertake. However, the reforms cannot be fully realized without a parallel effort to equip the county with market- oriented skills. 2.2 Russia has had little exposure to market activity, even compared to other Central and atemrn European countries. Foreign investmeent and private domestic activity have been lower in Russia, the concentration of employment in the state system higher, and the period of command economy longer. Consequently, the basic market skills that are taken for graned in industrial countries and in many developing countries have been virtually nonexistent in Russia. 2.3 A successful transition to a market-based economy requires large numbers of people to be trained in completely new commercial, financial, and management ski1ls. Millions of managers, entrepeneurs, and administrators are restructuring enterprises, creating small businesses, opening new bank, and transforming public finances and administration. Ihis is creating an enormous and growing demand for market skills in many critical areas: particularly enterprise management, accounting and finance, baoking, and public finance. eWplIw Mwagemat 2.4 Economic reforms, restructurmg, and privatization are desdned to alter radically the environment in which Russia's industrial and other enterprises operate. These changes will inevitably establish new criteria for distinguishing achievement from failure, provide new opportunities and incentives, and expose enterprises' decision makers to the discipline of the market. As tbs economy opens up so as to increase exports and attract foreign investment, enterprise management will increasingly be exposed to foreign competition, and its performance standards will have to be comparable to those in industrW market economies. However, ineither privatization nor economic reforms by themselves will guarantee the conversion of inefficient enterprises into excellent performers. Liberalizing trade and ;rices, ntoducing competition, phasing out government subsidies, and rigorously applying bankruptcy legislation can even undermine the pursuit of the reforms if managers are unabie to keep pace with the changes, take advantage of new business incentives, and comply with the new regulations. 2.5 For seven decades the managers of Russian enterprises were sheltered from the direct impact of market forces and were deprived of the possibility of learning from iternational management experience and thinking. Ideology, distance, and language formed major barriers. This has resulted in a unique context fnr economic reform, different in many ways from that in other reforming economies. Virtually every report on Russian management points to the need for an array of new knowledge and skills in areas that were nonexistent, underdeveloped, or distorted under the command economy. Such Page 4 Cahptr 11. The Sectoral Seating areas include business strategy, marketing and selling, costing and pricing, financial management and control, export promotion, and corporate governan".e. Equally important needs exist for skills in areas that-on the surface-existed in command economy management, but in reality were driven by different forces, applied different criteria, used different methods, and needed different cormpetencies from those in a market economy. This explains the demand for training and retraining in production organization, productivity and quality management, and human resource management. In all sectors, managers will ha-ve to cope with radical enterprise restructuring, to be implemented at an unusually fast pace.3 Most of the immediate, short-term skill requirements concern business and organizational diagnosis, business planning, enterprise organization and restructuring, changing management and implementation, business partner identification, forming strategic alliances, market development, changing employment and motivation patterns, corporate finance, working with banks and financial markets. Profound sectoral imbalances also exist that will have to be corrected by the process of restructuring, thereby creating unprecedented management skill requirements in service sectors, including distribution, logistics, and the financial and professional infrastructure of the market economy. The demand for new knowledge will be high in all these areas. The need to change values and attitudes will be even more pressing. However, in the short term few managers will be in a position to engage in educational and training programs of general nature and longer duration. They will be looking for training and advice that is problem- and action-oriented and has a direct and measurable impact on the improvement of their enterprises' capabilities, competitiveness and business performance. 2.6 Russia's managers at all levels and in all sectors will be learning mainly on the job, by seizing new opportunities and solving their firns' most pressing problems. However, learning on the job alone will not suffice. Such methods of acquiring market skills could prove too difficult, slow, and costly if not facilitated and supported by well-designed and effectively delivered training programs based on the best international and Russian practice and experience. Virtually all Russian managers need at least a short period of formal retraining. The target population for management training in industry and other sectors is between one and two million. In addition, the public sector must not be overlooked as the role of government changes. Enterprises and agencies that will stay under public ownership and control must also undergo major changes regarding their objectives, performance criteria, and management patterns. Managers of public enterprises will need training as urgently as their peels in the new private sector. 2.7 At the present time there is a major gap between already existing or emerging training needs of Russian managers, and their expressed demand for training services. This is due to factors such as poor experience with training to which managers were used to from the command system, a lack of understanding of what can be achieved if training is modernized and becomes highly practical, shortage of finance, and the managers' virtually exclusive concern with short-term survival, trouble shooting and crisis avoidance. These attitudes are likely to change with progressing privatization and increasing competition. It will be important, as well, to demonstrate in practice the contribution that effective management training amd development can make to enterprise restructuring and performance improvement. 3. Recent studies indicate Ihat lack of "managerial u" and the virtual noexistence of finncial and technic experdse in the areas of commercial practice and market transactions," consXtt importn obstacles to the pace of pnvatzaton. Privatzation in the Republcs ofthe Former Soviet Union: Framerk and til ResWu, World Bank, June 1993. Chpter 11. The Secorul S&big Page 5 he FlWi SecFor 2.8 Acwwuntng, AudWng, and Fnance. The embryonic marketbased economy has placed massive new demands on Russia's accounting and finance sub-sectors. The accounting field and related disciplines (auditing, financial munagement, budgeting. management accounting) are undergoing radical change in Russia. Under the central command system accounting was used as a tool of central planning and control at the national and regional levels. Under the market-based system, accounting identifies, measures, and communicates fims' economic activities so that economic resources can be efficiently allocated at the enterprise level. 2.9 This new role of accounting is unfamiliar to most practicing accountants in Russia, who were trained in accordance with the techniques and procedures of the uniform Soviet accounting system, as well as to top enterprise executives, government officials, ane other potential users of financial information. Current accounting practices are still rooted in Soviet-era requirements for detailed control and statistical book-keeping needs. Similarly, the focus of current audit practices are to establish compliance with Russian statutory accounting and tax rules. According to studies in 1993, none of the Russian audit firms has prepared financial statements in accordance with Generally Accepted Accounting rinciples (GAAP), and none has audited such statements in accordance with Generally Accepted Auditing Standards (GAAS). The development and wide acceptance of Russian accounting principles and auditing standards appropriate for a market-based economy are still some distance away. In addition, the Russian audit firms do not have the resources to train large numbers of accounting graduates under Russian stutory rules let alone international standards. In such an environment, there is considerable need for establishing educational and training mechanisms ained at teaching market-based accounting and auditing practices from the ground level to meet the new and emerging needs of business and management in a market-based economy. 2.10 Tle scale of the demand for new financial skills can only be gauged in approximations, but it will be huge. If chronically inefficient financial services are not to hobble the development of an open eoonomy, the r -. iber of employees in the financial sector will have to double from 500,000 to I million in the near future. These new staff will need training and most of the existing personnel will need retraining. The demand for accouants and auditors is also massive and will grow progressively as international standards are adopted. Estimates suggest that 300,000 new accountants and bookkeepers will be needed annually for the next five years to serve newly created private enterprises, and that 2 million bookkeepers will require retraining so that they can play a useful role in existing enterprises. More than 20,000 auditors will be needed annually. The training is complicated by the need for practitioners and trainers to learn both a new accounting system and how to apply accounting to financial management. 2.11 B dang. The Russian commercial banking system has existed for at most eight years, with both its structure and purpose changing at a fast pace. An in-depth transformation of the financial system in Russia began in 1987 with the creation of a two-tier banking system with central banking functions vested with the Gosbank and commercial banking functions separated and delegated to a number of specialized institutions. Further liberalization of the system has proceeded at a rapid, albeit disorderly, pace. New banks have proliferated, financial policies are precarious, the regulatory and supervisory framework is still embryonic, accounting and auditing refo.ns are not yet complete, and the payments system remains largely undeveloped and inefficient. Improved efficiency in the sector is essential for banks to successfully execute their key roles in a market economy, namely, mobilizing financial savings, allocating resources, and processing payments. Preparing Russian banks to operate in a market cnvironment requires new methods for managing banks, marketing and delivering banking services, Page 6 Chapter M Te Seaoral Setting controlling risk, ensuring banks' financial viability and performance, and carrying out basic banking operations. 2.12 Estimates suggest that the financial services sector employs some 500,000 individuals, which given a population of 160 million, indicates the sector's current state of underdevelopment. Emerging banks which currently number about 2,000 with branches approximating 4,000 are finding it extremely difficult to recruit staff with suitable qualifications and experience. They are resolving this problem in part by hiring away some employees from older, established banks. There exists a high demand, therefore, for the development of a wide range of banking skills, ranging from managerial capacity to operational skills. Of particular need are skills in the area of overall risk management, credit and financial analysis, and banking operations. Pablic Financ 2.13 Public finance decisions critically affect the business environment by influencing levels of taxation, savings and investment. The transition from a command to a market economy requires a fundamental change in the management of public finances at all levels of government. Instead of assuming secondary importance, the federal budget should now become the single most important tool for the achievement of national priorities and goals involving the allocation of resources, their distribution, and the stability of the macroeconomy. As it is, the Russian budget is inadequate as a tool of fiscal management. The adaptation of the budget to the needs of a market economy has been handicapped by the absence of trained skilled personnel in most areas of budgeting (including macro quantitative analysis, expenditure control and reduction. evaluation and audit). In addition, regional governments recently won budgetary autonomy and are now responsible for funding and implementing a large share of investment and social programs. Public finance officials at the oblast and municipal level are ill-prepared for their new functions. Likewise, important reforms in tax policy and administration have been introduced without preparation of human resources to implement the proposed changes. New revenue-raising measures will be critical to reduce the budget deficit and thus help stabilize the economy. If public finances are not managed carefully during the transition to a market economy, there is a risk of continued macroeconomic instability, resource misallocation and underfunding of critical social and investment programs. 2.14 The demand for new skills by public finance officials is expected to rise significantly over the next several years with the introduction of important reforms in budgetary and tax systems. Most of these measures have been included in the Government's April 8, 1994 Statement on Economic Policies to the IMF in connection with the second Systemic Transformation Facility (STF) purchase. For example, the gradual adoption in 1994 of a new budget classification system conforming to IMF standards, for full implementation in 1995, will require new expertise in budget preparation in the MOF, line ministries and parliament. In addition, there is immediate need for training 1,500 managers and 20,000 specialists in the new computerized Treasury system. The transition in the execution of the federal budget from the banking system to the Treasury is expected to be largely completed by the end of 1994. The Government also plans additional improvements in tax policy and legislation, as well as a special effort to increase significantly collections from the VAT tax. Both computerization of tax administration and the training of up to 200,000 officials from the State Tax Service will be vital to this effort. Finally, the financial position of local governments came under strain with the introduction on April 1, 1994 of a new system of federal-regional budgetary relations. Several hundred senior local govermment officials will require training in tax and budget policv in order to implement the new fiscal federalist system effectively. Chapter MLe Secral Seung Page 7 Other PloMiy SkAl Requments 2.15 The development of human capital and market-oriented skills is also important in an array of other areas, including and in particular small business development, public administration and market economics for the public at large. * Small enterprises mushroomed from a few hundred "cooperatives" when they were first permitted in 1987 to nearly 700,000 at present employing nearly 20 million persons. These are important not only for the range of goods and services needed in a market economy, but also for the generation of new employment. In addition to policy, banking and financial support, there is also an urgent need for appropriate institutional infrastructure to support the development of small enterprises, in particular training and advisory centers to train practicing and would-be entrepreneurs in business planning. * The transition from a command to a market economy entails a fundamental change in the structure and functions of government at all levels with concomitant changes required in the knowledge, abilities and skills of civil servants. Top government officials at all levels need training in such areas as the changing role of government in a market economy, including policymaking, economic management and regulation. * For 75 years-more than three generations-the population of Russia was largely cut off from contact with market economies. Reforms to create a market- oriented economy are underway, but the population at large lacks an understanding of how to live and work successfully, and of the proper role of government and business, in such an economy. Full implementation of economic reforms should be supported by massive education of the general public in the basics of market and business economics. B. TRAING SUPPLY ISSUES 2.16 Despite the presence of an extensive institutional network, the training system as a whole requires thorough restructuring and reorientation to serve a market economy. It is not preparing enough people with the necessary skills. Not only does the current training system produce too few graduates in market-oriented fields, those it does produce are ill-equipped to meet the requirements of open market practice. The present system of management and financial training is inadequate in four main respects: * It lacks a strategy for system-wide coordination and development; * Its content is not relevant to current Russian requirements for a market economy; * Its quality of instruction is low; and * Its throughput of students and practitioners is insufficient. System dcture and Management 2.17 Russia inherited an impressive institutional network from the command economy for the training and development of management and financial "cadres". This training system features thousands Page 8 Chapter .t. The Sectoral Seing of technical colleges (technikuns), about 500 degree-level institutions (universities, academies and nsrtitutes) including 220 management faculties and departments, plus an elaborate network of 100 sectoral upgrading institutes (IPKs) and some 540 faculties of upgrading at universities. The system was geared fully to the needs of a politicized and centrally-controlled economy. Senior and middle-level managers had an obligation to attend periodically in-service courses at these institutions. Training establishments had no problem in finding participants-government planning and financing guaranteed a steady clientele. In addition, larger enterprises had their own training departments and centers to provide training for lower and middle-level management and financial personnel. 2.18 The state training system was highly centralized, specialized and compartmentalized. Over twenty ministries controlled parts of the system of higher education and post-degree training. Each sectoral ministry established a large personnel departmnent one of whose functions was to operate its training infrastructure of the sector-specific upgrading institutes. The main lines of communication were vertical within each ministry. Decisions typically were made in Moscow. Policy-making and surveillance were provided by the Communist Party, both centrally and in the regions. Management from the center also included financing. The Government provided allocations through the national budget for both the development and financing of the training system. 2.19 The collapse of the command economy has caused major changes in the state system of training. Sectoral training institutes now get neither the customary financial support nor clear guidelines on how to adjust to new conditions. Most enterprises have been unwilling to finance the training of their managers under the traditional programs. As a result, some upgrading institutes have closed, some have sought alternative activities and many have attempted to reform and modernize their service offerings. University-based education for management and finance continues to be government financed' at the degree level, but outside funding has to be sought for the training and retraining of practitioners. Despite competing claims, no government organization or agency has the effective authority to coordinate traning activities. 2.20 The most significant and visible reform efforts have taken place outside the traditional institutional network through the establishment of new management and business schools, private or semi- private, some loosely attached to existing public academic establishments. Some of these business schools have become leaders and have been able to attract the best teachers and trainers available.5 Further development of private training, however, is constrained by (i) the lack of a legislative framework covering things as tax exemptions for not-for-profit training institutions; (ii) lack of capital to finance 4. Figures are not available on spending for mnagement and financial traimng wiin umversities. However, using aggregat expenditur dat for higher educaton over the last few yeass as a proxy, spending dropped sharply between 1991 and 1992, but recovered in 1993 so that overall expendiures only declined 5% over the period. Expenditures were Rubles 109.4 billion in 1991, Rbl 94.7 billion in 1992 and Rbl 104.2 billion in 1993 (in constant 1992 Rubles). Since ovenll enrollment also declined, spending per university student decreased only slightly over the same period, from Rbl 2763 in 1991 to Rb! 2624 in 1993. 5. In contas, hundreds of smaller private training centers and companies have been established in the last four years, in the vast majority of cases, without compdent staff or adequate fities. 'Me main aim of these oppotic institutions, in the absence of quality standards, has been to profit quicldy on the strong, but inexperienced demand for market-economy skills. ape IL The SetOr7 Setng Page 9 research and development work; (iii) lack of physical facilities in which to deliver training;" and (iv) the lack of most people's ability to pay tuition for private training. Collective initiatives include the creation of voltmary associations of business schools and management development establishments. They lack power to coordinate and make decisions as regards the management of the whole system, but have started a professional dialogue on common issues. 2.21 On the whole, the Russian system of management and financial training is far from stagnant; efforts are underway to bring It in line with the requirements of the emerging market economy. However, these efforts are sporadic and insufficient in the light of progressing privatization. They suffer from inadequate resources and lack of policies on how to reform and improve the system as a whole. There has been considerable fragmentation of efforts and little exchange of infonnation and collaboration among institutions and individuals. 2.22 A new division of tasks and responsibilities has yet to be defined. In particular, government must redefine its role in training and sector-specific training must give way to more generic and flexible forms of training. Government and private sector must work together to organize supplies of well trained professionals. A stra for development of management and financial skills is needed oulining the respective roles and approaches of government, academia, voluntary groupings and privadzed enterprises. 2.23 In the past, the training infrastructure was geared toward technical fields and ideological requirements. Engineering consfituted almost two-thirds of all enrollments in higher education. Market- oriented skills, by definition, were not taught in a command economy, making the traditional training infiastructure largely irrelevant to the economic reform. 2.24 The Knowle Gap. A whole new set of subjects that have been mostly or completely missing needs to be introduced. Examples are given below for key sectors: * Industry: genera management, including policymaking, decisionmaking, and problem solving; privatization and business and taxation law; production management, including product development, quality control, production methods, cost reduction, and the use of information systems; marketing, including market research, distribution, pricing, and export marketing; finance and accounting, including financial management and cost accounting; and human resource management, leadership, communications, and incentives. * Banking, financial services, accounting and auditing: accrual basis of accounting, management accounting, bank accounting, market-based bank management with emphasis on relevant risk management techniques, investment banking, corporate finance, money and capital markets, stock markets, insurance, investment 6. Ample tig ifiltWs exist, but these are all virtufy owned and controlled by the govermen. Some directors of private tainig Uons colain of the large costs and of the control exerted by the government over the rent of these fciwli"S. Pa8e 10 Chapter 11. The Sectoral Setfing management, and advanced financial theories, for example, portfolio theory, capital asset pricing, options pricing, and risk-hedging strategies. * Public finance: macro quantitative analysis, accounting financial management; and budget and tax policy and administration. The lack of foreign lanpgage skills is a problem that cuts across all sectors, but is particularly acute in international business and finance. The list is by no means exhaustive, but illustrates the type of knowledge currently lacking. An interesting feature of the list is the overlap.7 Almost every sector needs skills in management, finance, accounting, and personnel management. These subjects need not be developed separately for every sector. Substantial economies could be realized by developing standardized training across sectors. 2.25 Not only must new teaching programs be introduced, they must pursue relevant objectives. This seems to be missing in much of training in accountancy. Accountancy training is not focused on developing skills that enable the practitioner to identify and generate the information needed by the user for problem-solving purposes in a market-based economy. Instead, the training tends to be limited either to mechanical application of detailed rules in narrow sectoral accountancy topics or to broad theoretical treatment. Teaching programs also need to be adapted to Russian conditions. Teaching and training materials from market-economy countries started being used recently in many institutions. Their choice has not always teen judicious. Mere translations proved to be insufficient, but adaptation to Russian conditions and production of original local materials have hardly started. It is difficult to obtain taining usefiul for the work of Russian enterprises because conditions are so unpredictable and changing. Standard offerings from the west simply do not reflect current problems of enterprises. Imported blueprints tend not to be relevant. Despite the need, Russian case material has not been developed and used widely for training purposes, either in management training or for training of accounts and finance experts. Training tends not be rooted in diagnosis of real experience or needs analysis. 2.26 Trrab gforrPracUioners. Under the previous system, managers and technical personnel were supposed to attend periodically courses at the upgrading institutes. The courses were mainly technical, theoretical, and ideological. Part of this in-service training in industry related to economics, including management. However, this was mostly devoted to a grounding in Party ideology and the economics of production. The programs did not focus on enterprises' needs and did not provide practical, lob-related skills. In-company training programs were rare except for blue-collar workers. In market economies, managerial and technical professionals require training and development on a permanent basis that cover the practical problems people face on the job. This mode of training is best given within the firm, if it is iWge enough, or in institutions where the enterprise can influence the content of the training. 7. Undor a co_mand economy people could be tained in advance for a tpecifo sector, and even for a specific occupation witin the sector, wih somo assur of acally enting and staying in tIe sector. Under a maiket economy sectoral demund cannot eaily be predicted in advance and changes in one's career are far more frequent. A more general preprton is required dth permits ransfer among sectors. Occupations that are transfeile across sectors include, for example, accouants, budget speciaists, and umangers. ampter M The Sectoral Seting Page 11 2.27 In certain fields, such as theoretical science and mathematics, the quality of Russian education is second to none. However, this is far from true in other fields, particularly market-oriented subjects. The incentive system under the command economy tended to ignore quality in all forms of production and service, including training. The current education and training system inherited from the past is based on an "institution-financing" approach. All education and traiDing institutions supported from the federal or local budgets were-and still are-financed according to certain quantitative criteria related to the number of students and professors, teaching space, number of courses-in short, based on inputs. For example, management training emphasized the number of people trained rather than the quality of what they learned. 2.28 Low quality is, first, a reflection of lack of standards. Accountancy and audit training, for example, are not based on a professional framework that governs admission standards, content and performance targets. In management training and development, Russia suffers from the absence of recognized and widely applicable standards defining the content and length of training, the content and level of learning materials, the training methods used, the competencies of the teaching staff and the level of knowledge and skills that should be demonstrated after program completion. It is difficult to measure program quality, compare various recently introduced programs and set targets for quality improvement. 2.29 Low quality is also a direct reflection of the competency profile of most Russian trainers. Virtually all the teachers and trainers in the present system are products of the command economy and, thus, lack training in market-oriented fields. They are deficient in theory, practical experience and teaching methods. Generally they have been isolated and out of touch with their Western colleagues. Teaching methods under the Soviet regime were formalistic, stressing theory, lecturing, memorizing, and testing rather than understanding and application. Modem teaching methods, such as action-oriented training and group work, are little known and seldom practiced. Without retraining this generation of trainers will quickly become unusable. There is, then, an urgent need for major teacher and trainer retraining efforts, covering both the substantial field taught and the methods whereby market-oriented knowledge and skills are transmitted to adults. In addition, teaching aids, such as computers, video players, and reproduction equipment, are not widely available. Physical facilities are more than adequate in terms of space, but the state owns and controls many of them. A large number of facilities need refiurbishing and renovation. Finally, most training does not provide for periodic evaluation of results in relation to expectations and continuous adaptation of the training to the changing environment. Throughpu 2.30 As already noted, the scale of demand is massive if Russia is to have enough qualified people to function efficiently as a market economy. Formal training programs need to be redesigned to ensure that new entrants to the labor market have the required practical skills. Informal and in-service training needs to be expanded and modernized to keep pace with progressing privatization and economic restructuring. 2.31 Managers. The former system for training and retraining managers for state enterprises (para. 2.17) reportedly had an annual throughput capacity of 1.4 million persons. This capacity has 8. Qual is defined here as learnig achievement and the ability to make practcal applications of such leauning. Page 12 Chapter M The Sectoral Setting diminished significantly owing to several factors. Poorly controlled privatuation of training facilities has often resulted in the conversion or use of facilities for other purposes. Shortage of curricula and learning materials on management under market conditions also constrains output. However, the shortage of competent trainers is the main limiting factor. A majority of the teaching staff cannot provide market- economy related training meeting minimum standards. According to current estimates, the throughput of management programs staffed by relatively competent and already upgraded teachers and trainers is not more than 25,000 to 50,000 managers per year. The quantitative gap is wide by any standard. At this rate it would take several generations to satisfy the requirements for managexrent retraining (para. 2.6). Concerted actions are required to develop relevant teaching materials, disseminate them widely, undertake intensive trainer retraining programs and use distance learning methods to reach wide audiences. 2.32 T7he finanil4 secdor. More than 200 institutes of higher education teach modules on accountancy, banking and finance as part of a broad economics discipline, but-with the exception of several apex institutions-the subject has been dealt with superficially. The throughput is about 10,000 acceptable accountancy and finance graduates, and about 5,000 practitioners, per year9. This approximates only about 5% of annual requirements. The challenge of providing effective training is heightened because most trainer teaching accountancy and finance have had no practical experience. In addition, attrition among trainers is high; many are leaving to work in finance outside academia, where the pay is significantly higher. While several new institutions have been created with donor assistance (i.e. International Finance and Banking School in Moscow and International Banking Institute in St. Petersburg), their throughput and geographic reach are limited in comparison to the need. Other forms of training for Russian bankers and accountants assisted by the West have been through short courses that offer exposure to different banking and accounting cultures rather than intensive training. Russian banks and accounting firms (except the "Big Six") themselves have been unable or unwilling to commit resources to develop an in-house training capacity, and all but the strongest lack the critical mass and know-how to devote to the task. 2.33 PMblc fiwnce specials. Owing to developments in tax and budgetary systems (para. 2.12) the demand for public finance training will begin to accelerate in late 1994 and rise significantly in 1995-96. An accelerated training program is needed throughout this period. The present training capacity and planned assistance is inadequate to meet the projected scale of demand. Public finance is taught primarily in the Ministry of Finance regional system and three specialized academies in Moscow 9. 'he Financial Academy's anmnu outp is 170 grduates majoring in accounting taught by 36 professors with Ph.D. in accounting. Moscow State University, over a tree-years period 1991 to 1994, grduated 2,575 trainees in its Moscow Center. The Center has about 20 fill time professors with Ph.D. in accowuting. The Center has affiliates in Novosibirsk, Pem, Ektnburg, Nizhny Novgorod, Omsk, Vladivostok, Smolensk, where they trained about 2000 tranees in these areas over the same period. In and Academy of National Economy, tanng about 650 pratoners a year is delivered by about 100 ful time professors with Ph.D. in accounting. The Financial Institute for Upgrading Skills tnined about 1000 trainees a year with a permanent staff of 30 Ph.D.s about 400 part ime insnucto. The All-Russia Corresponding Instiue graduates between 600-700 graduates a year and taied between 300-400 graduates a year by 8 fu time accounting prossors. The Open University issued 1884 cerficates and 32 Diplomas since its inception three years ago. The Inratonal University graduates 20 master students a year. Chpter AL The Secoraul Se.ng Page 13 and St. Petersburg.t0 Public finance in a market economy is not integrated in undergraduate economics degree programs at the university level. A strategy is needed on three levels to meet present and future demand for skilled public finance officials: retraining of existing practitioners (including both technical concepts and practical on-the-job skills); training of master trainers; and the development of courses for economics degree programs. With respect to the first, the MOF training system, particularly the IPK, has made an impressive effort to retrain practitioners. However, it will soon face a severe challenge when demand rises steeply for new skills in budget preparation and execution. Likewise, the IPK of the State Tax Service will be overwhelmed by the demand for skilled new tax administrators in audit, taxpayer programs and computerized processing. The training of a core group of budget and tax policy experts in the Ministry of Finance, State Tax Service, Duma and sub-national governments will require the development of new courses adapted specifically for the Russian system as well as more computer- assisted training technologies to reach thousands of practitioners at low unit costs. This is essential because traditional training approaches based on resident courses (requiring payment of travel and subsistence) will simply be unaffordable. External assistance must be effectively paired with Russian institutions to generate both the intellectual inputs needed for course development and the finance required for computer-assisted training technologies. Marshalling appropriate and timely external support is essential to meet the throughput and quality requirements of the accelerated public finance training program. 2.34 Distance education. One important way to meet massive demands is through distance education. Russia has a strong tradition with using distance education. Over one-third of post-secondary students, or about one million people, have been following correspondence courses which have a tradition of over 70 years. Participation is based mainly on sandwich courses, with emphasis on individual study and formal classes which meet from 40 to 60 days per year. The major national correspondence school for business enrolls about 30,000 students. Yet, the quality of the courses suffers from the same problems of conventional education, particularly in relation to the quality of instructional materials, teacher training, support to students and evaluation mechanisms. The existing delivery systems are fairly rigid, inefficient in student throughput (dropout rates are high) and overly dependent on government financing. Moreover, little use is made of modern telecommunication facilities, particularly computer- based training, electronic mail and narrow- and broad-cast television, all of which are available. C. PLANS AND PROGRAMS The Govemment 2.35 The government is generally keen to promote training in support of economic reform, but faces difficulties because of its lack of proper coordination, expertise and resources. Several Government agencies are working on various programs and plans to promote training. Decree 65 of February 1992, gave the Ministry of Labor responsibility for developing and coordinating new government training programs. The government had been formulatir,g an intersectoral training program for the market economy, "National Program on Employee Training for the Market Economy," for the 10. The MOF system includes dhe State Finance Academy, the Insatute of Skill Upgrading (IPK) and assoctd programs at the Academy of National Economy, the Plekhanov Initute and the Institxte of Fnance and Economics in St. Petesburg. 'eMse imiiutions gmduate approximately 500 finane specialist annually and provide skill upgradg drough short corses to about 3000 offcials per year. Page 14 Chapter 11 The Secoral Sening past two years. The objective is to finance training in high priority subjects for people actively involved in the transition to a market economy at the central, regional, and local levels. At least a dozen ministries or agencies were involved. The approval process had been slow, but a government decree was signed on November 1993, approving the program in general (though without financial support) and calling for the Minister of Labor to chair an interministerial coordinating council to adopt high priority training programs and find ways to finance them. 2.36 Meanwhile the State Committee on Higher Education is preparing its own program of reforming the training sector. It recently proposed a plan for creation of a fund for "additional education" (the official name of post-degree training). According to this proposal, the fund would draw resources from a compulsory payroll tax and v' ild be responsible for strategy, standards and financing of state- owned training institutions. The proposal has come under attack from various ministries, but it is still under consideration. These government programs could benefit greatly from external expertise and financial support. Exterai Donor Ass tance 2.37 Most donors involved in Russia have identified training as important, especially training in management, entrepreneurship, privatization, and the emerging financial sector. The European Union's TA-CIS has been the largest contributor, targeting assistance to several key institutions. Among the bilateral donors, the following have been the most prominent: Canada, Germany, Finland, France, the Netherlands, the United Kingdom, and the United States. The Soros Foundation is also providing major assistance, nminly in the fields of science and education. The World Bank's Economic Development Institute (EDD) has been providing assistance for training policymakers and trainers, based on locally- developed case studies, largely in market economics, banking and finance, and privatization. In addition, many small private donors are active in developing programs, and often deal directly with local training centers. The European Bank for Reconstruction and Development (EBRD) is develuping new projects to develop curricula and establish decentralized business training centers, and the Organization for Economic Cooperatiois and Development (OECD) is working on ways to coordinate assistance to retrain discharged military officers. Several training activities have been proposed under the new post- privatization efforts to train managers of newly privatized enterp- ses. This initiative is being coordinated by the Russian Privatization Center of the GKI. 2.38 Regarding public finance, until recently, only EDI, the IMF, and the Joint Vienna Institute offered relevant training for public finance officials. This is now changing. The State Tax Service (STS) with OECD technical support and financing from USAID, TA-CIS, Denmark, and the IRS of the United States has opened the International Institute for Tax Administration in Moscow. It is intended o provide training in tax administration and also some tax policy, but capacity is extremely limited. No donors have identified to finance the delivery of training for the new Treasury system. There is an immediate demand for courses adapted to Russian tax and budget systems supported by strategies and financing to expand delivery capacity. 2.39 Overall donor assistance for training has several general characteristics. Much assistance has proved to be extremely useful. With a few eAceptions, however, the assistance has been directed toward specific institutions and, as a conseq 'ence, has tended to be dispersed. In addition, overall strategies for assistance are often lacking. A comprehensive, global approach is absent. As a result, in many cases assistance is piecemeal and haphazard. Much of the assistance is devoted to financing inernational experts and study tours abroad. Much less is spent on providing training in-country, aapw IJ. 7he &ctoria Setmg Page 15 developing Russian trainers, and adapting training materials to the Russian context. Some of the assistance has been designed without adequate involvement by Russian experts or recognition of Russian strengths, thereby compromising the long-term impact. Lack of provision for local finance limits the number of Russian participants working under existing technical assistance programs. Quite often the program will stop when foreign consultants leave. Nor has sufficient attention been given to dissemination of results obtained. 2.40 The view is widespread, both among donors and in govermment, that training needs to be linked more effectively to economic reforms, in prticular, to support emerging and newly privatized enterprises. Usually the training programs are identified with the help of relevant Government agencies which may not reflect the needs of the private sector. Emerging private enterprises and banks themselves are frequently unable to articulate their own training needs due to the lack of understanding of their requirements and underdeveloped professional infrastructure. Private professional and employers associations could play an important role in needs analyzing and standard-setting, but lack access to external technical assistance to develop their capacities. In the final analysis, the scale of demand for high quality programs far outstrips their supply. Assistance being provided or pledged by the foreign donor community, while impressive, falls short of the level required for high priority training needs. The challenge is to develop a coordinated approach that addresses the needs, fills the financing gap, improves the use of resources, yet avoids duplication. To make the best use of scarce training resources in the near term, particularly for cri2ical skills like accounting, an agreed framework on course content and professional development is needed, adapted to Russian conditions, that can be widely applied. D. A STRATEGY FOR MAK-ORINTED TRAIIG 2.41 Based on a dialogue between various government ministries, employers' associations, professional associations, and training institutions, an overall training strategy has been prepared to overcome the problems inherent in the current system. The objective of the strategy is to organize an efficient supply of the market-based skills needed for economic transformation and installation of the market economy. The core strategy to achieve this objective has four principal elements, which are discussed in the following paragraphs. Resmacfw*fg Sys*= Mwaagemwt ad Taww 2.4_ Having recognized that training cannot be managed by command economy methods, the government will have to redefine its role in training. The central government should not-and cannot-supply the bulk of market-oriented training for reasons of cost and relevance. The central government's role should emphasize coordination and policymaking. The government should also provide central services, such as curriculum and materials devclopment and training of trainers, where such activities could be carried out more efficiently centrally. Training should be decentralized to the regions and municipalities. Local control of training will help stimulate new initiatives that relate the content of training better to local problems. Regions and municipalities should develop the capacity to assess local training needs and existing training capacities, initiate new programs, coordinate training offerings across departments, and adapt and implement national training policies. Publicly-supported training centers and institutions should become more autonomous by using local boards of directors and should be required to raise some of their funds by selling services. Page 16 Chapter 11. The Sectoral Senmg 2.43 A major priority should be to develop a demand-driven training system. This involves encouraging the growth of training markets and generally moving the control of training programs and institutions closer to the customers. 'i'he government should establish incentives to encourage the growth of privately-financed training by firms or proprietary training institutions. It should establish a 'light" regulatory framework designed to ensure minimum quality standards without inhibiting initiatives. Training by the private sector tends to be more responsive to users and is more likely to be paid for by the beneficiaries. Development of a demand-based system can be done in several ways: (i) by ensuring thatjoint ventures with foreign partners give due attention to employee training programs; (ii) by building capacities for in-house training within larger privatized enterprises; and (iii) by encouraging professional associations to take responsibility for setting professional standards, controlling quality, and providing training for their membership. 2.44 Building new types of capacity to manage the training system will also be important. Partnerships should be established between the government, enterprises, banks, professional associations, and training institutions to develop training policies and initiatives. Coordinating mechanisms should be established at the central level. A national training management board, council, fund, or organization should be created to channel development funds into the system. Regional training capacities, lacking in the past, must be developed along with the expertise to manage training institutions and centers effectively. Buildng Rekevant Training Programs 2.45 A key feature of developing a market-responsive training system will be to improve the relevance of what is taught in the training system. The content of training must become less specialized along sectoral lines. Training should be increasingly oriented toward generic occupations, such as accountancy, rather than toward sectors. Training in disciplines important for a market economy, for example marketing, must be developed and introduced. Initially the training should focus on short courses to retrain current practitioners, but subsequently it should include courses designed for the formal education system, for example, market economics for general education and entrepreneurship for technical colleges. Employers, the consumers and users of skills, must be given a major voice in determining what training is given and its content. Other key measures include (i) adapting foreign coursework and textbooks to Russian conditions and generating local case studies and other training materials; (ii) expanding the use of varied approaches to short training programs outside formal schools aimed at solving practical problems in the local context; (iii) developing new services that training institutions can offer, for example, research, information, and consulting activities; and (iv) encouraging professional associations to upgrade their members' competence through a variety of shorter programs focused on high-priority topics in support of the economic reforms. Achivig QuaUty Stards 2.46 An effective training system requires that quality be built by (i) setting achievement standards through voluntary accreditation of institutions, preferably by professional associations; (ii) consolidating and rationalizing the currently overbuilt public training infrastructure and concentrating resources on a few excellent institutions; (iii) using competition (that is, open evaluation of competitive proposals by institutions according to standard criteria and transparent procedures) to channel development funds to training institutions; (iv) developing a cadre of master trainers and management consultants; (v) moving away from abstract and excessively theoretical training and introducing new, active, learner-centered teaching methods; (vi) developing practical in-company programs by which to Chaptw n. Te Secto,ra Se*ng Page 17 train managers and employees at work; and (vii) evaluating systematically the effectiveness of training and providing feedback to training suppliers. E&pd Ddvery C4pacly 2.47 Given the massive scale of training requirements, it is vital to extend the delivery capacity for market-oriented training by (i) fostering voluntary communicaton networks among training institutions; (ii) developing or extending capacities for distance learning, for example, through television, electronic mail, and open universities; (iii) using the mass media to raise general public awareness about market econo'nics; and (iv) using the cadre of master trainers and management consultants to pass on their expertise. 2.48 This broad strategy has been translated into a coherent set of acions. This action program, outlined in detail in Annex 1, establishes a sequence for the necessary steps that need to be taken. Annexes 2-5, respectively, present action programs in management development, accountancy and finance, banking, and public finance, respectively. T1E RUSSAN FEDERATION MANAGEMENT AND FiNANCAL TRAIN G PtOJECT mI LESSONS OF ExpnCE ANND BANK STRATEGY A. LESSONs OF EXPERIENCE Setor s kedenc 3.1 Experience in the design and implementation of training projects elsewhere points to the need for sustained support over a series of in-estments foacssing on institutional development. Particular lessons emphasize the need for: a. targeting of foreign training programs; careful selection of fellowship recipients; planning of logistical arrangements for administration of training programs abroad; and avoidance of brain drain or improper placement of returning fellows; b. using alternatives to training abroad, such as the development of local training programs which can tend to be more relevant in content, avoid problems of brain drain and be less costly; c. use of professional expertise, such as in accountancy, in project design and establishment of a dialogue during implementation with members of the profession; d. early and sustained involvement of enterprises, or the users of skills, in overall direction of training programs (this does not happen automatically and incentives must be provided for enterprise managers to participate); and e. strong instiutional management to support modernization programs. 3.2 Training funds managed by national training authorities, such as in Singapore, Chile and Mexico, hve proved useful in financing new training initiatives. Such mechanisms have proved to be eftcdve in adjusting training supply to meet changing needs and organizing training interventions to facilitate economic chage. The fimds with broadest impact are those available to both public and private training providers. Broadly based oversight by the public and private sectors, employers and NGOs has helped to avoid problems in fund administration. 3.3 In sum, the characteristics of successful training programs include: carefully targeted training; responsive, decentralized planning to take into account local requirements; strong linkages to eanprise; quality through standards and on-going staff development with systematic evaluation of resuts; a steady source of financing; strong institutional management; and sustained investment in dweloping institutional capacities. Page 20 _a_A III Lessons of Merence and Ban tratey Expeience in Russia 3.4 There is litte project experience from which to draw guidance in Russia; however, experience in Eastern Europe can be instructive. A shortcoming in the Bank's approach in transition economies was to wait too long to deal with training issues. The initial focus on short-term social safety net issues proved to be incomplete. As the transition took hold, there was litde strength or speed in the supply response partly owing to constraints on market skills and institutional experience. Given the required lead time, issues of market skills must be addressed early to ensure a vigorous supply response when needed in 3 to 4 years. One particular lesson from previous social sector lending in Russia warrants specW mention. Implemenation of the Employment Services Protection Loan has been hampered by lack of familiarity below the level of the head of the Service and points to the need to ensure widespread understanding of, and commitment to, project objectives and mechanisms before loan commitment. Lendg Ixsmwnents Epestenw 3.5 Bank experiences with training funds, social investment funds and sector investment projects (SECILs), somewhat analogous to the proposed project,"1 provide relevant lessons. a. Experiences with training funds point to the importance of (i) careful phasing to match the pace and scale of the project to management cP-abilities; (ii) building ownership and commitment through beneficiary particip&ion in the design of the project; (iii) flexibility, particularly in countries with unstable political and economic conditions, so as to be able to respond to changes at the macro level and to locally diverse needs; and (Iv) the role of consumers and users of skills in deciding on program priorities. b. Lessons from social investment funds show (i) their potendal to establish partnerships between central government, private intests and local authorities and build strong coalitions in support of reform; (ii) the importance of having a clear understanding of the responsibilities of the various partners; (iii) the need for transparent management and isolation from pisan politics; (iv) the potential to build local capacity and distribute benefits widely through trainifig and assistance in the fonmulation of small-scale projects; and (v) the general tendency to underestimate procurement and disbursement issues during preparation. c. Experience with sector investment projects suggests that implementation can be enhanced by a period of trial testing of the appraisal mechaism before loan approval. All the project instruments menioned above stress the importance of frequent and close supervision for these kinds of projects. 3.6 The proposed training project has incorporated these lessons in project design. The overall investment program has been divided into phases to facilitate manageability. The smaller first phase would focus on a core program in four out of the seven priority fields of training. The project 11. lese fuds typically ue a inrmediay to elect and financbprojew. However, the fands differ inat the organiztons are usuaRy government opered and make gras her dan loa. Chaper IIL Lessons of E&perience and Bank Strategy Page 21 centers on the establishment of a training fund with strong institutional management. Enterprises constitute a significant block in the boards of directors and this role should increase over time. The project emphasizes local training programs and economizes on fellowships. It decentralizes much of subproject selection. Systematic evaluation of results has been planned as an integral part of the project from the start. Finally, extraordinary efforts were taken in project development to widen the consensus and involve a broad range of Russian authorities in and out of government. Program proposals were developed by teams of Russian experts assisted by international advisors. Proposals for the training foundation and subsequently regional affiliates concept, and subsequently for each Regional Foundation, were prepared by joint teams incorporating government agencies, non-government organizations and international advisors. Thus, already in project development, an important dialogue has been created between public and private agencies over sectoral issues and policy. The result has been a remarkable degree of consensus and wide-spread commitment to the objectives and means of the project. B. BANK LENDING STRATEGY Triaing Sector Polic 3.7 Bank policy in the training sector is based on the premise that to the extent possible beneficiaries of training-individuals, families, and firms-should finance the costs of training services. Public support for training (in the form of financing for training, and only exceptionally provision of training) is, however, justified where it is necessary to compensate for market imperfections, such as distortions in the labor market; to capture external benefits; to offset weak private training capacity; and to improve equity. The policy, accordingly, calls for Bank support for (i) strengthening employer and private training capacity, (ii) improving the effectiveness and efficiency of public training, (iii) developing alternative sources of financing for training, and (iv) providing long-term support for institutional development. In particular, the policy explicitly recommends that consideration be given to support for national training authorities and establishment of "training funds as a source of targeted grant and loan assistance to employers and private and public training agencies." Further, the policy states that "Bank operations should generally: a. Finance part of a longer-term investment and institutional development program that supports a clear strategy for an evolving government role in training; and b. Seek to mobilize support from other international donors within an accepted policy framework."'2 Radonale for Bank Involvement 3.8 The main objectives of the Bank's Country Assistance Strategy (CAS) for Russia are to: (i) support the development of a market-oriented economy based on private sector initiatives; (ii) encourage the redirection of public sector involvement in the economy toward the establishment of open and competitive markets and the provision of physical, social, legal, and institutional infrastructure not normally provided through the private sector; and (iii) finance high priority public sector investments and policy advice on key economic and sectoral issues. 12. Vocauonal and Technical Education and Tranung: A World Bank Policy Paper," World Bank, 1991, pp. 69-70. Page 22 Chapter IlL Lessons of &erience and Bank Strategy 3.9 Bank involvement in the proposed project is justified by the contribution of the training project to the Bank's global objectives in Russia. The proposed Project is fully consistent with the Bank's CAS. First, by developing the core skills required for a market economy to function effectively, the proposed project would help accelerate the economic transition. As experience elsewhere has shown, economic reforms cannot be implemented adequately without trained personnel. Second, the proposed project would assist in restructuring the labor force. The project would contribute to employment generaxion and more rapid labor redeployment by training people for new professions and self- employment. Third, the proposed project would help to restructure key parts of the enormous training infrastructure and place it on a greater market footing. It would encourage the expansion of private sector training programs and innovations based on quality, standards for accreditation and increased cost recovery, with important spill-over effects on publicly-finance training programs. Finally, the Bank's support for strategic targeting of training investments and for the institutional development of a training foundation should help create conditions conducive to mobilizing official and private assistance for additional investment in priority training programs-a global objective of the Bank. 3.10 The proposed project complements other Bank assistance to Russia. It would provide training pertinent to accounting, auditing, finance and banking, thus supporting initiatives in the Financial Institutions Development Project (FY94). It includes assistance for management development that is consistent with the aims and modus operandi of the Enterprise Support Project (FY94). It would also coordinate assistance with the Employment Services Project (FY93), in part by participation of the Federal Employment Service and regional services in the direction of the training foundation. THE RUSSLAN FEDERATION MANAGEMENr AND FINANCIAL TRAIIG PRoJECr IV. THE PRoJEcr A. PROJECr OBJECTIVES 4.1 The proposed project constitutes the first phase of a long-term program to implement the strategy for market oriented training (paras. 2.41-2.47). The overall purpose of the program,is to increase the quality and supply of skills needed to support the transition to a market economy. Within this context the first phase project has three principal objectives, namely: a. to increase the supply of well-trained, market-oriented practitioners in three core areas: * management, * ithe financial sector, and * public finance; b. to establish an intermediary organization, the National Training Foundation, to mobilize and channel resources for priority market-oriented training programs on a competitive basis (Chapter V). Through the NTF the project would help make aid flows move more effectively and efficiently; and c. to establish the basis for a broader second-phase project by piloting effective interventions in other key market areas, such as small business development, banking, public administration and market economics for the general population. B. PROGRAM DESCRIONS 4.2 Program areas for training have been selected because of their priority in speeding the restructuring of key sectors vital to the accelerated establishment of the market economy, such as enterprise restructuring, financial sector reforms and privatization. The three pmgroga areas are management development; the financial sector; and public finance. Specific sbprograms have been pre- defined for support within the three main program areas. These subprograms are further divided into subprojects that are the principal packages to be financed under the project. Subprojeces are discrete investment packages with objectives, specific activities to achieve them, and a fixed implementation period. The subprojects would be of two basic kinds: (i) devlopment subprojecs whose objectives would be to prepare the structure, content, materials and instructors for training activities; and (i) deiveiy subproJects that would provide training for priority target groups (e.g. enterprise managers, practicing accountants) using various modes, such as classroom teaching, on-the-job training, fellowships and study tours, and distance teaching.) Loan proceeds would finance expert services (both external and Russian); the acquisition of books and other teaching materials; fellowships abroad; and the costs of organizing and delivering media programs as in distance education (including some equipment), local Page 24 Chapter IV The Projea training programs, and studies. The eligible program areas are summarized below. Detailed strategies, action plans, specific subprograms, subproject definitions and pilot projects are presented in the annexes. Magement Deeopment (US$14.8 million of base costs) (Annex 2) 4.3 The strategy for assistance to management development would assign first priority to reforming and upgrading enterprise-based management, relating the training of practicing managers to the main objectives of economic reforms and enterprise restructuring. Enterprises would be directly assisted to implement management development policies and programs that meet the needs of the reform and to retrain managerial staff in high priority areas. In this context, it is expected that the proposed project would collaborate with the Russian Privatization Center and Local Privatization Centers to identify specific enterprises for assistance. Second, external training programs for managers would be strengthened. The restructuring and modernization of both enterprise-based and external training systems would be supported in the critical areas of developing new programs and learning materials, training of trainers, and improving quality. Nonconventional training and learning rnethods, would be promoted, including distance learning and the use of mass media, and management consulting and information services would be strengthened. All programs would stress effective application of the best international and Russian experience. 4.4 Assistance would be structured in five subprograms supporting developments in the following areas: (i) policy framework and quality standards for management development; (ii) enterprise- based management development; (iii) learning materials, trainers, and teachers; (iv) management consulting; and (v) networking and information. Project activities would be started through four pilot projects focused on (i) identifying and removing the barriers hampering management development; (ii) assessing the priority needs of enterprises and preparing for enterprise-based, action-oriented training; (iii) collecting, developing, and distributing recent Russian case materials on management during the transition period; and (iv) designing a database and compiling the first Russian management development directory. 4.5 The project would finance international and national experts and advisers; fellowships for study abroad; contracts with enterprises, institutions, and associations for developing new programs and materials; publication and dissemination of the materials produced; purchase of training equipment and teaching aids; and the delivery of selected in-country training courses for training trainers and consultants. Ike flndl Sector (US$22.0 milon of base costs) (Annexes 3 & 4) 4.6 Accowtanc, auditng, and Jinance (Annex 3). The proposed project, developed in consultation with the Intrnational Accounting Board of the Russian Federation, would finance the implementation of a comprehensive, integrated training strategy focusing on (i) a "top-down" sequence attending first to higher levels of enterprises and ministries; (ii) the adaptation of international accounting and auditing standards; (iii) a focus on income determination, cash flow, and related concepts; (iv) the collaboration among training constituencies; (v) achieving a balance between immediate results (e.g. problem solving for practitioners) and long term institutional development (training of university faculty and curricula revision); and (vi) leverage through self-training and distance teaching to achieve a multiplier effect. 4.7 The principal emphasis would be on building training infrastructure, producing relevant training resources, and developing human resources with programs developed and delivered to senior and chapter IV The Project Page 25 chief accounting, auditing, and finance practitioners. In addition, training programs would be mounted for senior executives of enterprises in accounting and corporate finance. The project would also support revision of curricula in accounting and finance at the university level, development of teaching faculty within these fields and the preparation of computer-based training courses for distance learning. Implementation of the proposed project is expected to make an immediate impact on the financial sector in Russia. During the three year project period, some 750 trainers and master trainers would be developed as a means to train over 7500 practitioners. Also 100 senior executives as well as 80 faculty members would be trained, in addition to the production of 100 Russian case studies, 4 computer-based self-training courses and 4 adapted advanced textbooks, about twenty modules each. The program would be initiated by one integrated pilot project to (i) develop relevant training materials, (ii) train trainers, and (iii) build training capacity through the establishment of one information technology training center in the first six months of implementation. The project would finance international experts, fellowships, Russian experts, equipment, and teaching materials. 4.8 Banking training (Annex 4). The overriding goals of the banking program are to strengthen Russia's banking profession by focusing on staff skill development and to increase student throughput so as to respond better to industrial demand. The principal emphasis of the banking program is on training practitioners. The proposed banking program would help implement overall strategy (Annex 1) by financing subprojects under a banking operations subprogram including: (i) training chief executives in bank management (strategic planning, risk management, financial management, human resources management, and compliance and control systems); (ii) training mid-level managers and officers in credit risk management (credit evaluation, lending policies and procedures, financial analysis); (iii) training mid-level managers and officers in international banking areas (trade finance, foreign exchange, and correspondent banking relationships); and (iv) training managers and officers in investment banking areas (underwriting, securities analysis, securities trading, risk management, investment policies and procedures). A second major subprogram would cover training accountants in bank accounting and managers in how to use this information as a decisionmaking tool. In addition to training the practitioners indicated, each subprogram would emphasize the training of trainers and the development of resource materials, including the 'Russification" of international source materials The program would commence with the initiation of one pilot project to train chief executives in bank management. ublic Finwe T1aIning (US$4.6 miion of base costs) (Annex 5) 4.9 The proposed project is divided into two main subprograms designed to enhance Government capacity to manage public finances efficiently and effectively. The aim of both would be to train civil servants at the federal and regional levels in sound fiscal management practices. The first subprogram would be designed in modular format to provide parliamentarians as well as senior and mid- level officials with relevant training in budget and taxation policy. The second, which would be developed in collaboration with the International Monetary Fund (IMF) and the European Bank for Reconstruction and Development (EBRD), would train new Treasury officials in the Ministry of Finance in budget execution. Each subprogram would involve a limited amount of overseas training for senior federal and regional managers and for master trainers. Both subprograms would begin the process of developing capacity within universities and technical schools in public finance policy to create a talent pool for future recruitment into the civil service. Implementation of the two sub-programs is expected to train an initial Page 26 Chapter IV 7he Project cohort of 20,000 Treasury officials in budget execution and 4,500 senior and mid-level managers who require training in budget planning and tax policy over the next 3 to 4 years.-3 Phase 2 Pilot Projects (US$2.3 mion of base costs) 4.10 In addition to the above three priority areas, the proposed project would build a knowledge base on appropriate interventions in other critical areas for a possible second pbase project. Pilot projects and preparation studies would be financed in such areas as small business, public administration and market economics education for the general public. A pilot project would be financed for the creation of two small business training and support centers. A national strategy would be developed for civil service training. Assistance would be provided on a limited basis for senior federal, oblast, raion and municipal officials in training for public administration. Support for market economics education includes training of journalists in market economics and pilot efforts at popular programming via such mass media as television. 4.11 Dstance education. To be fully successful the project needs to reach large audiences in a short time. If limited to conventional teaching methodologies, the project would likely reach only a few thousand beneficiaries. Distance teaching can help expand this audience exponentially without loss of quality. Even though distance education is not conceived as a self-standing program, all the other subject-matter programs may finance distance education subprojects either as additional or alternative delivery mechanisms or as self-standing subprojects. Distance education activities may be provided through instructional packages for individualized instruction, electronic-mail based systems, narrow- or broadcast means or through teleconferences based on existing technology centers. In the case of individualized instruction, distance education programs may incorporate a range of technologies, particularly printed instructional materials, video-tapes and computer software. These programs may incorporate student support in specified learning centers. Narrowcast or broadcast visual materials can be delivered to support distance-based training activities. Teleconferences could be primarily used for self-standing, short-training activities. 4.12 During negotiations the Government provided assrnces, and the negotiated Project Agreement states that any major change in the definition of the programs described above or addition of new programs would require prior approval by the Bank. 13. A separate stand-alone project would finance administrative modenzton, computzaon and taig programs to improve the revenue collection capacity of the Stae Tax Service. Development and preparation of the project is presendy in progress with the assistance and fini support of the IMF. THE RUSSIAN FEDERAIMON MANAGEMENr AND FINANCIAL TRAINNG PRoJEcr V. INTERMEDIARY: THE NATIONAL TRAINING FOUNDATION (ANNEX 6) A. APPROACH AND JUSTIFCATION 5.1 Faced with myriad and urgent needs for market skills in the above areas, the question is not so much what to do as where to begin and how to manage the process without being overwhelmed. The choice is complicated by Russia's size and the number of sectors, institutions, and levels of administration involved. The project strategy to cope with this complexity is to focus on the development of an intermediary for channeling funds to the beneficiaries. The intrmediary is the Nadtonal Tlbig Foundadon (NTF) (US$5.3 million of base costs), including a central institution with a network of regional analogues. The NTF would be the vehicle for defining strategies, identifying priorities and allocating funds to various locally-generated training projects of high priority. 5.2 The NTF is an independent legal entity, wholly owned by its members. It is located outside the government to avoid vested interests of the sectoral training infrastructure and weak implementation capacity and to cement the involvement of the private sector in its management. A semi- independent agency is less affected by changes of government and would be better at mobilizing resources from the private sector and other donor agencies. Moreover, the NTF is more manageable, in terms of financial control and external donor support, than would be assistance for individual, disparate projects. In sum, the National Training Foundation has been established on the basis of the following organizational principles: a. it is a partnership between government and the private sector; b. it is a non-governmental, independent organization; C. it involves clients, or 'users' of skills (i.e. employers) in establishing priorities for training; d. it is partially decentralized to respond to diverse local needs; and e. it is non-profit, but operates on sound financial principles in managing its investments, resource allocations and in recovering costs. Jakluason 5.3 The central training organization is, in fact, consistent with the need to decentralize training initiatives. The national foundation is intended to be a vehicle to promote training outside the central government. It plans to channel resources both to private training activities, for which development finance is acutely lacking, and to regions in support of decentralization. Regions would select subprojects for financing independendy. 5.4 The creation of a new organization to finance training development and delivery was justified by the following reasons. First, training needs are massive and exceed the scope of any current agency. No single institution currently exists in Russia that has the mandate or ability to deal with the broad range of issues and cross-sectoral activities envisaged by the project. None has the organizational effectiveness to carry out an objective project selection and supervision. In addition, there is a need: Pare 28 Chapter V. Intennediay: the Natonal Thninlng Foundation a. to involve a wide variety of entities from the private sector as leaders in the overall process, to create a partnership between government agencies and private consumers of training (employers, enterprises, professional associations), so as to make it demand-driven; b. for a vehicle through which to provide support flexibly and on a large-scale to the different requirements of the regions; c. for a body that would function as an investment bank in terms of an objective operating and financial culture, which is capable of generating income and operates on a basis of cost-recovery, providing loans as well as grants; d. for a vehicle through which other donors, public and private, could channel funds to high priority projects; and e. for maximization of financial benefits through coordinated activities. No mechanisms existed to coordinate training programs to ensure that priorities are covered and duplication is avoided. lbe NTF is intended to do these things. The NTW is unique to Russia, but has been patterned after succfssful international experiences elsewhere, such as western private foundations, training funds, or govement-sponsored institutions (e.g. the U.S. Science Foundation, or the French Foundation for Mamagemn Development). B. CLNS AND CUSOMERS 5.5 The clients of the NTF, those who are eligible to apply for financing to undertake development work or training projects within approved programs, are restricted to the following: a. educational and training establishments (public and private); b. companies and banks (state and non-government); c. government administrations (central, regional and city); and d. associations (employers, professional, educational). lhese clients may include Russian entities, joint-venture firms or exteral agencies in partnership with Russian organizations. The NWT would not deal with individual applicants. C. ORGANIATION AND STAFFING OF THE NTF Sweuh and Orgmadon 5.6 The National Training Foundation would operate at two levels: a Central Foundation (CF) and Regional Foundations (RFs)-initially in three regions (Moscow, St. Petersburg and Nizhny Novgorod). The criteria for selecting the initial ihree regions were: (i) population coverage; (ii) commiment to economic reforms; (iii) commitment to the project; and (iv) ease of access for supervision. See Diagram S.1. 14. EveatuRy, some 8-10 Regional Foundati would be needed to cover the entie county and could be consdered for odfAldl uas a hlow-on phase to the project Chawer V. Irmendiaiy: Th Nadonail Tinlng Foundaton Page 29 Diagram S1: The Nationl Training Foundation (NMF Network C;entral Foundaion (CF) R0810801 Pegiong F-dR FURTeEin Foundation Foundsaon Foundation REGIONS am (RF) am(R1 OVER TDE SL MM"ow Niziny Pl wsbug ,Novgorod 5.7 The principal activities of the Central Foundation are to (i) mobilize resources within Russia and abroad for the foundation's work; (ii) set standards, accredit regional affiliates and approve new programs; (iii) provide central services; and (iv) select and manage national projec. The principal acvities of each Regional Foundation would be to (i) mobilize resources from within the regions; and ;ii) select and manage regional projects to pre-defined standards. Within agreed programs, criteria and procedres, Regional Foundations would be able to allocate funds from resources earmarked specifically for Regional Foundations. The relationship of the Central and Regional Foundations would be one of specialization by type of project. 5.8 The NTF would funciion as a network. Regional Foundations would apply to the center for accreditation. The accreditation review process would ensure consistency of principle objectives and procedures, but would leave much autonomy for flexibility and local initiative. Even though dependty chartred, each foundation would share common characteristics, such as a general assembly comprised of mixed goverment and non-government members, a board of directors, a small pemannt staff and ad hoc experts for project analysis and evaluation. In addition, the Central Foundation would have a Consultative Council comprised of external donor organizations, to advise on policies, programs and performance, and to serve as a catalyst for external funding. Diagram 5.2 shows the organizatonal structure for the Central Foundation. Page 30 Chapter V. Intermediay: The Natonal Training Foundaion Diagram 5.2: Structure of the Central Foundation of the NTF 50% Govemment p 50% Non-government Assembly I jBoard of Directors Consultative Ad Hoc Expert' |Council (Donors) . . . . ..------------- Advisory Panels' Executive Director External Relations | Programs I Administration and Finance - Extenxal Donors - Management Development - Russian Business Community - Financial Sector - Regional Foundations - Public Finance - Phase 2 Pilot Programs 5.9 The major function of the general assembly would be to nominate the board of directors, analyze their reports, give advice, and help ensure that theNTF handles its financial affairs in accordance with its mandate. The assembly would have the authority to appoint independent inspectors to review operations, and to make its finding public. In the case of the Central Foundation, the voting power on the general assembly is equally split between government and non-government organizations. The non- government representation includes banks and industrial firms, employers and professional associations. No individual training institutions are permitted as a member because of potential conflict of interest. Membership at the regional levels would follow similar principles; participation would be through regional and city administrations, local companies and associations. 5.10 The boards of directors, elected by the respective general assemblies, is the main policy making bodies of the NTF and has overall responsibility for its fund-raising, operations and financial performance. The Directors have a fiduciary duty to members to ensure that funds are spent in accordance with the NTF's mandate. Regional boaras of directors are justified by the need to raise finance from local sources, to ensure that local priorities are established and met, and to appoint staff and apter V. Intermtedary: The National Training Fowdoii Page 31 supervise regional operations. There is a cleardivision of responsibilities between the centerand regions. The board of the Central Foundation would set overall policy for the NTF, approve programs, establish guidelines for operations, accredit Regional Foundations, set financing procedures and operational policies and would select the Executive Director of the Central Foundation. Regional boards would have considerable autonomy-within national policies, programs and procedures-to establish regional priorities, 5 raise external funds, and manage their own internal activities. Coordination would be sought through cross membership: the three Regional Foundations would each have full voting membership on the Central Foundation, and vice versa. The boards would also take decisions concerning the financing of specific categories of subprojects. Boards of directors are expected to meet quarterly or more frequently in the first year of operations. The central Board of Directors, thus, has a total of twenty-one representatives, nine from the Government, nine from non-government and three from Regional Foundations. The nine Government members are the Ministries of Finance, Economy, Labor, the State Committee for Higher Education, the State Committee on Management of State Property (GKI), the State Comittee on Anti-Monopoly Policy and Support of Entrepreneurship, the Office of the President, Office of the Government, and the Central Bank. 5.11 The Central Foundation is managed by an Executive Director, responsible for overeeing planning, management and overall operations. The functions of the Executive Director include providng a secretariat for the general assembly and for the board of directors. The Executive Director is also responsible to advertise programs eligible for assistance, provide adequate information to potential bidders, evaluate subproject proposals, decide on certain categories of projects (e.g. those below $100,000), monitor project execution and evaluate results. *a ffg 5.12 A basic aim is to make each foundation efficient, lean and flexible. Keeping staff umnbers down is an imperative, and ad hoc experts would be mobilized to help with technical activities which do not justify a permanent staff. A total of about 20 employees is foreseen for the initial operation of the Central Foundation, and 6 employees are envisaged for each of the Regioal Foundations, as shown in Table 5.1. Table S.1: Inital NTF Staff Estimates Central RF RF RE Nizhny 0TOAL Lewe Foundation St. Petersburg Moscow Novgorod Stff Management 1 1 1 1 4 Professional 9 2 2 2 15 Support 10 3 3 3 19 TOTAL 20 6 6 6 38 The NTF would need to rely on outside experts to evaluate subprojects and provide overall advice through informal technical advisory committees. Such committees already exist in the management and accounting fields. The numbers of permanent staff required by the NTF would be kept under constant review and adjusted according to actual implementation experience. 15. Regional boards would need to obtain central approval to add new programs, but may select those to be emphasized locally from among centrally spproved programs. Page 32 Chapter V. Intermediary: The National 7Traiing Foundaon 5.13 During negotiations the Government provided assurances, and the negotiated Project Agreement states that the following would be acceptable to the Bank: (i) NTF's accreditation of each regional entity; (ii) NTF structure and organization; (iii) NTF staffing policies, and (iv) employment of key NTF management staff. D. PROCEDUREs 5.14 Transparent and effective allocation of funds to relevant, high quality projects is the principal NTF rationale. The NTF's procedures, there.fore, would follow a clearly defined project cycle, including identification, preparation, appraisal, selection, implementation, supervision and final evaluation. An operations manual has been prepared for the NTF incorporating the main features. Adoption by the NTF of an operations manual acceptable to the Bank is a condtion of Board presentation of the proposed project. The key steps in the process are presented in Annex 6 and are summarized below. 5.15 The Central Foundation of the NTF is responsible for selecting and overseeing the implementation of national subprojects. National subprojects have national scope and applicability, and typically contain components such as training of trainers, curricula and materials development, studies and activities related to national policies or issues of national interest such as professional standards, pilot testing of new project components as well as projects which require or would benefit from national coordination such as mass media or distance education. While managed through the Central Foundation, a substantial share of the products produced by these national subprojects would be implemented outside Moscow, i.e., in the regions. The Regional Foundations select and oversee the implementation of regional subprojects, mainly the training of practitioners in training of trainers and training of practitioners within the region, as well as studies and activities of local scope. In all cases, the actual implementation of subprojects would be done by the winning bidders from the process of competitive selection. The central and regional staff coordinate all decisions relating to implementation of national projects in a region, and would monitor implementation and evaluate results of subprojects in their respective spheres. idendflcation 5.16 The various eligible subprograms have been formulated by Russian experts in collaboration with international experts. Apart from several pilot projects within each category, the specific subprojects that would be financed under each program have not yet been identified. These sub-projects would be decided during the course of implementation. Subprojects could be identified by the NTF (and its donors), training institutions and enterprises. Generally, subproject identification falls into three categories: a. Pre-identfd. These are pilot projects that have already been defined in detail. Proposals for their development and execution would be solicited from a limited number of pre-qualified suppliers. Pilot projects would absorb about 10% of project funds. b. Solited proposals. The NTF would make program announcements about the availability of financing, the requirements for applications, the items eligible for Chapter V. Intemtediwy: 7he National Training Fowndation Page 33 project financing and would solicit applications. Potential beneficiaries interested in working with the NTF and possessing sufficient capabilities for this purpose would be preselected and short-listed. Proposals would be submitted mainly by training suppliers and end-users, e.g. enterprises. Proposed subprojects in this category would be evaluated competitively and would account for about 80% of all project funds. c. Unsolicited proposals. To stimulate innovation and creativity, the NTF would entertain unsolicited proposals for subprojects on any worthy topic within the defined areas of the project. Such proposals, by their nature, would arrive at irregular times and could not be submitted to competitive evaluation. Clear norms have been developed for the evaluation of such proposals. The financing of such subprojects would not exceed 10% of project funds. PrWaration 5.17 Entities wishing to apply for funds under eligible programs would be responsible for preparing their own subproject applications. Standard forms have been developed for this purpose, setting out the important information and analysis required. However, most institutions, particularly at the start of the project, could not be expected to have the technical competence to prepare applications of acceptable quality. Consequently, the NTF would offer training in proposal preparation to interested organizations. The training would include short seminars, repeated frequently, and the preparation of handbooks and guidelines. The training effort is expected to be heavy initially, but could gradually be reduced as potential applicants gain experience with the system. 5.18 Securing sufficient qualified institutions for bidding for the subprojects included in the project is a key ingredient in overall project success. Potential qualified domestic contractors come basically from two sources and are estimated to be sufficient for project purposes. First, the growth of private training institutions has been substantial. Many of the 200 members of the Russian Association of Business Schools, which sets high standards for its membership, have the potential to do quality work on curricula and materials development and training, but cannot do so largely because of insufficient financial resources. Many talented recent graduates are choosing these institutions rather than foreign universities or private companies. Second, external assistance programs implemented during 1992-94 have enabled some Russian educational institutions to establish good working relations with foreign partners thereby forming the basis for establishment of consortiums and joint ventures for bidding under NTF programs. Appralsal 5.19 The program staff of the NTF would screen subproject applications initially to determine whether agreed criteria have been met. Applications that meet agreed criteria would undergo a two-stage review. Proposals would be divided according to category, as specified in Annex 6. Proposals costing less than US$50,000 would be sent to two outside reviewers for a written appraisal; proposals above that amount would be evaluated in a panel review meeting of about five experts, dealing with a group of similar proposals. Each reviewer would receive intensive training in project evaluation techniques and application of the criteria. For very large proposals, the program officer and two experts would visit the applicant's site to check facilities, interview managers, analyze the financial data and probe ability to Page 34 Chapter V. Intermediaiy: The Nati'onal Training Fowdation repay. The program officer would pass the results of the reviews to an Internal Evaluation Committee (IEC) comprised of the Foundation managers. Approval 5.20 The IEC would consider the recommendations of the outside reviewers and decide whether to support the application or request revisions. For those subprojects costing less than $100,000 the committee would make the final approval. Subprojects above $100,000 that the IEC deem worthy of support would be sent to the Board of Directors for approval. General conditions for financing, terms and rates would be pre-defined by the Board for eachi type of subproject, and would not be a matter for case-by-case review. During negotiations the Government provided assurances and the negotiated Project Agreement states that all subprojects estimated to cost more than $250,000 would be sent to the Bank for prior review and approval. The levels for Bank prior review would be adjusted upwards as the NTF gains experience in project selection, so that the Bank would review in advance about 10 to 20% of all subprojects. The vank would make random ex post reviews of smaller sub-projects during supervision visits. The World Bank would evaluate the first 5 subprojects from each entity before official approval regardless of size to verify that the NTF's capacity to undertake subproject appraisal and exercise quality control. The program officer would provide feedback to all applicants whose submissions were not accepted, explaining the reasons. The internal evaluation committee or Board would pass approved proposals to the Administration Manager who would be responsible for signing a performance-based grant or loan document with the beneficiary. Inpkmenaton, Monitoalag and Superision 5.21 The basis for monitoring and evaluation of the contractor's performance would be explicitly included in the forirulation of the contract. The contracts would include, inter alia, definitions of: (i) objectives and scope of work; (ii) milestones of key steps or intermediate outputs necessary to achieve overall objectives; (iii) outputs expected to be delivered; and (iv) criteria that define satisfactory output. The contractors would be provided with guidelines and training on project execution. Monitoring of contractor performance would be the responsibility of the program officer, using a computerized subproject monitoring system (under development). Contractors would be responsible for submitting monthly progress reports on subproject execution using standardized NTF forms. The program officer responsible for checking implemenation, would review the progress reports, visit key subprojects periodically and would prepare quarterly summary reports on the status of the NTF implementation portfolio for the NTW management, Board and for Bank supervision missions. The project monitoring and control system would aim to identify implementation problems and deviations from plan as early as possible. Payments to contractors would be based on verification of achievement of agreed milestones or outputs. Contracts would provide for revisions, reductions in payments or-in cases of irredeemably poor performance-cancellation of the contract. Arbitration would be provided to resolve disputes. Evalnaton 5.22 The proposed project includes provision for evaluation at several levels, from the specific to the general, as follows: individual subprojects, clusters of subprojects with similar goals within a program area, the impact of the NWF programs on Russian needs, and the operations of the NTF itself. Evaluation of individual subprojects would be designed and managed by subproject leaders. These would be self-evaluatibns using NTF guidelines for the purpose of improving subproject performance, and would be supplemented with data provided by program officers collected during their visits to subprojects. Chapter V. Intermediary: The National Training Foundation Page 35 Highly targeted 'cluster" evaluations of program areas would be conducted occasionally by NTF staff or outside experts across a sample of projects. The purpose would be to inform and guide future strategies, policies and objectives of the NTF. Evaluations would also be made periodically about the effectiveness of the NTF's policies, procedures and operations, based largely on feedback from NTF customers. Project impact studies would document the aggregate achievements of the NTF's financing. Annex 7 explains the project evaluation plan in greater detail. Annex 8 presents a set of indicators for monitoring and evaluation of impact. E. CRImERmA 5.23 The NTF would work according to objective technical criteria agreed with the Bank. Essentially, the NTF would select subprojects for financing through a process of evaluating competitive proposals for pre-determined programs. Various sets of criteria would be used, including: a. eligibility criteria, i.e. definition of the kinds of programs and recipients that would be able to apply for financing; b. appraisal criteria, i.e. norms and standards that any successful proposal would have to meet; and c. approval criteria, that is, definition of the basis on which the Board or IEC would accept proposals for funding. 5.24 The NTF would evaluate and approve subprojects on the basis of four (4) major criteria: (i) relevance to NTF goals, (ii) effectiveness, (iii) efficiency in using resources, and, where applicable, (iv) financial capacity. The first criterion, relevance, addresses the extent to which the goals and objectives of subprojects would meet the overall objectives of the project and promise the most direct and worthwhile results in meeting the needs of transition to a market economy. Judgments of relevance would be based on an assessment of the immediacy of impact, and whether projects aim to provide urgently needed skills to high priority target groups. Effectiveness deals with the likelihood that the proposed project would work. Assessments of effectiveness would be based on analyses of clarity, the completeness and feasibility of the proposed design, degree of innovation, mechanisms for customer involvement, the qualifications of staff and consultants involved, strength of management experience, institutional support and provision for leveraging results. The third criterion of efficiency assesses the extent to which projects are designed to produce intended results with the least expenditure of resources. Judgments of efficiency include assessment of training unit costs, utilization rates for equipment and physical facilities, security provisions, and integration of cfforts with other projects to avoid duplication and increase cooperation. The fourth criterion, financial capacities, focuses on the ability of the sub-project to repay loans (if applicable). generate revenues, provide sufficient financial or in-kind resources, and financially support other needs and commitments of the subproject. Annex 9 provides a listing and further explanation of sub-points under each criterion. F. FiNANCIG AND CoSI-REcovERm POLICIES 5.25 Contributions to NTF resources. The NTF would operate as a financial institution and would adopt the accounting structure and financing policies typically found in financing institutions. The members of the NTF system would be expected to provide statutory capital equivalent to US$1.0 million over the life of the project. Initial paid-in capital by the foundere would be nominal. The NTF would undertake a vigorous membership campaign in the first three years to enlist support of the private sector Page 36 Chapter V. Inteirmediary: 7he National Training Foundation undertae a vigorous membership campaign in the first three years to enlist support of the private sector and expand statutory capital. In addition to inquiries by Russian firms, a feasibility study has indicated interest in the NTF by foreign business corporations, which could participate in the work of the NTF through their association with Russian legal entities. Regional and municipal governments would also contribute in "kind" to the costs of the project, e.g. by providing premises and facilities, or by financing training programs and by budgeting resources for NTF operations. These governmental contributions are expected to total US$3 million equivalent in the project period. 5.26 Cost sharing. The project would engage in various forms of cost sharing and cost recovery. Recipients of funds for the execution of subprojects would be expected to contribute at least 15% to the total cost of the subproject. These recipients would usually be training institutions and their contributions would include use of facilities, utilities, supplies and staff. In aggregate, subproject recipients would contribute about $6 million in cash and kind to the financing of total project costs. 5.27 NWT revenues. The NTF would have three principal sources of income: fees, tuition recovery and loan repayments. a. Fees. The NTF would charge modest application fees to help finance the costs of processing applications and to discourage frivolous requests. In addition, it would develop mechanisms to collect "user fees," a commission charged to subproject beneficiaries on the funds it disburses. The NTF would aim at financing an increasing share of its own operating costs through income from these fees. b. Captue oftuition. Participants in training programs developed and financed by the NTF would be expected to pay a share of the costs through tuition fees. Part of these revenues would be returned to the NTF by the organization delivering the training. c. On-lending. The NTF would develop mechanisms for on-lending to qualified beneficiaries, such as enterprises or private training institutions for various development programs, including staff training. Loans for this purpose would be repayable in 3 to 4 years in hard currency at a two percentage point spread over the cost of borrowed funds from the Bank. 5.28 The criteria for determining whether a subproject would be a grant or would require partial or full repayment as a loan would be, first, the nature of the project, and second, the nature of the income stream to be generated by the investment. For the most part, subprojects for practitioners or enterprises would be subject to cost-recovery, while those benefiting the wider public (e.g. curricula development, training of trainers) would be grants. Partial cost-recovery would be imposed on subprojects in which income generation is expected, such as development of textbooks for sale. The training foundations would enter into contracts with subproject recipients specifying terms of repayment. Annex 10 projects possible levels of cost recovery from tuition sharing and on-lending using two different methods. During negotiations the Government provided assurances and the negotiated Project Agreement states, that the NTF would consult with the Bank on the financial terms of the grants and loans provided to subproject recipients. Chapter V. Intermediary: The National Training Foundation Page 37 5.29 NTF repayments. The various means of cost-recovery by the NTF would enable it to repay the government about 20% of the total loanw funds it receives under the project,'6 with higher repayment percentages in any follow-on projects. Annex 11 presents cash flow projections for the NTF. The Regional Foundations would channel repayments to the Central Foundation for final repayment to the Govenunent. During negotiations, the Government and the Bank agreed in principle on the terms for financial transfers from the Government to the NTF. The terms of financial transfers are to be specified in the Subsidiary Loan/Grant Agreement, including repayment of US$8 million within eight years at the cost of World Bank borrowing plus 0.5%, including a four year grace period. Under this formula, about 80% of the funds transferred to the NTF by the government would not be repaid. This is justified because the project is the first phase of a larger program and concentrates on capacity building (curricula, materials development, training of instructors) for which there is little scope for immediate financial return. In a broader sense, government subsidies for this kind of training are justified by the enabling or supporting role of market oriented training for wider economic reforms (i.e. externalities) and by the fact that labor market imperfections prevent full cost recovery. Submission of a letter from the Governmlent confirming the negotiated terms for the Subsidiary Loan/Grant Agreement to the NTF was a condition of Board presentation and it has been met. A condition of effectiveness is signature of the Subsidiary Loan/Grant Agreement acceptable to the Bank between the Government and the NTF. G. LEGAL FRAMWORK 5.30 The NTF, in the absence of a federal law on non-profit organizations, was established by means of registration of its charter with the Ministry of Justice. Because of the gap in the existing Russian legislation governing activities of non-profit foundations, the NTF's charter and internal regulations were drafted in such a way as to provide detailed guidance on the institutional structure and operational procedures of the NTF. The following steps were taken for legal establishment of the NTF: (i) a founders' meeting was held on September 7, 1994 resulting in the signing of a founders' agreement, adoption of the charter and statutes, approval of internal regulations and election of the Board of Directors; (ii) the NTF was registered as a legal entity with the Ministry of Justice on September 29, 1994; and (iii) the NTF has opened a bank account. Legal establishment of the NTF in form and substance acceptable to the Bank, and election of the Chairman of the NTF's Board of Directors following procedures acceptable to the Bank were conditions for negotions of the project. During negotiations the Government provided assurances, and the negotiated Project Agreement states that the NTF would maintain internal rules and regulations acceptable to the Bank. The two additional condiions of Board presentation were met: (i) appointment of an Executive Director of the Central Foundation acceptable to the Bank following procedures acceptable to the Bank; and (ii) adoption of an Operational Manual by the NTF acceptable to the Bank. 5.31 Under the project the Russian Government would channel funds to the Central Foundation, and then the Central Foundation would channel funds to the Regional Foundations in accordance with the allocations specified in the Loan Agreement. In addition to the Loan Agreement between the Bank and the Borrower (the Russian Governent), the Govermnent would sign a Subsidiary Loan/Grant Agreement with the Central Foundation; and the Bank would have a Project Agree.ment with 16. Calculated on the basis of expected returns of loan funds received for the components on which some cost recovery can be expected, i.e. mainly management development ($14.3 million of pro-rated loan amount) and the financial sector ($22.0 miltion). See Annex 11. Page 38 Chapter V. Intermediary: The National 7aining Foundation the Central Foundation describing the project related covenants and conditions. To ensure the Borrower's control over effective allocation of project funds, the representative of the Ministry of Finance in the Board of Directors of the Central Foundation would have veto power over any decision concerning the use of loan funds from the government considered incompatible with the rules of the Foundation. Upon signing of the Subsidiary Loan/Grant Agreement, the Central Foundation would conclude Subsidiary Financing Agreements with three accredited Regional Foundations on allocation of funds for regional programs to be selected and financed by the Regional Foundations. Concluding a Subsidiary Financing Agreement acceptable to the Bank with the Central Foundation would be a condon for disbursements to each Regional Foundation. 5.32 During negotiations the Government provided assurances, and the negotiated Project Agreement states, that the NTF would maintain procedures, guidelines and criteria for subproject processing, approval, implementation and evaluation acceptable to the Bank. Assurances were also obtained at negotiations that the NTF entities would submit annual work programs and budgets for the following year to the Bank for review and approval by December 1 annually during project execution. T1E RUSSIAN FEDERATION MANAGEMeNT AND FINANCIAL TRAINING PROJECr VI. PROJECT CosTs, FINANCING, MANAGEMENT AND bLEmENATION A. PROJECT Casr SuMMary of Project Cost 6.1 The total project cost is estimated at about US$51.4 million equivalent (158.5 billion Rubles), including contingencies amounting to US$2.4 million which represents 5% of the base cost (US$49.0 million). The project costs include the provision of technical assistance (both foreign and local), fellowships, equipment, training materials and software, minor refurbishing and furnishing of facilities, in-country training, and incremental operating costs incurred during the implementation period. The estimated costs by component and category of expenditure are shown in Tables 6.1 and 6.2, respectively, and are futher detailed in Annex 12. Table 6.1: Project Costs Summary by Project Componentt (Rubles billions) (US$ millions) % % Total For-eign Base Locl F Lreign g xcn Costs Management Development 20.5 25.1 6.6 8.2, 55 30 The Financial Sector 35.7 32.2 , 11.6 10.4 k 47 45 Public Finance 6.9 7.3 I 2.2 2.4 5 52 9 Phase 2 Pilot Projects 4.2 2.9 1.4 0.9 4 41 5 Program Areas Subotl 63.1 64.6 20.4 21.0 51 84 Nadtnal Trn 7 Foundaion 11.7 4.7 3.8 1.5 28 11 Totald sadine Costs 79.0 72.2 7 25.6 23.4 48 100 Pdce Conatgendes 3.8 3.51.2 1.1 48 5 TOTAL PROJECT COSTS 82.8 75.7 26.9 24.5 48 tFPigores tay not total due to rounding. Page 40 Chapter K1. Project Costs, Financing, Mawgement and Implmenation Table 6.2: Prject Costs Summary by Category of Exedltr' (Rubles billions) (USS millions) '§'; i % Tota Foreign Base Local Foreign p Local Forign Exchange Coas I. Ipavslae Cosw A. Technical Assistance 1. Consulting Srvices Foreig 0.0 240 . 0.0 7t8 Z 1oo 16 Local 304 9.9 00 0 20 &ubotl Cosuing Svnices 30.4 24.0 4 9.9 7.8 ? 44 34 2. Overseas Felowships 0.0 19.0 0.0 6.1 100 12 Subtotl Tedhical Assistance 30.4 43-0 V 9.9 13.9 s 59 48 B. Eqipmet 0.0 18.9 0.0 6.2 to 10 13 C. Production and Disiibuton 6.2 0.7 2.0 0.2 to 10 s D. Training Delivery 38.6 9.6 12.5 3.1 * 20 32 Totl hawveme Cot 75.2 72.2 | t 24A 23.4 4 49 97 II. Reelorre CAWz Operatg Costs 3.8 0Q0 1.2 0.0 $ 0 3 TtalW Reciu Cats 3.8 0.0 1.2 0Q0 0 3 Toa Back" Cost 79.0 72.2 48 25.6 23.4 4S 100 Pfice Contigenis 3.8 3.5 1.3 1.1 48 5 TOTAL PROJECT COSS 82.8 75.7 f 26.9 24.5 48 'Figures may not total due to rounding. Bass of Cast E3dha 6.2 Project costs were estimated as follows: a. Base est,. All project costs are based on November 1994 prices and conversions from the local currency at the market rate of 3085 Rubles per US dollar prevailing on November 4, 1994. Technical assistance (TA) costs are derived from recent fees quoted for comparative specialized services in Bank- financed projects in Russia and are estimated per person-month at US$20,000 for external specialist services, US$2,800 for local specialist services, and US$6,000 C%ater VI. Project Cots, Financing, Management and Ihplementation Page 41 for overseas fellowships. The person-month costs of TA include fees, recruitment/relocation costs, housing and subsistence, and overheads. Equipment unit costs are based on recent quotations from suppliers of equiprmt and software; staff unit costs are based on current competitive rates for local staff in the private sector. b. Contngency allowances. Price contingencies for domestic prices increases are calculated at projected Ruble inflation rates of 72.2% for 1995; 29.1 X for 1996; and 15.7% for 1997. International inflation rates are included at 2.0%, 2.5%, and 2.7% for 1995, 1996 and 1997.17 c. Foreign exchange component. The foreign exchange component was estimated as follows: (i) foreign expert services and fellowships - 100%; (ii) local expert services - 0%; (iii) equipment - 100%; (iv) production and distribution - 10%; (v) training delivery - 20%; and (vi) incremental recurrent costs - 2% of base costs. The total foreign exchange component is estimated at US$24.5 million including contingencies, or 48% of total project costs. d. Customs dies and taxes. Project costs include taxes at 33% on local salaries, and customs duties of 20% on equipment imports (computers, software and service). Taxes and duties are estimated at US$2.8 million. e. inremental recurrent costs. Incremental recurrent costs are estimaed at US$1.2 million and include cost elements such as local staff salaries and facilities operations expenses, and include proportionate contingency allowances. Technial Ass _tance 6.3 The proposed project would finance a total of about 33 persons-years of external expert services, 293 person-years of local expert services and 86 person-years of fellowships abroad. Annex 13 gives a detailed picture of technical assistance included in the project. Table 6.3 summarizes technical assistance by category. 17. Source: Index of Unit Value of Maufactured Exports (MUV); 3 ...rter. 1993 Page 42 Chapter VI. Project Costs, Financing, Management and Implementation Table 6.3: Proposed Technical AssistanW (staff/years) _ peit Senlces Category Foreip Local Fellowships Policy" 0.4 3.6 1.1 Project SupportY 3.6 16.6 7.4 Institution Development 28.5 273.1 77.2 TOTAL 32.5 293.3 85.7 d'Subprogram on -Policy Framework and Quality Standards" under the Management Development Component hi Half the planned assistance to the NTF plus Phase 2 pilot projects. B. PROJECT FINANCING 6.4 The proposed Bank loan of US$40.0 million would finance 78% of total project costs. Of this, US$23.9 million would finance 98% of the foreign exchange costs of the project and US$16.1 million of local costs (excluding taxes and duties), or 60% of local costs. The Government of Switzerland would finance US$1.1 equivalent (2% of foreign exchange costs and 2% of local costs), or 2% of project costs. The members of the National Training Foundation would finance US$1.0 million of the project's local cost requirements, or 2% of project costs, through financial subscribers to the NTF; Russian subproject beneficiaries would finance US$6.3 million or 12% of project costs; and the Federal and Regional Governments would be responsible for US$3.0 million or 6% of project costs. Table 6.4 outlines the distribution of financial responsibility. Table 6.4: Thiann Plant Total Foreign Exchange Local Costs Project Costs ~~% US$M % US$M % Project Government (Fedeal/Regional) 0 0 3.0 11 6 National Training Foundation 0 0 1.0 4 2 Subproject Beneficiaries 0 0 6.3 23 12 Govemment of Switzerland 0.6 2 0.5 2 2 World Bank 23.9 98 16.1 60 78 =~ =.- = TOTAL 24.5 100 26.9 100 -- 100 t Figures may not total due to rounding. Chapter VT. Projed Costs, Financing. Management and Irplementation Page 43 6.5 The effort by the central and regional governments to fund the project during a time when traintng resources and GDP are collapsing demonstrates a clear conumitment to the project objectives. The justifications for local cost fmancing by the Bank are: (i) the project follows a strategy of concentrating training development and training delivery in Russia so as to build local capacity and economize on project costs; inevitably this makes the local cost component high as a proportion of total cost; (ii) pressures on federal and regional budgets at this time preclude a full govermnent contribution to these local costs; and (iii) the Bank's participation is necessary, as an extraordinary measure, to establish the NTF and start its preparations and training activities with significant expenditure on local costs. 6.6 The Swiss Govermment intends to cofinance the project with USS$1. I million equivalent for an integrated pilot project in the field of accountancy, audit and financial management. In addition, several donors have indicated an interest in parallel financing for specific NTF subprojects that require foreign experts or fellowships. These resources are not likely to become available until after the NTF is operational and subprojects are tendered. C. MANAGEMET AD IMPLEMEATON 6.7 The proposed project implementation would be monitored and supervised by the staff of the entities of the NTF. Subproject execution would be implemented by winning bidders. Stff Seletion 6.8 The NTF would be almost entirely dependent on the quality of its staff. There is not likely to be a ready-made reservoir of people experienced in the work of the Foundation and an exhaustive process of staff selection is needed to ensure recruitment of the right talent. The Executive Director, eventually assisted by a committee of managers, is responsible for staff recuitment. The intention is to advertise available positions widely, followed by a seminar for applicants on Foundation activities, written tests and interviews by the selection panel. Sfff Developmen 6.9 Training of NTF staff, consultants and clients is essential for project success. Various training programs would be implemented, as shown in Table 6.5 Table 6.5: Types of NTF In-House Training Target Group Content 1. Board Members Role and Functions of NTF 2. NTF Staff, Board State-of-the-Art in Program Fields 3. NTF Staff, Clients Project Preparation Requirements 4. NTF Staff, Consultants Project Evaluation 5. NTF Staff, Clients Subproject Evaluation Design 6. Clients Subproject Execution Page 44 Chapter VI. Project Costs, Finandng Managemen and Implementation The proposed project would finance these training programs both for entry-level and in-service on a continuous basis. Training would include a variety of methods, namely: seminars, simulations, structured feedback to participants, self-learning packages and audio or video tapes for subjects that need to be repeated. Annex 14 provides details on the NTF staff development plans. Implemenaion Schedule 6.10 A detailed and reasonable schedule for implementation of the proposed project has been prepared. The number of subprojects to be approved under the pro,ject can only be tentative at this stage, but it is estimated that the program would encompass about 120 subprojects (Annex 15). Approximately half of the subprojects would deal with the development of training programs and would be managed and supervised by the Central Foundation. The balance of 60 subprojects pertain to the actual delivery of training programs and would be managed largely through the three Regional Foundations. However, to speed project start-up, the Central Foundation would be allowed to engage in the delivery of training to practitioners in the first year of project implementation. The implementation schedule allows implementation procedures to benefit from experience in processing the initially approved subprojects. Pilot projects have been prepared so that implementation may begin early. Announcements for eligible programs have been phased so that the volume of applications would not exceed staff capacity. Implementation of approved pilot subprojects would begin within three months of loan signing, all subprojects would be approved within two and a half years of signing and implementation of the last subprojects would be completed within three and a half years. The Completion Date and Closing Date for the proposed loan would be June 30, 1998 and December 31, 1998, respectively. Annex 16a shows the overall implementation schedule for the project and Annex 16b presents the steps necessary for the start up of the NTF. 6.11 Status of Preparation. The project is at an advanced stage of preparation. The NTF has been established by its Founders and has been registered as a legal entity. A Board of Directors has been elected by the NTF membership. The Board has elected a Chairman and has appointed an Executive Director for the Secretariat. Regional Foundations in Moscow and Nizhny Novgorod have already been created and managers appointed, although these need to be accredited by the Central Foundation. A detailed and extensive operations manual has been prepared, consisting of administrative manual and operational guidelines. Pilot projects have been designed in detail in accountancy and management training and the Project Preparation Facility (PPF) has been provided to finance initial project start-up. D. PRocuREmENr 6.12 Procurement under the project would be mainly the responsibility of subproject contractors, i.e. individuals, firms, agencies, consortia, etc. winning competitive bids to execute subprojects. The Central Foundation would also purchase goods and services for itself and the Regional Foundations. 6.13 Primary responsibility for overseeing implementation of procurement procedures would rest with staff of the Central Foundation. The Central Foundation would communicate with the Bank on procurement matters and actions requiring clearance. Staff of the Central Foundation would provide technical guidance to Regional Foundations regarding the organization of procurement activities, documentation and administration of contracts. NTF personnel involved in procurement would be guided and trained by technical assistance hired by the NTF, by specialists engaged in companion, Bank-financed ataper VI. Project Costs, Findnng. Managemt and Impl_bme ion Page 45 programs, and by standardized Foundation procedures, manuals and standardized contracts already under development. Technical assistance personnel would provide intensive, recurring training acceptable to the Bank for staff. 6.14 Table 6.6 summarizes the procurement approach envisioned by category of expenditure and approximate funding amount. Procurement in all categories would be implemented in accordance with relevant World Bank Guidelines and standard bidding documents (SBDs) and contracts, as appropriate. Table 6.6: Summary of Proposed PrKurenent ArraDemnentst (US$ milion) Procuremeut Method tnurn_dwal Local Competive Compeive Not Bank Bidding Bidding Other Pinanced MI A.Tefhnlcal Assistanem F_ndaEonxpeu - - 2.9" 0.2 and Staff (2.7) Trining Delivesy/Services - - 27.3a 8.1 (25.S) Fellowships - 6.3" .''. (6.3) S. Equlmmt 1.5 - 10 0.6 (1.5) (1.0) C. Maerals Producto - 0.70' 0.8" 02 1 (0.7) (0.8) (4 D. NTF Opeatins is0.9" 0.2 (0.9) F. PPF 0.6" 0. * _ _ _ _ _ _ _ _ _ _ _ _ ___________ (0.6) TOTAL. 1.5 0.7 39.8 9.4 . (Dank portIons) ~~~~(1.5) (0.7) (37.8) tPms may ot tota due to rounng. Fgres parenhss are the respective amounts fnanced by dte World Bank. I Services ($36.5 millon) would be procured in accordance widh Wodd Bank Guideline.: Use of Consultants, 1981. bI bcudes DC (propriebuy) training inpuft (US$0.4 mlion) for comuter sysem and software upgrades, and IS purse for off4he4ielf items (US$0.6 milion). d Goods and mateials (uding soue ancila instlaio sevice) prowured locally, usig advertised, mpetve bidding, total $0.7 million. ' Local Shopig for taiing materis and supplies (US$0.8 million). Expenditur (US$0.9 million) will be effcted using Loca Shopping and SOE diburswment procedures, per following *lt "A Poet Praration Facility (PPF) of $0.6 milion is intended to be used for initid consuling services ($0.3 million) for sWt-up equipent ($0.1 million), and for opeating coss ($0.2 million). Page 46 Chapter VI. Project Costs, Financing, Management and Implementation 6.15 Overall approach. The loan would finance about 190 contracts for subprojects and equipment and services procured by the Foundations. Three-fourths of the loan funds would finance contracts for training development and training delivery services, including fellowships. These consulting services would be procured in accordance with Bank Guidelines: Use of Consuiltants by World Bank Borrowers and the World Bank as Executing Agency, 1981. Solicitation methods and terms of reference for all consulting services procurement would be subject to prior Bank review. Tender documentation, contracts and amendments for consulting services valued at greater than US$100,000 would be subject to Bank prior review. Also, tender documentation, short lists, contracts and amendments for the engagement of individual consultants would be subject to prior Bank reviews, as indicated in the Guidelines (Appendix 1, paras. 2-4), for all actions expected to exceed US$50,000 per transaction. Consulting services contracts below these levels would be subject to ex-post reviews during supervision missions and as part of annual procurement implementation reviews. Contracts below US$50,000 that are paid by withdrawals from the loan on the basis of Statement of Expenditures do not have to be submitted to the Bank for its records. Procurement and contracting would use standard Bank formats wherever possible or Bank-approved Foundation procedures and documentation for the smaller-value purchases. Some of these training delivery services might also be performed under requirements-type contracts to cover repetitive courses for similar target audiences. The implementors would be responsible for acquiring all the equipment necessary and otherwise provide the complete training services referred to in the contracts for which they are responsible. One subproject would require ICB procurement for computer technology centers that would be handled by procurement staff of the Central Foundation. The Central Foundation may also do some minor materials procurement for subproject contractors if they are not, in the view of the Foundation, able to manage the procurement process. Procurement of services and goods are expected to be undertaken within the following categories. 6.16 Consulting services. Consultant services for training development and training delivery subprojects would be packaged in about seventy contracts, averaging US$364,000 each and totalling $25.5 million. Procurement of fellowships would involve about fifteen contracts, totalling about US$6.3 million. Subproject contracts would be structured to minimize contract administration and duplication of training activities within and between regions. For example, curricula, teaching programs and training materials would be developed only once in any given subject area. Subproject services would be tendered on a competitive basis and using short-listing of qualified firms as provided for in the Bank's Guidelines for the Consultants. Subproject contracts would utilize either a lump-sum approach according to pre- established payment milestones or, when appropriate, use a time-based schedule of services paid for at rates specified in the contract. Advertising of contract opportunities, applying Bank-approved short-list competitive selection procedures, and proposal evaluation criteria would ensure access to a qualified service providers and participation of a variety of institutions, both local and foreign. Partnerships, joint ventures and other forms of association would be encouraged. Potential bidders would be encouraged to combine technical expertise with local and/or international experience for handling the more complex training activities. Within a subproject, the winning contractor would be responsible for providing the resources, including executing subcontracts, if needed, with other firms or individuals to purchase goods and services to carry out the defined work. 6.17 Procurement of services by the NTF. The Training Foundation would also procure consulting services for itself, e.g., analyses, evaluations, advice on curricula development and training of Foundation staffs. Three types of services contracts would be solicited. The first group of larger contracts would be with firms or individuals for technical advisory services to the Foundation or for subprojects directly managed by the Foundation. A total of about fifteen such contracts is envisaged, Chapter Vl. Project Costs, Financing, Managemen! and Implementation Page 47 averaging about US$180,000 per contract and totalling about US$2.7 million. The second grotup of services would be obtained via smaller contracts with firms or individuals to help Foundations administer the program while minimizing the number of permanent employees. In accordance with this principle, the Foundation would enter into consulting contracts with individuals or firms for specific tasks, such as evaluation of subproject bids, supervision of training delivery and evaluation of results. These contracts are expected to number about fifty, but would be relatively small, averaging under US$36,000 each and totalling about US$1.8 million. The NTF would also do some small-value purchasing of services, such as plant repairs and maintenance, in connection with its office operations budget. 6.18 Procurement of goods. International competitive bidding, direct contracting, local comipetitive bidding and local shopping would be utilized, as appropriate, for the procurement of goods and perhaps small-value supporting services required for the program, facilities upgrades, and operations of the Foundations. Loan-funded procurements are expected to total about forty actions and would amount to about $4.3 million. ICB procurement will be in accordance with the Bank's Guidelines: Procurement under IBRD Loans and IDA Credits, dated May 1992. The NTF might assist subproject entities by procuring services on an aggregated basis where it would be more efficient to do so centrally. 6.19 International Competitve Bidng. International Competitive Bidding (ICB) procedures, outlined in the above mentioned Guidelines, are expected to be used for the purchase of computer equipment for the proposed accounting skills technology centers. These procedures would apply to all such purchases that would exceed $300,000 per contract. Of the total of $1.5 million scheduled for these purchases, there would probably be four individual tenders to support the centers' installation schedules. These procurements would be administered by Central Foundation staff to ensure compliance with the Guidelines and Bank standard documentation and approvals. In the comparison of bids for ICB purchases, Russian manufacturers would be allowed a preferential margin of 15% or the existing customs duty that a nonexempt importer would have to pay, whichever is lower, over the competing price of imports, providing domestic value-added is at least twenty percent. Bank staff would review and clear these ICB procedures, documentation, evaluations and resulting contracts on an ex-ante basis. 6.20 Direct Contracting. Direct Contracting (DC) may be used for the procurement of some proprietary computer system upgrades, spare parts, audio-visual equipment, and vendor specific training technologies. It is expected that Direct Contracting documentation, awards, contracting and, if any, amendment actions would be subject to prior Bank review and approval. Direct c ontracting is expected to involve about three purchases, averaging $133,000 each. 6.21 International Shopping. International Shopping (IS) may be authorized for readily available, off-the-shelf purchases under US$300,000. IS will be based on quotations from at least three suppliers, from at least three different Bank-member countries. Training activity International Shopping is expected to total about US$600,000 and involve six purchases averaging US$100,000 each. Bank reviews of IS actions will be conducted on an ex-post basis during supervision missions or procurement reviews. 6.22 Local Competitve Bidding. Local Competitive Bidding (LCB) procedures would be established by the Central Foundation for its own use and that of the Regional Foundations and would be cleared in advance by Bank staff to ensure such purchases will be carried out in a manner acceptable to the Bank. These procedures will be used for actions that are expected to result in contracts of less than US$300,000 that would not likely be of interest to foreign competitors, such as minor works that are Page 48 Chapter VI. Project Costs, Financing, Management and Implemenation labor intensive, i.e., modifying office space to accommodate computer networks and equipment. LCB actions (about five) are expected to total about $700.000 and would be subject to Bank prior review. 6.23 Local Shopping. Local Shopping (LS) procedures would be primarily used for purchases under US$50,000 of locally available, off-the-shelf itemis of small value and standard specification. This method may also be used for small value transactions for simple equipment installation work. Local Shopping purchases to support course development and training delivery would total about $800,000 and involve about twenty contracts. Most of the US$900,000 of NTF operating expenditure purchases would use local shopping procedures for obtaining office supplies, consumables and some minor, off-the-shelf equipment. Utilities and similar periodic expenditures would be paid as invoiced and included in NTF Statements of Expenditures. Small value operating expenditure purchases would number about twenty- five actions and be subject to ex-post documentation reviews, with payments probably being effected under the project's Special Account. 6.24 Procurement implementation The NTF would be responsible for overseeing a total of about eighty-five medium-sized and about one hundred small contracting actions under the project. This is a sizeable number, particularly for a new organization. However, steps have been taken under the project to ensure that it is a manageable workload for the NTF. (i) The total number of medium-sized contracts for subprojects averages out over the life of the project to about eighteen contracts for the Central Foundation per annum and about five contracts per year for each Region. In addiction, the approximately fifty small contracts entered into by the Foundations for local expert services would be limited in scope (similar to 'work orders"), would follow standard terms of reference and formats, and, being mostly fixed-price, would not be difficult to administer. (ii) The Foundation would have a procurement advisor and several full-time, permanent staff responsible for initiating procurement planning and administration. (iii) External technical assistance, both long and short-term, would also be retained in the Central Foundation to assist with development of a procurement procedures manual and implementation plans. These expatriate specialists would help Foundation staff develop personal skills and institutional capacities, via intensive coaching and site visits, and establish required reporting systems. (iv) There would also be extensive recurrent training of program staff, including the development of actual terms of reference and discussions with Foundation staffs on the implications for solicitations, proposal evaluation, negotiations and monitoring. (v) Given the nature of services to be procured, the project would make extensive use of standardized procedures and standardized contracts. Sample operational documents, terms of reference and criteria to identify potential participants are being developed by experts currently in place. In view of the above, the number of contracts is considered to be manageable. Bank Procurement Oversight 6.25 In summary, all terms of reference for services, subproject and consulting services contracts over $100,000 and with individuals over US$50,000, and ICB and DC purchases of goods would be subject to prior Bank reviews and "no objections," as outlined in the Bank's respective procurement Guidelines. Prior reviews of the Foundations' LCB procedures and standard documentation would also be required. Bank reviews of subproject and other consulting services procurements would cover an estimated eighty percent of total Bank funding for these inputs. The procurement plan would also result in the Bank's prior reviews of approximately seventy percent of direct NTF purchases of goods. Given the nature and value of the contracts and because standard bidding documents will be used, this percentage of prior Bank review is justified. Chapter V7 Project Costs, FOnancing, Managemem and Implmation Page 49 6.26 The Bank would monitor procurement activities, contract management and project record- keeping during periodic supervision missions. To assist wihi implementation of the project, the Bank would recruit a full-time local staff menber to work at the Resident Mission. The main function of this Implementation Specialist would be to provide close reviews of the procurement process, including the review of documentation in Russian. In addition, the Bank would undertake a focussed project launch workshop and an intense mid-term review of project progress; both activities would place emphasis on procurement matters. Additional information on procurement supervision can be found in Annex 18. Procurement Reponing 6.27 As outlined in relevant guidelines, NTF managers would promptly inform Bank staff regarding procuremnent plans, solicitations, proposed contract awards and progress in the implementation of their procurement activities. NTF contracting and purchase actions would also be summarized in periodic submissions and operations reports prepared for the Bank by the Central Foundation. E. DISBURSEMENTS 6.28 The proceeds of the Loan are expected to be filly disbursed over a 48-month period, with most of the disbursement occurring in the last two years. Funds disbursed under the PPF would be repaid from the proposed Loan funds at effectiveness. The estimated disbursement schedule is shown in Table 6.7 below. Table 6.7: Estmated Loan Disbusement Schedulet Disbursements Over 48 Months Bank Fiscal Year 95 96 97 98 99 semester (6 mondhs) 1 2 3 4 S 6 7 8 Per Semester (%) 1.0 2.5 5.0 8.0 16.0 24.5 28.8 14.2 Per Semester (US$ millions) 0.6 0.8 2.0 3.2 6.4 9.8 1 1.5 5.7 Cumulative (%) 1.0 3.5 8.5 16.5 32.5 57.0 85.8 Cumulative (US$ millions) 0.6 1.4 3.4 6.6 13.0 22.8 34.3 $ t Figures may not total due to rounding. Page 50 amChpter VI. Projea Costs, Flnandng, Management and Imlementation Chlart 6.1: Dlsba M lustrated Per Semester and Cumulatively (Dank Fia Years) 40 35 30 *Pet5em aS OQmnutaie 20 S~ FY9S-2 FY9E-1 PY96-2 PY97-1 FY97-2 FY8-i FY9-2 FY99-1 &emeowm 6.29 Disbursements would be made as follows: Amount Allocated PerceAge of Exendtu Ca£eQorv (US$ milnion) to be Financed 1. Subprojects US$ 22.1 85% 2. Goods US$ 4.3 100% of foreign expenditures; 100% of local expenditures (ex- factory cost); and 75% of local expenditures for other items procured locally 3. Technical Assistance US$ 9.7 100% 4. Operating Costs US$ 1.2 100% of expenditures incurred during the first year and second year; 75% of expenditures incurred during the third year; and 50% of expenditures incurred thereafter 5. Refinding the Project US$ 0.6 Amount due according to Preparation Advance PPF Agreement 6. Unallocated US$ 2.1 Chapter VI. Project Costs, Financing. Management and implementaton Page 51 6.30 Funds for subprojects to be administered by Regional Foundations would not be pre- allocated to each region in the disbursement schedule. The reason is to give regions an incentive for good performance. Those regions that can commit and disburse funds for sound subprojects would be able to absorb additional resources above their nominal shares. Documentation 6.31 Disbursements would be fully documented, except that contracts for goods and training under US$50,000 equivalent, and operating costs, would be made on the basis of certified Statements of Expenditure (SOEs). The required supporting documentation would be retained by the respective foundations for at least one year after receipt by the IBRD of the audit report for the year in which the last disbursement was made. This documentation would be made available for review by the auditors and by Bank staff on request. Special Accounts 6.32 To facilitate payments of small amounts and payment in local currency, the Government would initially establish one Special Account denominated in US dollars in a commercial bank acceptable to the IBRD. The selection process and qualifications of this bank are described in pam 6.33. The authorized allocations of this Special Account would be US$800,000. All requests for replenishment would be submitted to IBRD through the Central Foundation. Documentation requirements would follow the same procedures as described in para. 6.31 and in the Disbursement Handbook. In addition, monthly bank statements of the Special Account which have been reconciled by the Foundation would accompany all replenishment requests. 6.33 The commercial bank holding the Special Account would be selected from a number of acceptable banks. This bank would also collect repayments on loans from sub-borrowers. The cost of the services provided by the commercial banks will be compared. Some commercial banks in Moscow have already received IBRD approval and the names of those banks will be communicated to the Borrower once the Loan is signed. The bank selected to hold the Special Account should (i) be financially sound as demonstrated by their latest accounts, as audited by an internationally recognized auditing firm; (ii) have a significant foreign correspondence network covering all major currencies; (iii) have a reasonable capacity and experience for issuing letters of credit, for making direct foreign payments and other international transactions with appropriate arrangements for the training and development of their staff; (iv) be capable of performing a wide range of local banking services, including cash payments, transfers to other domestic banks, issuance of debit notes, application of conversion rates from foreign currencies, and maintenance of adequate accounts for the Special Account; (v) preferably be a member of Swift and should routinely use or plan to use this facility to expedite payment transfers; (vi) be willing to issue a Comfort Letter to assure that amounts deposited in the Special Account will not be set off or otherwise seized or attached to satisfy amounts due to the commercial bank by the Borrower; (vii) be willing to maintain adequate accounts as required by IBRD, provide montfly bank statements to IBRD, and any other information considered necessary (copies of all transactions would be routinely submitted to the foundations); and (viii) be willing to charge reasonable rates for their services and provide reasonable interest on balances held from time to time. Page 52 ater Vl. Project Costs, Flnandng, Management and Implementation Project Accownts and Audits 6.34 A single set of project accounts would be maintained by the Central Foundation for the Project. The project account, including the Special Account, would be audited in accordance with the "Guidelines for Financial Repordng and Auditing of Projects Financed by the World Bank" (March 1982). During negotiations assurances were be obtained from the Govenunent that the IBRD would be provided, within six months of the end of each Govermnent fiscal year, with an audit report of such scope and details as the IBRD may reasonably reques, including a separate opinion by an independent auditor acceptable to the IBRD, on disbursement against certified SOEs. The separate opinion should mention whether SOEs subniitted during the fiscal year, together with the procedures and internal controls involved in their preparation, can be relied upon to support the related withdrawal applications. Bank Supervision 6.35 The proposed project focuses on the establishment of a new institution, the National Training Foundation, through which many small subprojects would be implemented in high priority fields. Intensive Bank supervision would be required to help identify and solve problems, and to ensure that the activities of the NTF conform with plans and agreements. The Bank's supervision strategy would focus on early and frequent supervision of the institutional development of the NTF. rhe Bank's supervision of subproject activities would concentrate on the monitoring and review processes of the NTF as the first-line supervisor of subprojects. Steps have been taken in the operational guidelines to require adequate reporting procedures by the NTF. A total of 30 staff weeks has been budgeted for the rest of FY95 and about 40 staff weeks for subsequent years. Such close supervision by the Bank would also be a likely requirement of other potential donors. A major mid-term review of the project and the NTF's operations is planned for about 18 months after loan signaure. See Annex 18 for the supervision plan. F. SusrAINABILITY 6.36 The aim is to put the NTF on a permanent, self-sufficient, financial foundation that could finance its activities in the long run. In the short run, the NTF would finance some of its operating costs through user charges to subproject beneficiaries. It would also generate revenue by receiving a share of tuition payments collected by training providers. In the medium term, the NTF also needs to develop on-lending arrangements for its commercial clients. Any spread between its income from repayment of loans by sulborrowers and its repayment to the central government could be applied to operating capital. In the long run, fill self-sufficiency in its financing activities would require an endowment fund and/or sustained contributions from outside donors. These can only be defined and realized on the basis of the NTF's actual performance. Technical sustainability of the NTF would be sought through staff training programs, counterpart training by technical assistance experts and maintenance of a competitive salary structure for staff. THE RUSSIAN FEDERATION MANAGEMENT AND FINANCIAL 1TAINING PROJECT VII. PROJECT BENS AND RISKS A. BENEFITS 7.1 The project's immediate outputs would be (i) the development of an umbrella organization able to define training strategies and evaluate locally-generated subprojects according to technical standards; (ii) the creation of new and better training resources in management, accountancy, banking and public finance, including teaching standards, curricula, trainers, teaching materials, and networks; (iii) the creation of new capacity to deliver training, including the development of a private training nfrastructure, local training capacities, distance learning, and in-company training. The ultimate output would be substantial numbers of people trained to standard in the core skills required for a market economy. The outcome would be increased productivity and more rapid labor redeployment. The benefits of the project would be accelerated transition to the market economy, greater individual and cital earnings and increased tax revenues. 7.2 The financial and economic benefits of the project are shown in Table 7.1. Page 54 Chapter VII. Projea Benefits and Risks Table 7.1: F1nandal and Economic Benefits Financial Benefits Economic Benefits * Generation of income in two ways: 0 Comprehensive and integrated cover- age to achieve a massive and coordi- - In the short term, the project would nated attack on building the human generate revenue from repayment of resources needed for installation of the tuition fees for training programs market economy. enabling the NTF to pay back part of the loan to Government. 0 Critical support for market reforms in key sectors by providing the quality - In the medium term, increased pro- trained human resources needed for ductivity and incomes of trained indi- privatization, for financial reforms and viduals and their employers would for the reform of public administra- lead to increased tax revenues. tion. * Substantial substitution of training 0 More rapid labor redeployment within Russia for training abroad, through training for new occupations saving foreign exchange for use in and self-employment. other priority areas.* * Adoption of a strategic approach * Use of competitive tenders to ensure whereby critical issues are analyzed in effective and efficient use of project depth and strategic priorities are resources. chosen to guide investments in training. * Mobilization of substantial additional donor resources. * Immediate impact by concentrating on practitioners already employed in key * Better coordination of donor assis- occupations for the market economy, tance, avoiding overlaps, and making e.g. managers and accountants. more efficient use of resources. * Medium to longer term impact by placing the training industry on a greater market footing and by helping restructure the content and delivery of higher education in economic and financial subjects. * Another of the benefits of the project would be to demonstrae the results obtainable through a 'product-flnancing' approah. In contrast with the curnts 'input-finncing" approach (pam. 2.19), the NTF would finance the delivery of specific products, like textbools, courses, curricula, selected on a competitive basis. This would promote competition and market forces in training, would ensure the most effective use of limited resources for training, and would exemplify possmble reforms in education finaning. Chapter WI. Project Benefits and Risks Page 55 B. RISKS 7.3 Any project focusing on institution development and dealing with multiple sectors, levels and agencies is subject to substantial risk, particularly in a country undergoing massive economic and social change. The main instrument of the project, the National Training Foundation, is expressly intended to address project complexity and permit flexibility to respond to changing conditions. Other significant risks may be lack of quality control in the NTF and lack of quality content and innovation in the projects it finances. Pressures undoubtedly would mount to use personal or political, rather than technical, criteria for allocation of project funds by the intermediary. Resistance to these pressures and ultimately the quality of the NTF's work would depend-above all-on the quality and incentives of its staff. Project design includes careful, competitive selection procedures for staff and intensive in-service training programs along with competitive wages. Moreover, the NTF, through the development of its evaluation and supervision capacities, could help ensure that prior failures of training would not be repeated. Another risk concerns potential leakage and loss of funds because of inadequate financial controls. Detailed financial procedures have been designed to minimize this risk. Additionally, insufficient demand for loan funds by potential sub-borrowers from the NTF, or failure of sub-borrowers to repay loans, could undercut the reach of the project and its financial self-sufficiency in the long run. These matters have been studied in depth, but, given current rapid changes, they must be kept under constant review by the borrower and Bank staff. There is a risk of dispersion of funds across an excessive number of subjects and regions such that impact is dissipated and diffused. Careful strategic selection of priorities and phasing of activities would be required to avoid this risk. Finally, the project entails development work such as curricula and materials development and training trainers. Full realization of the benefits from these investments depends on additional, as yet unsecured, financing to support their dissemination and application. THE RUSSIAN FEDERATION MANAGEMENT AND FINANCIAL TRAINNG PRoJEcr VHI. AGREEMENTS REACHED AND RECOMmNDATIONS 8.1 During negotiations, assurances were obtained from the Govermnent and the NTF and were incorporated in the negotiated Loan and Project Agreement regarding: definition of NTF programs (para. 4.12); maintenance of structure, organization and staffing policies and appointment of senior staff of the NTF acceptable to the Bank (para. 5.13); accreditation of Regional Foundations acceptable to the Bank (para. 5.13); consultation with the Bank on the financial terms of grants and loans to subproject beneficiaries by the NTF (para. 5.28); maintenance by the NTF of intemal rules and regulations that are acceptable to the Bank (para. 5.30); subproject processing by the NTF entities in accordance with the approved Operational Manual (5.32); submission of the each NTF entity's annual budget and work program for Bank review annually (para. 5.32); prior Bank approval of subprojects over $250,000 (paras. 5.20); and audited accounts by independent auditors (para. 6.34). 8.2 The following conditons of Board presentaton were met: (i) receipt of a letter from the Government confirming the negotiated terms of the Subsidiary Loan/Grant Agreement between the Government and the NTF (para. 5.29); (ii) appointment of the Executive Director of the Central Foundation acceptable to the Bank following procedures acceptable to the Bank (para. 5.30); and (iii) adoption of an Operational Manual by the NTF acceptable to the Bank (paras. 5.14 and 5.30). 8.3 A conditon of effectiveness is signature of an acceptable Subsidiary Loan/Grant Agreement between the Government and the NTF acceptable to the Bank (para. 5.29). 8.4 A condton of disbursement for each Regional Foundation is conclusion of a subsidiary financing agreement satisfactory to the Bank between the Central Foundation and that Regional Foundation (para. 5.31). Recommendaton 8.5 The proposed operation is suitable for an IBRD Loan of US$40 million equivalent to the Government of the Federation of Russia, for 17 years including a 5 year grace period, at the Bank's standard variable interest rate. EC3HR November 23, 1994 T1E RUSSIAN FEDERATION MANAGEMENT AND FWANCiAL TRAING PROJECr ANNEX 1. ACtION PROGRAM FOR MARKEr-ORIENTED TRANING PROBLEMS ACTIONS INSMRUMENTS 1. Weak Mwngaent and No Near Term lanal Capaety * Encoure the provision of aining * Create an autonomous truining Problem: Coordination of atitdjg by the private sector, e.g., foreign organiztion and fund with inhtiatives missibg, leading t exc joint venturwa; private, fee-paying government representation (but not capacity in somo. areas and glpS in training; in-company trai'ng, and control) to attract foreign resources others. Excessive sector specilization by private professional associations for triining and invest in priority in training s iutions inew cizaai tior * Redefine central government projects mobility. Excessive direct responsibilities for training to focus * Use the taining fund to finance government control of taining on policy development and private initiatives insditutions inhibits initive an coordination * Establish local boards and control reform. Tradtional provision of * Establisb partnerships between for training institutions trining by the state inhibits financial government, enstprise, * Introduce occupation-oriented reform, relevanc. Exstng big professional associations and programs in place of sector- capacity is shriig due to lack of traimng institutions for guidance specific training finacing by government and by and initiatives eneprises. private atining * Restructure ministry-sponsored uamgulaed. ftraining by requiring institutions to be self-managed and increasingly self-financed * Devolve responsibility for management of trng to loalities and trainng institutions * Reduce sector specializaton in favor of training for generic occupations trnsferrable across sectors * Introduce cost recovety from students and/or companies Medium to Lone Term Medium to Long Term * Develop dual public/private training e As private self-financing sytem with freedom to choose institutions develop, gradually * Establish reguly and reorient the role of the fund as a supervisory authority and source of finance for training accreditation procedures for private towards encouraging special instiutions initiatives and odher selective * Divest governmtent of all direct interventions management of training institutions, * Increae administrative role of the except for skill-upgrading for fund as a faciitator of private government staff training * Build regional and municipal * Develop nanagement programs for capacity to assess training needs, heads of training institutions and supply and initiate programs local admdnistations Page 60 Annex 1. Action Programfor Market-Oriented Training PROBLEMS ACTIONS INSTRUMENTS 2. hytelrance of training to Near Term Neat su| maket re rMents * Introduce training in new E Establish working groups of Problem: System not training for disciplines impwoita for the market government, private and market economdis or jobs of the economy, e.g.. management. iternational experts to authonze fuwte. accountancy, private-sector banking new intewnectoral curicula and * Reduce sector specialization in teaching nuatrials in core subjects training * Provide technical assistance for * Ensure supplies of teaching material establishinglstrengthening by translating foreign textbooks and professional association in key adapting them to Russian conditions professions * Develop case materals based on * Finance the pVeparation of Russian Russian experiences based case materials * Adapt the pattern of in-service training programs Medium to Long Term Medium to Lore Term * Revise training programs in the * Institutionalize lnkages between formal system of education domestic and foreign institutions to * Upgade members through ensure infusion of new ideas professional associations * Implement results of above study * Decentralize control of training content to regions and institutions * Develop new services by training institutions, e.g. research and conulting Annec 1. Action Program for Alarket-Oriented Training Page 61 PROBLEMS ACTIONS INSTRUMENTS 3. Low Quality of Near Term Near Term Instruction e Establish voluntary systems for e Provide twchnical assistance for Problem: Teachers trained in accrediting institutions developing of accreditation systems command economy with old-style * Acquire effective teaching rooms e Provide fellowships and overseas teaching methods constrains rapid and aids, e.g., computers, video training for master trainers dissemination of relevant skils. players, etc. * Develop center(s) in Russia to train Private training expanding, but much * Retrain teaching staff in modern trainers of it is low quality. teaching content and methods * Use competitive ptoposals to * Establish light regulatory allocate investments to training framework for private training, to institutions through a training fund encourage tmimmum standards * Renovate and equip key training without inhibiting initiatives institutions e Provide technical assistance on alternative methods of regulating private training Medium to Lone Term Medium to Lone Term * Concentrate resources in a few key e Select and develop rgional cens institutions or centers of excellence of excellence c Introduce systematic evaluation of * Provide technical assistance of learning achievementc evaluation methods Page 62 Annex 1. Action Programfcr Market-Oriented Training PROBLEMS ACTIONS INSTRUMENTS 4. Limited Capadty to Deiver Near TermNe Te Market-Oiented Training * Develop cadre of master trainers to * Provide technical assistance for Problem: Trairing needs for the pass their expertise on to other instructor training matket economy are massive, yd trainers * Provide assistance to expand supply capacity is limited and even * Expand the capacity of distance programs employing: shrinking due to financial cotraints. learning systems - open universities Macro economic refomsa * Use self-stdy methods, such as - distance learning by hampered by skills gap. computer-based training correspondence * Encourage the provision of quality - TV/media training through private providers, * Develop computer-based training e.g., joint venture, fee-paying programs institutions * Decenb-alize authority to regions * Foster voluntary communication and local institutions networks among training institutions * Develop policies on private training and programs to assist the expansion of private training insXtitons e Provide assistance to establish communication networks Medium to Long Term Medium to Long Tern * Expand the capacity for in-company * Assist companies to develop in- training programs house training capacity * Expand publishing capacity for * Assist professional associations to booklmaterials on the market develop training programs economy , Build capacities of professional associadons to train members * Advise company and bank management on methods of enhancing training through joint ventures and twinning arrangements TaE RUSSIAN FEDERATION MANAGEMENT AND FINANCL TINIG PRoJECr ANNEx 2. MANAGEMENT DEVELoPmENT: AcorON PLAN AND PRoGRAMS A. ACTION PLAN 1. Overaa Relvawe and Orientaion PROBLEMS OBJECTIVES ACTIONS INSMRUMETS 1. Major changes in thinking and 1. Enhance relevance and make 1. Organize government-private 1. Formal strategy and policy behavior of managers are an effective contribution to the sector dialogue and conceptual stements adopted and urgently reqmured by current reform process, by reorienting work on strategy and policy published. economic reforms. and upgrading management issues, both short- and long- Management development development, both enterprise- term. 2. Strategy and policy reflected in needs to be restructured and based and external. govenment decisions and reoriented to provide 2. Develop a research agenda and legislation. knowledge and skills for 2. Define short- and longer-term program on current practices, managing under market strategies and an effective problems and priority needs of 3. National Training Foundation economy. A new strategy and policy framework for Russian management. guided by segy and policy policy framework should be restructuring management choices in selecting specific defined in order to facilitate development and focusing it on 3. Assess and reassess priority projects. reforms, encourage initiative, national, sectoral and regional needs by sectors and regions. stress priorities and avoid priorities. 4. Periodic (annual) report on waste. 4. Revise legal texts hampering state and prospects of 3. Create a dynamic and improvements in management management development in competitive market for developmnt. Russia. mangement development services. 5. Establish and operate 5. Information services on information system on maagemet training and management training and development opeated on development. commercial principles and widely used. Page 64 Annex 2. Managemem Deveopme 2. Management Development in Enterprises PROBLEMS OBJECTIVES ACTIONS INSTRUMENTS 2. Enterprises are slow in 1. Achieve major change in 1. Inform senior managers on 1. Workshops for managers in adapting management attitudes of managers to current trends in management problem/need identification. development to new needs and enterprise-based management development. conditions created by the development. 2. Pilot projects in enterprises. economic reform. Urgent 2. Train human resource and training needs of most 2. Increase the role of enterprise- personnel managers from 3. Joint programs of enterprises managers and specialists are based and action-oriented private and public sector. with Russian business schools. not met. Action is delayed due management development in to lack of perspective, enterprise restructuring and 3. Help enterprises to upgrade 4. Training of Russian managers information, role models, improving the management of their management development in foreign enterprises. experience, finance and policy privatized enterprises. approaches and programs. support from the top. As a 5. Networking of human resource result, the role and 4. Train human resource managers for experience contribution of enterprises to consultants for assisting exchange. management development is enterprises. below their real potential. 6. Brochure on best enterprise Major resources are wasted experience with management and time lost. development. 7. Participation of users in central bodies to orient and promote management development. Annex 2. Management Develoj ment Page 65 3. External Management Trining and Development Sernices PROBLEMS OBJECTIVES ACTIONS INSTRUMENTS 3. Most existing training I. Make external management I. Review and evaluate the I. Voluntary associations of institutions were created to development services relevant existing institutional base for institutions serving their serve the command economy to priority needs of economic management education and common needs and interests. system. Adjustment to market reforms. training. economy needs is difficult. 2. Training workshops for Centralization prevails, many 2. Modernize, rationalize and 2. Assist inst.tutions in devising directors, deans and other regions are poorly served. decentralize the institutional and implementing strategic and managers of institutions. Privatization is progressing, base for external management performance improvement but many obstacles have to be education, training and programs. 3. Advisory services on auditing, ove,c3me. The whole development. restructuring and developing institutional network needs to 3. Train managers of training individual institutions. become more relevant and 3. Make management institutions. efficient. development institutions 4. Information services on increasingly self-supporting, 4. Organize voluntary co- existing institutions and demand-driven and able to operation through networking. program offerings. operate on commercial principles. 5. Help to create new institutions 5. Twinning with excellent to meet the needs of poorly foreign institutions. served regions. 6. Individual co-operation arrangements within Russia with other institutions and with eI_erprises. Page 66 Anne 2. Management Deveopment 4. Leandig Resorea for Mangew t Dewdopmt PROBLEMS OBJECTIVES ACTIONS INSITUMENTS 4. A shortage of suitable 1. Provide the Russian 1. Identify priority needs, develop 1. Plans and programs for crricula and learning mangme development and a work program and produce concerted action in curricula materias is a serious problem traning system both in- needed curricula and materials. and mateials development. of both enterprise-based and company and external) with extenal training. Materials modern curricula and leaming 2. Carry out research into specific 2. Joint projects, author teams, addressing issues of transition materials, reflecting best problems and practices of consortia and other forms for are lacking. So are materials international experience and Russian management to ensure producing specific materials. for self-development and specific condidons of the adaptation of translated distance learning. Little serious countty, with special regard to materials and generate original 3. National bank and information work has been started to needs of economic reforms. Russian materials. system on learning materials. reform curricula to adapt them to specific Russian needs while 2. Achieve a major shift from 3. Design and install distance- 4. Series of textbooks, case making best use of academic to active, problem- learning and self-development studies and other materials. international experience. oriented methodology. systems. Foreign materials are used 5. Series of publications for self- without adaptation and their 4. Design and operate development and distance selecion is often arbitrary. information banks and services learning. Active and action-oriented on curricula and learning methods are rarely used. materials. 6. Distance-learning facilities as Modern teaching aids are regular service for lacking. 5. Ensure publication and wide management education and distribution of new materials to short-term retraining increase impact. AnnA 2. Management Developmwun Page 67 5. hwan Resowres for M a_Wmnt and Devdopment PROBLEMS OBJECTIVES ACTIONS INSMUMETS S. Human resourcs (teachers, I. Achieve substantial 1. Retrain existing and ta-in new 1. Workshops, courses, tainers, consulstas, anig improvements in competence, teachers/tainer using various fellowships and other managers and organizers, motivation, terns of models and program types. programs for teachers/traines human resource managers, employment and performance development. researchers), are the main of teachers/trainers and other 2. Train enterprise trainers and factor affecting quality of professionals active in training directors. 2. Handbooks and guides on how management deve!pieent, management development. to teach particular topics and both in-company and external. 3. Organize special training in on methods of instruction. With few exceptions, their 2. Improve competence, new market-related topics and competence is insufficient. intervention methods and subjects. 3. Guidelines and information Reasons are the heritage of performance of management materials for trainers to assist the past system, poor consultants. 4. Organize programs on use of their self-assessment and self- knowledge of foreign case method, action-learning, development. languages,low motivation for computer-based instruction and achieving excellence and weak other active methods. 4. Networking facilities for links with industrial practice. individuals by areas of Intake of young 5. Provide briefings on how to use common interest. teachers/trainers is new curricula and leaming insufficient. materials. 5. Special program for management consultants and 6. Train management consultants. trainers keen to improve consulting skills. Page 68 AnnFe 2. Managemm Development 6. QuaUy &andals PROBLEMS OBJECTIVES ACTIONS INSTRUMENTS 6. Quality of management 1. Achieve major improvements 1. Develop and apply quality 1. Legislation and guidelines on development is a serious in the quality of management measurement, assessment, quality standards and problem due to a number of development both in certification and improvement certification. reasons. Many suppliers and enterprises and at university- programs. users of training services are based and other management 2. Certification bodies and unaware of the problem due to education and training 2. Assist enterprises and procedures. the absence of recognized establishments. institutions in self-assessment standards and comparison of their training and 3. Peer audits and advisory bases. Some progress has been 2. Promote the use of certification educational program. services on how to assess made to assess quality and to set and achieve high program quality and make introduce certification. Actions standards and help suppliers to 3. Inform users on certification improvements. and programs to enhance meet these standards. and quality improvement quality and meet international criteria and practices. 4. Directories and information standards have hardly started. services on programs meeting 4. Harmonize work on quality quality standards. improvement with production of learning materials and 5. Workshops for human teachers/trainers development. resource managers and heads of institutions on quality assessment and improvement issues. AMa 2. Management Develapnt Page 69 7. Use of In*e_nI Expmfle_ PROBLEMS OBJECTIVES ACTIONS INSTRU IENTS 7. Access to international 1. Improve selection, transfer, 1. Choose suitable sources of 1. Workshops on technical co- management development adaptation and use of expertise, co-operating pattners operation project design, experience has been liberalized international management and transfer channels. partner selection and methods and an impressive number of development expertise. of implementation. projects aims to channel it to 2. Design and implement studies Russia. Yet the results 2. Give priority to transfer of comparing Russian and foreign 2. Information services on achieved fail to meet needs practical management know practices in selected areas of important operational and and requirements due to how before transfer of management. programmed actions of reasons such as lack of co- elementary knowledge. technical co-operation and ordination, overlapping or 3. Compile and circulate their results. poor adaptation. The results information on suitable achieved do not match the expertise available locally and 3. Twinning and partnership resources used. internationally. arrangements with selected foreign sources of expertise. 4. Organize collaboration of Russian and foreign experts for 4. Active participation in producing new learning international networks and materials. meetings where important new expertise can be acquired. 5. Mixed author teams for learning materials. Page 70 Annex 2. Management Development B. SUBPROGRAM DESCRIPTIONS 1. PolW frmework and qudaly stdards (US$1.2 million of base costs). Management development in the Russian economy requires a new policy framework encouraging initiative and change, improving the motivation for training and development, ensuring the utilization of scarce resources and increasing quality. Improvements are needed in legislation, taxation, financing of training, principles and methods of government and private sector support, trainers' terms of employment and motivation, resource allocation to priority areas and the definition and application of quality standards. The objective would be to remove major instituticnal and legal barriers hampering modernization of management development in the country, assess and improve the policy framework and set and apply high-uality standards for management training programs. The project would finance studies of demand, supply and the state of the art of management development, the development of specific proposals for necessary changes in legislation and government policies, an assessment of current quality standards and the development of new standards, the design of effective certification procedures and assistance in applying the standards that would be adopted. 2. Enerpie-based management devlopment (US$5. 7 mIion of base costs). In preparing Russian enterprises for operating and surviving under a market economy, a substantial improvement and reorientation of enterprise-based management development and training is a top priority. This training must focus on key issues of enterprise restructuring, privatization and efficiency and help enterprises to appreciate the need to change and count on their own resources above all. The subprogram would assist some 25-30 larger and medium-sized privatized enterprises in implementing action-oriented management development programs using recent market-economy experience. It would be fully decentralized to the participating regions. Local training institutions and consulting firms would be strengthened to provide support through advice and training. Internships in foreign companies would be organized for Russian manarers in collaboration with industry and trade associations. Training guides would be produced and a wide dissemination of best experiences organized in order to stimulate enterprise initiative in human resource and management development. The project would finance diagnostic and program development work, training of human resource managers and trainers from enterprises, training of management teams, management internships in foreign companies, the production and dissemination of practical guides for enterprisebased management development and workshops for sharing experience. Individual proposals would seek to incorporate alternative delivery modes, including distance education, to expand the reach of training programs. 3. LIng materials, tndne,s and teachers (US$S.1 mfilon of base costs). The development of high-quality training for management under a market economy is seriously hampered by an acute shortage of suitable learning materials and competent trainers and teachers. Initiatives already taken to overcome this shortage have been fragmented and of limited impact. A major systematic effort is required to fill the existing gaps, produce and disseminate a wide range of materials on various aspects and functions of management, retrain the existing management trainers and teachers and start preparing their succession. The objective of the subprogram would be to develop 17 sets of learning/training materials covering management topics where the needs are most acute due to the profound differences between a command and a market economy (entrepreneurship, strategic management, marketing, business finance, innovation, change management, enterprise restructuring and others), retrain 100 trainers in each subject area covered (1700 in total), and provide further training to selected trainers willing to serve as master trainers in various regions of Russia. Materials for distance learning would also be produced. This work would be based on a thorough assessment of existing materials and projects pursuing similar Annex 2. Management Development Page 71 objectives to avoid overlapping. Emphasis would be put on sharing learning materials and experience with their use. Translations of foreign materials would be combined with adaptations and the production of original Russian materials as necessary and feasible. Special training in managing training establishments and programs would be provided. 4. Management consulting (US$1.4 million of base costs). In adapting to market economy conditions and standards, Russian enterprise managers will increasingly require the support of professional services, including management and business consulting. However, the currently available Russian consulting services are short of market-economy experience and their knowledge of consulting principles and methods needs to be improved. The objective would be to strengthen and upgrade the emerging Russian consulting profession, increase the consultants' competence in substantive areas of management know-how and consulting methodology, improve the management of local consulting firms, help enterprises to build their internal consulting services and train managers in the use of consultants. The target is to train 370 consultants and managers of consulting firms. The project would finance the training of consultants and their trainers, internships in foreign consulting firms, the training of founders and managers of Russian consulting firms, the translation and production of professional consulting guides, and advisory services for structuring and developing the consulting profession. Proposals would attempt to maximize the effectiveness of this training program by using various means of delivery including distance education. S. Networking and nfonnation (US$1.4 mion of base costs). A major issue of management development in Russia is how to overcome the current fragmentation of efforts and enhance the sharing of experience and information without imposing uniformity and stifling initiative. There is an urgent need to improve the collections, evaluation, distribution and utilization of information on best Russian training programs, materials, and institutions and on the results obtained from the numerous technical assistance projects. Major benefits could be drawn from collaborative networks established as voluntary associations or using other convenient collaboration formulas, through which enterprises, institutions and individual management development professionals could exchange experience, help each other, coordinate activities, organize joint programs, acquire information and learning materials and deal with other matters of common interest. The objective would be to increase the impact and the multiplier effect of current and future management development initiatives and investments by ensuring circulation of information, sharing experience and resources and working jointly on issues that exceed the possibilities of one single organization. The project would encourage the development of voluntary professional associations, including employers' associations, pursuing speciftc objectives consistent with the objectives of their members. Rather than providing general unspecified support to associations, the project would finance specific technical services required by members. It is hoped that strengthened voluntary associations would become eligible for implementing other subprograms of the project. In information services, the project would finance the development of databases, directories and clearing houses on programs, institutions, training and learning materials, technical assistance projects and their results and other information essential to effective management development. Piot Projects 6. Study of barriers of management development. Work on a new policy framework and quality standards would start by a pilot project aimed to identify the critical barriers hampering the modernization of management development and its rapid adjustment to the requirements of the emerging market economy. This pilot project would identify the main legal, institutional, administrative, financial, Page 72 Annex 2. Management Development cultural and other barriers and formulate specific proposals for needed changes. The proposals would be reviewed in workshops with government and private sector representatives. 7. Hundred Russian cases. The pilot project "Hundred Russian Cases' has two main purposes: (i) to compile and disseminate the valuable case studies on the management of Russian enterprises under the current economic reform. which have recently been produced in various Russian institutions, often with the support of foreign aid projects; and (ii) to stimulate the production of original Russian training materials reflecting the current conditions and needs of enterprise management. The project would finance the collection, evaluation, selection, editing, reproduction and dissemination of some 100-150 case studies and train several groups of teachers/trainers in producing and using cases. Video-based cases and dissemination of cases and teacher training through distance learning will also be encouraged. It would serve as a launching and testing activity for developing Russian learning materials, trainers and teachers. The first collection of cases would constitute a nucleus of a Russian clearing house of learning materials on management and business. 8. Managementdevelopmentdatabase and diredtory. The purpose of this pilot project would be to design a data base for the Russian management development sector and compile the first directory of relevant institutions and programs. The data base would be an important element of the whole program and would include information on institutions, programs, services, learning/training resources and materials and technical assistance activities. It would be available for wide use in Russia and by foreign partners interested in supporting the sector. A directory of principal institutions and programs should be the first product of this information service. It would help to identify organizations eligible for support under this project and capable of carrying out various project activities under contracts granted by the Central Foundation and/or the Regional Foundations of the NTF. THE RUSSIAN FEDERATION MANAGEMENT AND FINANCIAL TRAMNJG PROJECT ANNE 3. AccouNTANCY, AUDIT AND FINANAL MANAGEMENT: ACTION PLN AND PROGRAM A. ACTION PLAN PROBLEMS OBJECTIVES ACTIONS . ITuMNT. 1.1 Lack of awareness on the part of Incresse awarene of de top Deign tang programs for tp Offer twelve-week tring programs top execuives in tbe enrdprise ad execuives in the enterprises and executives in the enpres and to 27,500 to 45,000 chief p minsries of the nw role of coate ministries of the accounting and finance minisbtries focusing on the role of in the first year of operton. Three accounting and finance as decision roles as decision naking tools. aecouning and finance as decision- types of programs in this categoty are maidng twls. making tools in the mket-based to be offend to: economy. - pracficing accouts - practicing auditors Develop and offer additional tramng - senior financial mamgers programs to accomodat specalized, sectorial or regional needs. 1.2 Lack of undersanding of the new Increase the understanding of the Cover the roles of account and Offer a four-week training progam role of the accountant in a market- pracitioners of their new roles in a financial mana in a market-driven for 5,000 technicians in basic driven economy. market drven economy. economy in enough depth in taining accounting and auditing skills using a programs and in the curricula. compuer based distce learning technique. The traming matrial would be based on the intenatoral accouning and auditing standards and the Revised Russian Chart of Accounts. 1.3 Insufficient exposure of leading Expose tainees to subject mats of a Inclde in the curricula and tring Develop and offer a training pogm pro&Wonls an enpis market-based econonmy. programs an appropiate potion on the to seior executives and expert in managt to subject matters related fures and functons of a market- public finances. to a market4msed ecoy. based conomy. Page 74 Am= 3. Accou mcy, Audit and Finanal Managment PROBLEMS OBJECTIVES ACTIONS INSTRUMEET 1.4 Inability of managers at various Increase trainee's abiity to use the Include quantitaive and qualitative Develop and offer a training program levels to make decisions at the decision making procss in making tools of deciion making in die training for senior executives and expets in enterprise. decisions. programs. the Treaswy. 1.5 Inability of management to think Increase tainee's ability to think Make extensive use of case studies, Develop and offer two-week iteive critically about issues. critically about issues, undertake simulation games, discussion analysis training programs to 300 top necessary analysis to define a problem, and role play. executives in the enrprises and identify alternatives, and formulate a ministries on the tole of accounting solution. and finace in implementing the market-based economy. 1.6 Inadequate numbers of specialists Train adequate nunbers of speciasts Offer adequate numbers of training to seive the new needs of a market- to serve the new needs of a market- programs in critical accoundng and based economy. based economy. finance areas that did not exst under the oentral planing economy. Adopt appropriate channels of training such as distance learnng and sdfbased, self- paced programs. X~ I iii |[s0 1:1 1. E.g.~~11 .~~a -- [0 # -gl1 lilt 11Wt1jht Page 76 Annx 3. Accowtaney, Audit and Fancial Managm t PROBLEMS OBJECTIVS ACTIONS INRUMETSIM 2.5 Insufficient trining in the practical Train taners in the prtical aS of Stu e tainr' programs to aspects of accounting and finance. accounting and fianc. emp tasize pracetical aspects of accouting and finance. Provie oqporities to those trins to have on-de-job waining for an appropriat lengL of ime in aprpat western insti"tons. 2.6 No attention to ehical issues in accounting edcaton. Incorporate ethil issues in accounting Incorporae ethical issues in accounting education education and training. Employ innovative approaches using case studies, role play and situational analysis. Incorporate ethics as a component of the required examinton for certfiaton and entrance to the accounting profession. 2.7 Deficient on-the-job tining on Incorporate taining componens on the Provide opportunities for internslips in accounting an finance. objecives of each audit step. accounting firms in Russia and in the West. Encourage the use of practice ses in accounting and auditing curricula. 2.8 Lack of exposure to western Expose traneus to western experiences Develop opporturuties for on-t-job countries' experiences in accounting in accounting and finance. taning in western counties. and fnce to fuincton as role models. Annex 3. Accoutancy, Audit and Finaciat Managem Page 77 PROBLEMS OBJECTIVES ACTIONS INSFTUE 3. TRA G C 3.1 Lack of rolevsot acounng Revise acc and fince curicula Develop new curicula to _ Generate upt-date reevan taining curricula to accommnodat the needs of to uake them relevant to the Russan the needs of a madket-based econoy. resources to acconmodate te pi a maket-based economy. mawket-based economy. Adopt a long-term concept of inome, projects and prioty pogrm cash flow and rated concepts as dhe sbedul for offering in the _a cent of accouing education ad tm includig frang. - self sandg aning modes; - local case suies; - authored and a'apted textbok; 3.2 Lack of relevant text books in Pubis new and up-to-date txtbooks Commission prominen individuals - video-audio taining aids; accountng and finance to meet the in accunting and finacee to meet the from Russia and the West to wrting - trainers' manud neods of a Russian mket-oriented needs of the Russian economy. needed texs. - work-snady and workbooks economy. - self-study muaerial - computer-bsed simuaion games. 3.3 Lack of sufficient Comniission the tanslaon/adaptation Estabish a publishing program for a translationladaption of western books of western textbooks and reference list of well-chosen western texxooks and training materials in accounting materials in accounting and finance. and traiDing materias to ensure the and finance. availability of sufficien translao andtor adapttion of traioing resources in accounting and finance. Page 78 Amum 3. Accowntwny, Audit and Finacual Managmen PROBLEMS OBJECTIVF ACTIONS INSIIuMENTS 3.4 Shortage of _ppr%iate foreign Acquire appropriae foeigp txtbooks Colaborat with sedleed wesetrn See above. tetbooks and reference maerials for an re e materials in accntig universiies to develop a lst of research and eaching. and finance. appropiat refe e book, pridics, data boos an other resources tht are relevant to the current economic condition in Russia. 3.5 Nonexistence of locally deveoped Create locally developed case studies Deveiop poSam for writing case case studies reflecting major issues rflectg major ise ad chalenges stdies on issues 6cing the economy facing the Russian economy. facing the Russian economty. through the colwabo n of pact t academicias and western 3.6 Inappropriate and out-daed Develop and aque up-odte Set up a program to acquire, develop teaching aids in accouning and teaching aids. Materials include study and adapt sludy guides, intructors finane for trainers and tranees. gudes, insiructor's nuas, manuals, computer pro"ets and self workbooks, computer projects, study modules. smulation games and self-paced, self- study manuals. 3.7 Lack of a distibution system to Establs a ditbution system to Establish a dstibution system to disseminate informaton to fculty, disseminae informain in a timey disseminate iformation in a timely trainers and students in a timely maner. manner. manner. Anex 3. Acountny, Audit and Financsd Manageme Page 79 PROBLEMS OBJECTIVES ACIIONS INSMTUMEIES 3.8 Lack of use of computers and Establish support systm for the Develop programs to assess the See above. inron tchnology as: desig, devlopment and offeing of integration of computer and -ngated systems the cucula ta g mals and programs. ination technology in taining. * teaching and learning aids by Help trainers to equip tainees with saukb and ficuly; computr skils. - practal toos to autmate accounig functiot to relieve the massive ned of a cadre of accounting and finance para-rofessoals andlor tehnicians, - vehicle of mass communication in deivesing training progams via disLnce leang tools and technques. Page 80 Annex 3. Accountancy, Audt and Financial Management PROBLEMS OBJECTIVES ACTIONS INSTRUMENTS 4. TRAINING ENVIRONMENT 4.1 Lack of coordinad efforts in Coordinae the current and Take inventory of current and Use computer-based programs and strucuring a comprehensive system for contwlepd diverse training efforts by contemqlated training activities by distance learning to disseminat taining, structuing a comprehensive system for different instiutions. Solcit the accounting and finance techniques to a trang pacitoners. cooperaion of hose institutions to u.Sz of technicians. develop a coordied action plan. 4.2 Emphasis on technical sectoral De-enihai teaclhng and tnimng in Shift tning and teaching from a Form a Technal Advisory Commitee accounting procedures rather than sectoral accounting and emphasize sectoral-bad to a discipline-based. to the Natonal Training Fouaon i nterdisilry and concepual teaching and trainig in the different (NTF). training. specialized disciplines such as - Membership to represent budgeting, financial an management professional orgaizons academc accounting, and corporate finance. institutions, entpises, government and die NTF; - Safegard to prevent conflict of inrest between those who approve a project and those who implement it. 4.3 Predominate use of traditional Abandon die traftional tanumng Develop new training approaches and The Commission: approaches in teaching and taring in approach where the trainee/student is a training resources. Train qualified - Recommends criteria for proposal content, deivery and objective. passive party in the learning process trainers on these new approaches. evaluation and participates in and turn to interactive modes where the screening proposals submitted for trainee is an active participant in the funding to the NTF; process. - Monitors the activities of other donors that may affect project's scope 4.4 Insufficient understanding of the Increase understanding of the role of Train top executives in the enterprises and functions; role of accounting and finance as accounting for efficient allocation of and the ministies on the roles of - Observes and coordinates activities means for efficient allocation of scarce scarce ecoomic esources. accounting and finance. of the eisting and emerging academic economic resources in a market-based and professional organizations in the economy. fields of accounting andfinance which may impact Annex 3. Accounay, Audit and Financial Management Page 81 PROBLEMS OBJECTIVS ACTIONS INSTRUMENTS 4.5 Inefficient use of available finds Use available resources efficienty and Develop a flexible trainig strategy it Training to ensure that a proposed from different donors, us scattering coordinae training efforts. concert with the current stage of project or one of its elements is not taining efforts and forced usable development in the accounting being performed by other entties trinng aproaches, someimes professions dtat is capable of changing within the accouiting and finance conflicting. with the environment. Set training fields; priorities in light of existing - Ensures that p , in terms consraints. of environent and/or irastructure, dth are necessary for successfil 4.6 Lack of accounting and auditing Establis accounting and auditing Help accelerate the development of the utilization of the output of a proposed standards. standards to guide training. In the accounting and auditing standards. In project exist and that the results wil interim, use interational accountig the teim, tansate the intenaional have a positve and needed nmpact; an auditing standards. accounting and audting standards and - Initates and/or n _mages, subject to use them as guidelines for training and the approval of the NTF, activities to teaching. implement the program of action specified in this proposal. 4.7 Lack of admissioo standards to the Establish admission standards to the Establish admission standards for the accounting professions. profession including entance profession (exams, experiece and examtions, experience and educational requrement.) educational requrementa, ethics and licensing. 4.8 Lack of sufficient analysis of Analyze ainig needs to accommodate Analyze the training needs for each training needs in the different regions differences in the educational group of tranees of different of Russia. backgrounds and the diverse needs of backgrounds. different regions. Page 82 Awn 3. Accomag, Auda and Funancial Managemet PROBLEMS OBJECTIVE ACTIONS INSTRUMENTS 4.9 Acute shortage of spciist in Develop pecalsts in budgeting, Develop tng programs and u aggmaa_ wwacung, finwana and managemat accounttg, mmaseri for vanous leves of financial ma_ment, auditing and auditing ond fimme. qeialists in budgeting, finnia and pubHc finance. managea accounting, and public finance. 4.10 Lack of crteria specifying Establish citeria pcifyin quali Set up enforceable quality stndards for standards for contes of training standards for contents of trang contets of anig programs, aines' programs, qualcatio of triners, proms, taines' quaifications, quaficas, taing rsources and minimum resources ad issuance of training resources and issuance of issuac Of licnses. aditing lHenses. auditing licenses. 4.11 Absence of special systems for Develop specialists for the Treasuy to Develop a system for training trainng speciasts for tde Treasuy to cope with dh new envoent in the specas for dle Treasury. cope with dte new role of the Minstry Minsy. of Finance. Annex 3. Accouorwwy. Audi and Finacial Management Page 83 B. SUBPROGRM AND LoT PRoJcr DFscRimoNs 1. The principal emphasis in the proposed subprograms and pilot project for phase one will be on producing locally developed training materials; adapting software programs for developing computer-based self-training courses; establishing information technology training centers in existing institutions; and training managers. In addition, training programs in accounting and finance will be mounted for senior executives of enterprises. The program will also design and develop new curricula in accounting and finance at the university level, and develop teaching faculty within these fields. Implementation of a the proposed program is expected to have an immediate and long-term impact. 2. The accountancy and finance program will be initiated bv one integrative pilot project. The pilot will build training capacity and training infrastructure through (i) developing 20 local case studies, (ii) establishing one information technology training center, and (iii) training 45 trainers and master trainers in the areas of accounting, auditing and finance. Development of Rdevwa Trhuin g Resources incLudJng the PMot ProJect Component on Case Studies (US$2.8 million of base costs) 3. The existing educational environment in the acceunting field is almost void of relevant training resources. The required reform in the content of the curricula and training necessitates careful incorporation of computer-based training, the use of locally developed case studies, the appropriate selection of adapted textbooks, and the introduction of innovative and up-to-date delivery methods and resources to meet the current and emerging needs of the new environment. This subprogram calls for the, development, testing, and publishing of training materials, including 100 case studies, four advance text books with teaching aids, and four computer-based self-training courses that are relevant to the Russian economy which focuses on problem solving and developing decision-making skills. The objective of the pilot project component integrated in this subprogram is to develop 20 case studies in the St. Petersburg region. Budig Tnn t) bwtruu n8u&Wg a Component of the P Project of Builng One 4fomaon Teclnolog Center (US$2.8 millon of base costs) 4. The reform of accountancy and financial training requires the use of more effective information technologies. Under the project, eight information technology centers would be established and equipped. The purpose of the centers would be to: a. support the automation of enterprise accounting systems by training practitioners in the use of information technology; b. pilot the application of new interactive teaching environment for greater effectiveness in learning accountancy and finance concepts; such new training tools would include computer-based self-teaching programs; business simulations and accessing global networks and data bases; and c. extend training to larger numbers through mass training. As a distance learning facility, each center would be set-up as a receiving and a broadcasting station ready for satellite hook-up with Western institutions and with in-house training programs in the Russian enterprises within the regions, establishing a network of Page 84 Annex 3. Accownancy, Audit and Financial Mawagem training facilities capable of collaborating, coordinating, and expanding training activities, achieving a multiplier effect. 5. This subprogram on building training infrastructure will upgrade selected existing training facilities and establish eight information technology centers capable of delivering up-to-date computer- based training in the three regions, Moscow, St. Petersburg, and Nizhny Novgorod. Each of the eight centers will be equipped with forty net-worked, high-capacity work stations each with a CD-ROM and a modem/fax; two laser jet printers; overhead and slide projectors, a multi-media project, a plotter, a scanner, two monitors, 2 VCRs; a video camera, a screen, white boards, a photocopying machine, a software library, furniture, and a security system. Each center will be designed as a receiving and transmuting stations and ready for satellite hook-up for distance learning in the second phase of the project. Each center will be headed by a qualified computer technician assisted by support staff. A component of the Pilot project is to establish a leading information technology training center in the Moscow region. Develop ag and Trainng Human Reswurces (Pra oners and Senior Executives) induig a Component of the Plot Project on Trainng Trainers (US$6.3 million of base costs) 6. This subprogram on developing human resources includes two main components: (i) training practitioners and (ii) training senior executives. 7. Tranin prac*toner. The component on training practitioners will offer selected practitioners three-months o, class room training in Russia. The training program wIll be composed of twelve modules in two main segments, a core segment that is applicable to the three groups of practitioners, i.e., accountants, auditors, and financial managers, followed by a specialized segment for each of the three groups. The newly developed training infrastructure of eight information technology training centers will train about 7680 practitioners including 765 new trainers and master trainers in the two-year life cycle of the project. The pilot project includes a component of this subprogram, i.e., developing trainers. The pilot project will train 45 carefully selected practitioners, i.e, accountants, auditors, and financial managers, over a 16-week period. Those selected for the pilot project will be screened for technical ability and potential for becoming trainers upon completing the training program. The pilot project would be assisted by an equivalent number of carefully selected Russian professions and/or academicians. Upon the completion of the training program, the top 20% would be sent abroad for intensive, 12-week. on-the-job training in western firms. The objective of this internship is to develop the newly developed trainers into master trainers. This process of training trainers will be repeated once during the first year of operation, as part of this subprogram on training practitioners. 8. Accoung and iancefor senior execives. Russian economic restructuring will be thwarted if the top officials of the enterprises and government ministries do not understand or appreciate the role of accounting and financial management in developing a viable economy. Without the support of this group, appropriate and effective policy changes in accounting and financial management will not be possible, and the technical skills learned by accountants and financial managers will be ineffective. The objectives of this component of the subprogram are to (i) provide senior executives with a better understanding and appreciation of the use of accounting information as an effective decision making tool at all managerial levels and (ii) support the establishment and the functioning of effective accounting and financial systems in the enterprises. These objectives will be accomplished through offering a variety of short-term training programs and conferences to accommodate the time pressure of these executives. Annea 3. Accoumancy, Audit and Financial Management Page 85 DevelPi Accowdlng, Audding and Finane Faculty (US1.3 million of base costs) 9. The objective of this subprogram is to develop existing accounting, auditing, and finance faculty with the necessary skills and tools to build strong academic institutions capable of producing graduates that function effectively in a complex environment. Upon completion of this training program, these faculty members will become catalysts in initiating and implementing changes in curricula, in adopting new delivery methods, and in introducing new approaches in accounting and finance education. To ensre the achievement of this objective, ! minimum of 50% of the faculty of any selected institution will go through the training program. This training program will be designed through a meticulous planning and implementation process. Alternative faculty training modes, including sandwich courses and distance learning will be encouraged. For each faculty member, there will be a statement of measurable objectives that is attainable at the conclusion of the training program. RevIon of Accoting and Finance Cunicda (US$1.4 miion of base coss) 10. This subprogram will design and develop an up-to-date relevant national accounting and finance curricula at the university level. The purpose of the new curricula is to provide professional finance professions in the twenty-first century and who will be working in public accounting, private businesses, or government. The outcomes of this subprogram would specify the scope, content, and the structure of future education in accounting, auditing, and finance in Russian universities and institutes a basis for accreditation of accounting and finance academic programs. The national curricula will specify standards, objectives, and requirements for content and approach of curricula including a syllabi of required core courses, specialized courses, business courses, and general education courses. Standards for qualifications and activities, admission and retention for students, budget requirements, library and computer facilities, and administrative structure of the accounting and finance programs will also be specified. The national curricula will be available for adoption and adaptation by selected universities in the second phase of the project. THE RUSLAN FEDERATION MANAGEMENT AND FINANCIAL TRAINING PROJECT ANNEX 4. BANNG TRAINIG: ACTION PLAN AND PROGRAMS A. ACnON PLAN 1. Sectordl Consftints PROBLEMS OBJECTIE ACTIONS INSIRUMENTS 1. While liberalization of Russia's 1. Develop a comprchensive strategy 1. Tighten prudential banking regula- 1. Train a core group of Central financial system has proceeded for the fnancial sector to push tions. Improve the capacity of the Bank supervisors to improve rapidly, the appropriate regulatory forward Russia's trarsition to a Central Bank for conducting both regulation and supervision of and supervisory frameworks are market economy. off-site surveillance and on-site banks. Develop appropriate still at an embryonic stage. examination of banks. supervisory material and 2. Undertake reforms to enable the procedures. 2. Bank acconing and auditing banking sector in Russia to 2. Support the introduction of new standards are lacking which become more efficient in accounting principles for banks. 2. Retrain and upgrade existing measure the condilion of mobilizing financial savings, accountants and accounting ffrns enterprises and banks allocating resources, and 3. Increase understanding of basic in banking issues, based on the transpareotly and consistenty. processing payments. concepts such as credit risk, establishment of a chart of collateral, liquidity, financial accouns in general consistency 3. One of banking's chief roles, the discipline and bankruptcy to with international accounting and processing of paywents. is severely facilitate the establishment of an auditing standards. Train bank bhanpered by the lack of a appropriate legal framework. managers how to use bank and developed or efficient paymeots enterprise accounting informaton system. as a decision tool. 4. A flal legal framework is not in 3. Promote passage of new place to protect private property legislation and continued nghts and the effcient transfer of legislative reform covering areas such rights; to support debt including insolvency, secwurity, recovery, liquidation and bank- property ownership, company law, ruptcy; and to enforce the law. and banking law. -~ . I I ..I Page 88 Anme 4. Bankang Traning 2. Insuffiient S&Wply of Tained Rank Staff PROBLEMS OBJECTIVES ACTIONS INSTRUMENTS 1. Preparing Russian banks to 1. Strengthen the banking profession 1. Develop programs in both core 1. Organize and deliver seminars for operate in a market enviromnent in Russia by focusing on skills and in more sophisticated senior management to develop requires new methods for managerial and technical skill banking areas. While a large strategic, tactical, financial managing banks, designing, development for senior and percentage of banking personnel is planning and risk management marketing and delivermg banking middle-management. Increase almost totally unprepared to carry skills to orient or reorient the way services, controlling risk, ensuring student throughput to better out the banking activities the new banks are managed. banks' financial viability and respond to industry demand. roles require, the pace of change is performance, and carrying out fast, and certain parts of the 2. Organize and deliver seminars for basic banking operations. With 2. Link development of a banking market are increasing in middle management and lending over 1900 banks in operation, training strategy to current and sophistication rapidly. officers on credit, since this is at banks are finding it extremely planned institutional/structural the core of banking and is the difficult to recruit staff with development work. 2. Forge a firmer link between banks area of highest risk. suitable qualifications and and provders of training in experience. 3. Introduce financial sector banking; as ultimate users, banks 3. Extend the curriculum to include qualifications as part of a career need to take a more pro-active topics such as international 2. Most Russian banks have been development process for banking role in specifying to educational banking. trade finance, marketing. unable or unwilling to comnmit employees. This would involve institutons the type of training capital markets and retail resources to develop an in-house coordinadon between associations, they require. banking. training capacity, all but the training institutes, and universities. strongest tier without the critical 3. Encourage bank-based trahiing, mass and/or the know-how to with bankers' associations playing devote to the task. a lead in this effort. 4. Create training which is interactive and which provides for flexibility, a requirement when undertaking training within a famework undergoing radical transformation. 5. Create programs to twin Russian banks with leading foreign banks where staif from foreign banks provide ad' ice on strategic planing, organization, daily management and operations through on-the-job advice and _ _ _ _.__ _ __ traini._ g. _ _ _ __ __ Anna 4 Banking Training Page 89 3. Inadequate Supply of Appropriate Trainers/Effectdve Tranl g Insdiudons PROBLEMS OBJECrIV ACTIONS INSTRUMENTS 1. Russia's current education and 1. Improve quality of basic financial 1. Enhance the qualifications and 1. Organize and deliver training of training network is inadequate to education at the university and capabilities of a core of master trainers programs for master supply aned staff to the rapidly institute level and banker training tawiners by exposing them to trainers, and on-going programs emerging fiancial sector. at the professional level. Westemn practices. This training for subsequent waves of trainers. Although more than 200 institutes core must include a combination Coordinate all training of trainers of higher education teach modules 2. Involve bankers more in the of professional educators for skills offerings directly wth professional on banking and finance, in most training process to improve upgrading and practicing bankers training covering the same topics cases the subject is dealt with in a relevance and usefiltness of the who can participate in and and related materials cursory fashion. educational product. influence training at their development. The training of respective institutions. trainers must focus primarily on 2. The majority of trainers teaching 3. Encourage consutative content and secondarily on banking have had no practical arrangements between trainers at 2. Revamp university curriculum to teaching methodology. experience in the field, educational institutions and banks. include courses of study relevant compounding the difficulty of to basic financial education for a 2. To provide a cost effective and teaching a new subject even if re- market economy. broad venue for making training training is undertaken. available to all banks in Russia, a 3. Increase the geographical spread network of banking institutes 3. While several new institutons of available training locations to should be developed. Wbile a have been created with the key regional centers throughout number of initiatives have already assistance of donor aid (IF&BS, Russia. been made, this effort must be Moscow; EBI, St. Petersburg), rationalized and extended through their throughput is limited. Other the nation. Ideally, bankers' fonrs of training for Russian associations would play a pivotal bankers assisted by the West have part in their design and been through short courses that implementation to ensure offer exposure to a different relevance. banking cuhure rather than intensive traiing. 4. There is a high degree of atrition of trainers imo the fmancial work place where compensation is at a significandy higher level than in academia. Also, participating in the industry boom is. in and of itself, a strong enticement to make the shift. Page 90 Annex 4. Banking Training 4. Lack of Sufficient Relvant Training Materials PROBLEMS OBJEC_TVES ACTIONS INSTRUMENTS 1. The curricutum for the teaching of 1. Produce a capability to develop 1. Create a clearinghouse of 1. Initially anslate relevant finance and banking which is indigenous training materials, information on financial training Western training materials. Using based largely on command including case studies, exercises, materials. Contact all active data from Russian banks and economy disciplines is widely ineractive computer programs, donors to identify available enterprises, create case studies regarded as out-of-date and videos and textbooks. resources for inclusion. and examples to bring added fimdamentally irrelevant. relevance to program materials. 2. Leverage Western and 'Russified Involve banks directly to the 2. While isolated training programs fnancial training materials already greatest extent possible. from the West have generated availble in Russia. some effective banking training 2. Develop multimedia training material. in general there is resources (tapes, video, television inadequate supply and distribution programs) on various banking and of materials covering the activities finance topics for use in tramiing of market-based banks, activities and through distance particularly in the Russian education. language and tailored to Russian realities. 3. The magnitude, character, and speed of chunges in the commercial banking system is unprecedented, making it difficult to *Russify material and have it maintain usefuhless. Ann" 4. Bankng Training Page 91 B. SUBPROGRAM AND PIR= PROJECr DESCRIMONs Banking Operations (US$317 milon of base costs) I. Traii of chief executves in bank management. This subprogram, as well as the following subprograms, includes components for training of practitioners, training of trainers and materials development. Each subprogram would be vertically integrated in this manner in order for the parts to form a cohesive whole. Oriented to senior executives, the objective of the subprogram is to introduce banking as practiced in developed narket economics, with a focus on areas for managerial decisions. Topics would include planning (strategic, tactical and financial), risk management, financial management (capital planning, asset/liability management, information systems), human resource management (optimizing staff mix, recruitment, work culture, compensation), and compliance and control systems (auditing, internal controls, asset quality reviews). Focus would be on the basic procedures, practices and functions essential to sound, efficient and competitive banking. Over the three-year project period, the subprogram contemplates training 2800 senior executives and 850 trainers. 2. Trahing of msnagers and transadors in credit manaegement. The initial thrust of this subprogram would be on credit risk management, as lending is a core bank activity representing a key area of risk. The objective of the subprogram is to articulate the key elements of effective credit management and provide useful information and methodologies for managing credit risk. Topics would include credit policies, directives and procedures; credit origination; borrower analysis, including risk rating of loans and repayment source analysis; loan structuring; credit approval; loan documentation; collateral and guarantees; loan supervision; problem loan management; and reserving against potential loan losses. Organizing the credit function including staffing the credit policy, lending, loan documentation, and loan review areas would also be included. The subprogram must be focused on the short-term lending characteristics of the Russian economy, and the operation of banks in the absence of reliable financial information and a legal framework supporting loan recovery. Over the three-year project period, the subprogram contemplates training 4500 mid-level managers and credit officers and 1300 trainers. 3. Trainn of managers and transactors In ternational banking. The principal objective of this subprogram is to strengthen the capabilities of banks' international departments to respond to their clients' requests for transactions involving the international sector. Specifically, this subprogram would focus on trade finance (the mechanics of trade, collections, letters of credit and bankers' acceptances; interbank funding and syndicated loan facilities), foreign exchange (spot and forward exchange markets, price determination, hedging techniques), and international correspondent banking (developing correspondent relationships, products and services of correspondent banks, credit and informational requirements of foreign correspondents). If the lending profile of banks significantly lengthens and Russian banks begin participating in more term financing, training could extend to include international project finance (structuring financial packages, identifying and mitigating project risk, countertrade, government and private insurers). Over the three-year project period, the subprogram contmplates training 2800 managers and transactors and 1400 trainers. 4. Train of managers and transactors investment banking. Private-sector Russian banks have adopted the "universal" banking structure found in most western countries. The inclusion of an investment banking department and investment banking activities within the overall scope of the bank's services is considered a vital element of the banking industry's ability to adequately meet the financial Page 92 Annex 4. Banking Training needs of the enterprise sector. The current lack of both managerial and technical competence in this highly specialized area is widely viewed by the banking community as a matter of considerable concern. This subprogram would focus on the organization of an investment department, the securities underwriting finction, financial analysis techniques, securities trading and market making, investment risk determination and management, and internal control. Over the three-year period, the subprogram contemplates training 4,000 managers and transactors and 1,200 trainers. TiWing In bank accowing (US$3.7 mloen in base costs) S. This subprogram would have two major thrusts: training accountants in the mechanics of bank accounting, and training of bank management in the use of accounting information as a decision tool. Main weaknesses in the current Russian accounting system for banks relate to the fact that the current statements do not present a fair appreciation of the return to shareholders, do not ensure depositors and other third parties of the soundness of banks, and do not demonstrate that their capital base has been maintained and that they satisfy prudential criteria. With World Bank and European Union support, the introduction of new accounting principles for banks is being carried out in a staged process, with a new chart of accounts and detailed instruction manuals to be introduced not later than 1994/95. Over the subsequent two to three years, a detailed reform of the accounting system would be carried out, pertaining to financial accounting, regulatory accounting, and statistical reporting. Management accounting would also be introduced, once these transformations have been effected. It is envisaged that the bank accounting subprogram would adopt the materials and other resources currently being developed, and those already developed within the accounting and auditing component of the Bank's Financial Institutions Development Project being implemented in Russia at present. Over the three-year project period, the subprogram contemplates trainiing 2500 accountants, 1500 bank managers and 1600 trainers. PAo Prject 6. Trvlg of chief executves in bank manageen. The pilot project covers the early commencement of three levels of activity related to the subprogram described in point 1: the direct training of senior executives of banks, the training of trainers, and the development of required training materials. This particular project was selected due to the desire to initiate training using a 'top down' approach; achieving senior management's participation in and endorsement of a program of training, followed by more in-depth staff skill-based training. Refer to point 1 for a detailed description of the subprogram's coverage. Ths RUSIA FVD3AION MANAGENi FAND MNL TRAWG Paojucr ANNEX S. Pwuuc FeUNCI ACnON PLA AND PROGRAM A. ACrIaN PLAN ft.lenu Objective Acdoas Inat.umets Budgutad Tai r Pdiq 1. lhe demise of ental planning 1. In the near tem, increase the 1. Identify the primary modular 1. Cooperat with EDI, tie IMF, placed new responsibility on the undertanding of pariamentaria courses needed (e.g., accountng, the OECD and Fiace Ministries MOF, line ministies and the Stae and senior and mid-level officis financa management, in OECD counies in the Tax Senrice to plan and evaluate of the impact of tax and budgetay macroeconomics, budget and deveopmnt and delivery of the budget and design and rase policy on mareconomic budget process, tax policy and training courses, both for revenues. In addition, in 1992 stabili2ation and the production and administion) and the target practitioners and master trainers. Rusa rapidly decentalized many allocotin of goods by the private audience (approximatly 4,00) to exnditure aigments and and public sects. receive a given course or 2. Establish traiing of traines revenue responsibilities to combination of courses. (Note prog_m at leadig financial and sub-federal levels of government. 2. Provide specific skils needed to some would aso receive the sildl upgrading institutes in Officials at the federal and egional implement new tax and bWdget modular course in budget execution Moscow, St. Pewesburg and levels are ill-preped to assume reforms. desiged for Trasaiy officials.) Nizhuy Novgorod. ese new tas. 2. In the medium term, develop 2. Upgrade the capacity of existing 2. Without adequate trig, capacity in universities and institutes to delver the modular there is a danger of resowee teial sools to delver core courses by imprvinghe skills of misallato and under-funding of subjct in public finane in a naster traines. cdrl social and ivestment mnketx- d economy, to ensure a pogrm if public finances are not flow of civil sornts for future -ed carely dwring te recuinm trmianion to a milstet4bued, Page 94 A4n 5. PmNb Finanoe Problems Objectives Actions Indr _ Sarbproga:.- Tminng of 7Naswy Qfcas 1. The former system of fiscal 1. Reatign govermment financial 1. Establish a Working Group 1. Under the PPF, prepare the mnanagement, in which Gosplan operations to reflect the needs of a with MOF, IMF, EBRD and Bank training strategy and begin training allocated the budget and the market economy, adapted to participation to agree on the of up to 100 top MOF and oblast Central Bank maintained Russian conditions. substantive parameters of required officials and master trainers from expenditure and revenue accounts, training both for managers and Moscow, St. Petersburg and collapsed in 1992. 2. Contribute to formation of a staff. Nizhny Novgorod. competent managerial class to 2. With IMF assistance, a new execute the federal budget 2. With the participation of 2. Employ twinning arrangements Treasury has been designed within effectively and efficiently. distance tearning experts, design a with Finance Ministries in OECD the Ministry of Finance (MOF) to training strategy appropriate to countries to provide senior manage the financial operations of 3. Train staff in operation of the managers (in budget execution plus officials and master trainers with the federal system. computer-based Treasury system appropriate modular courses experience in practical Treasury (EBRD to partially finance mentioned above) and to staff (in operations. 3. Senior Treasury officials in hardware and software computerized applications.) Moscow and in Finance development.) 3. Use master trainers and Departments of all 89 oblasts 3. Mobilize donor support for dictance leamning to begin require training in budget overseas training and for foreign disseminating skills to 1500 execution. In addition, at least specialists to provide initial training managers and up to 20,000 staff in 20,000 Treasuty staff must be h Russia. four years. trained in the operations of the IMF designed computerized system. Annex 5. Public Pinace .age 95 B. SuBPROGRAM AND PILOT PRojECT DESCRIONS Subprogow BSdguS and Tax Polcy (USS1.6 illon of awe costr) 1. More reliance must be placed on fiscal management in Russia as direct economic control mechanisms weaken. Under central planning, the budget merely provided financing to carry out the plan, while revenues were mobilized by redistributing resources generated by the enterprise sector. The system focused on the correctness of fund appropriations, not on the efficiency of public expenditures. With the transition to a market economy, the budget will become the most important instrument to mobilize public resources and allocate them among competing uses according to national priorities. Sound public finance means confining (or extending) public expenditure to those areas in which the public sector can act efficiently; it also means raising the necessary revenues in ways that distort prices as little as possible. Decisions on these issues critically affect all areas of economic activity. To carry out the transition effectively, improvements will be needed in all phases of the budget process-preparation, execution, and evaluation and audit. Many important reforms were recently initiated, for full implementation in 1995, including the introduction of a new budget classification system based on IMF standards. Tbis will entail new expertise in budget preparation in the MOF, line ministries and parliament. In addition, considerable effort has been made to reform tax policy and administration. Legislation passed in 1992 introduced a value-added tax, excises and taxes on enterprise profits and personal income. However, the new taxes were introduced without affording the State Tax Service sufficient time to prepare appropriate systems, accounting procedures, training or information programs. In addition, with the inception of a new fiscal federalist system in 1994, regional governments will have greater responsibility and opportunities to make fundi'ng decisions and levy new taxes. These could have adverse effects on enterprise activity if not developed and executed properly. 2. The objective of this subprogram will be to provide public finance officials at the federal and regional levels with the specific skills needed .br these new tasks. Two primary fields are contemplated. The one on budget and budget process would include modular courses on: (i) budget structure and interaction of federal and regional budgets; (ii) budget process (classification and preparation, execution, evaluation and audit); (iii) budgetary expenditures foi social programs; (iv) revenue sources; and (v) for selected staff, computer applications. The contents of the second field on taation policy and tax system would include modular courses on: (i) basics of taxation; (ii) taxation policies and systems in Russia and abroad; and (iii) operations of the State Tax Service in tax collection. In addition, a core group of senior executives, parliamentary officials, and master trainers would receive modular courses in macroeconomic policy analysis, accounting and financial management to put the more specific concepts above into a conceptual framework. 3. Master trainers will also participate in the subprogram. This will begin the process of developing capacity in universities and technical schools to deliver core subjects in public finance over the longer term. The second phase of the Training Project could support the further development of such capacity, to ensure a flow of civil servants for future recruitment. Supngrw afl of laswy Offidals (US$3.0 ndilon of base oo&) 4. The former system of fiscal management, in which Gosplan allocated the budget and the Central Bank maintained expenditure and revenue accounts, collapsed in 1992. The Ministry of Finance created a Treasury Department to assume these responsibilities that is modelled on federal systems in market economies, but adapted to Russian conditions. It will address major weaknesses in the present Page 96 Anna 5. Public Finne system of government financial management. With IMF assistance, a computerized system has been developed to help the Treasury manage the financial operations of the Federal Government. The system will permit the MOP to minimize disruptions in government programs caused by fluctuations in revenues, and to define priorities more appropriately in the event of expenditure cuts. The successful work of the Treasury depends on the generation of about 20,000 qualified personnel in Moscow and every oblast. This subprogram was selected in view of the very high priority attached by the Government to development of an effective Treasury system. 5. The objective of the subprogram is to train up to 1500 senior executives and professionals of the central apparatus of the Federal Treasury, trainers in academic institutes, and chief executives and experts of the regional treasuries in budget execution, i.e. in exercising control over timely receipt of revenue and the efficient and rational use of budget-allocations. In addition, preparatory activities would be undertaken for up to 20,000 staff to receive instruction in application of the computerized system. After designing and developing course requirements and a training strategy in collaboration with IMF and EBRD experts, training would begin in Moscow with the intensive preparation of a core group of up to 100 federal and regional managers and master trainers. These would then receive practical experience in treasury operations under twinning arrangements with finance ministries in OECD countries. Additional training of trainers would continue along with development of distance learning techniques to enable the completion of the project within two phases over four years. The training would take plp ne in a pre-identified Russian institution with the participation of local and foreign experts and instructors. Master trainers would trained along with executives. The training subprogram would be designed and developed with the participation of the MOF, IMF, EBRD and World Bank, coordinated by the Institute of Skill Upgrading of the Ministry of Finance. The contents would include: (i) the organization and functions of the Treasury; (ii) the execution of the federal and regional budgets (revenues and expenditures); (iii) relationship of the Treasury to the Central Bank of Russia and State Tax Service; (iv) for selected senior participants, issues related to a market-based economy, including modular courses prepared for the public finance program (tax and budget policy, accounting, macroeconomic, capital markets); and (v) for staff and selected professionals, accounting and operation of the computer-based system. THE RUSSIAN FEDERATiO:., MANAGEMENT AND EINANCIAL TRANG PROJECT ANNEX 6. ORGANIATION, STAFFING AND PROCEDURES OF THE NATIONAL TRAING FOUNDATION A. INmRODUCTION 1. The main objective of establishing the National Training Foundation (NTF) is to support the restructuring of the Russian economy from a planned to a market-oriented system. It aims to do this exclusively by financing training activities undertaken elsewhere; it would not provide training itself. It would be a new form of institution for Russia which combines features of western private foundations, training funds and government-sponsored institutions such as the U.S. Science Foundation. It is intended to be permanent with a mix of government and non-government Russian shareholders, increasing participation of the non-government sector, and it aims to generate sufficient revenues from its cost recovery to cover its operating costs. It is designed to operate as a network in which all accredited participating foundations work on a not-for-profit basis with common, sound policies and practices concerning matters such as finance, project analysis, project approval control and auditing. At the same time, the individual foundations would work independently and be autonomous in decision making. Initially the NTF network would consist of a Central Foundation (CF) and three Regional Foundations (RFs) in Moscow, Nizhny Novgorod and St. Petersburg. The benefits generated by the NTF would be for the economy as a whole as well as for the generation of direct and indirect financial income. Efforts have been made to keep the NTF operations simple, but some complexity is inevitable in processing and analyzing applications and financial operations. In addition, this type of financing activity would be new in Russia. A publicity campaign and training efforts would therefore be necessary to disseminate information about the NTF and its operational procedures to its potential members, staff and beneficiaries. B. MEMERsI AND FNDING 2. In order to achieve a high degree of participation and independence, the composition of members include a mix of government and non-government members. Each participating member entity would contribute a subscription fee (to be determined by the Founders). This amount would be large enough to require serious commitment from all shareholders, but small enough to be affordable for organizaions that are financially constrained. Annual fees would also be assessed from members. In addition, government members would be asked to contribute in-kind (e.g. the physical space for the Foundations at the regional levels). A non-voting Consultative Council formed by non-Russian organizations would be created, in whicb participants would pay an entry fee of US$25,000 and annual membership fees to be established. Moreover, the Board of Dikectors, the Executive Office and the members of the Consultative Council would be actively engaged in fund-raising. The funds raised under such mechanisms would be used for increasing its direct operations. Finally, each successful applicant for funds would be required to contribute about 15% of total subproject costs in counter-part funds. 3. The intention is to establish, over time, a membership basis which reflects the demand ("users") for trained personnel. Hence government bodies, enterprises, banks and professional associations would be invited to join the General Assembly. Individual training institutions (i.e., Page 98 Annex 6. Organization, Staffing and Procedwres of dhe N suppliers of trained personnel), as the principal recipients of the Foundation's funds, would be excluded from membership on grounds of conflict of interest. C. ORGANIZATION AND STRUCTURE 4. Controls and lines of command are clearly spelled out in the Charter and the structure of the organization. The General Assembly is ultimately responsible for the Foundation, and appoints its Board of Directors. The Boards of Directors appoint the Executive Directors of esch Foundation. The Executive Director of the CF is responsible for the executive office, for supervisLkig the regional training foundations, including for the replenishment of special accounts. The General Assembly and the Board of Directors receive auditing reports from external auditors and are further supported by the recommendations of the Consultative Council. The Minister of Finance's representative has veto power over mars concerning the misuse of project and other government funds allocated to the Foundation. Some of these key functions and responsibilities are summarized in Table A6. 1. 5. GewiwAssembly. The ultimate authority of the NTF is the General Assembly made up of all its members from both the government and non-government side."9 The Assembly meets annually and the government members comprise half the voting power of the non-government side (no matter how many members) also has half the total votes. The functions of the Assembly are to admit new members, elect the Board of Directors, monitor operations of the NTF to ensure compliance, review and approval annual reports and, if necessary, amend the charter. 6. Consuaive Co'u *1, External donor organiations to the NTF would each have the right to place a representative on the Consultative Council. The functions of the Council would be to review the plans and performance of the NTF, advise the Board and General Assembly on policies, assist in fund raising and give specific advice on relevant topics as requested by the Board. The Council would come into, and remain in, existence when a minimum of five members join. It would meet at least annually, usually before the meeting of the General Assembly. 19. Founders of the NTP include the following: ypGe:nt (1) GKI (Goveamnt involvement in the establishment of a non-governmta organzaion is rricted to dhe GKI by Russian law). oleQvanmet (1) Chamber of Trade and Commerce; (2) Al-Russia Association of Ptivatized and Private Bnteprise; (3) Russian Union of Industrialists and Enteprener; (4) Assocition of Russian Banks; (5) Bank Assoiation Russia; (6) Association of Consukats on Economy and Management; (7) Assoiaon 'Support to the H;ighe Edcational Instisutionse; (8) Russian Association of Busne Schools; (9) Association of Management Devaepment; (10) Credo Bank; (11) Incombank (12) Bank Stolichnyi"; (13) Mosbjizesbank; (14) AKB *Stga"; (15) MAPO-bank; (16) ANPO ONaukaw; (17) NPO hrei Lavotchkina; (18) Electro_echnical Manufactur in Saonovo; (19) Eergia; (20) Joint Stock Company ParfuimFlakon'; and (21) Joint Stock Company Tat"r. Subsequet to the esblhmen of the NTF, the foUowing Goverrment agencies became members of the Foundaton: (1) Minisry of Labor; (2) Minity of Economy; (3) Ministy of Finance; (4) Office of the President; (5) Office of the Govr _ment; (6) GKI; (7) State Commitee on Anti-Monopoly and Support of New conomy Structures; (8) State Conmnitee on Higher Education; and (9) the Cental Bank. Ame 6. Organidn, Stffing and Procedures of the NTF Page 99 Table A6.1: NTF Struen -Key Functions and Resporulblles General Consutative Board of secretaria Technical Assembly Councl Direcors Staff Experb * Elct the Bod * Advisw on * Promote NT's * Do main operan * Do teccal of Directors potential pograms goals and and suppot evaluation of * Docide on critea * Avalyze suateg objectives f wctios (pam. 6) projects to and admitne of and annual plans * Approve sategy, * Selct expetsa for pibed new nmem of oe NTP and policy, aSiwal teculial review criteria * Amend the subm plans and * Screen subprojects Foundaon's recommendations programs * Decide on Charter to Board and * Expand NTF's financing * Decide on Assembly resources though subprojects below liqudation of th * Contbute fnids fiad-rasing $100,000 (neral Foundatko and promote fund- * Accredit Regional Evauation * Monitor .aising Foundations Coee) compline of * Review other * Approve structural operaions with matters as operaig nles of mandate and requestd by the fhe secretadiat polcies Board e Appoint die . Reviewand EBxecutve Director approve annu * Decide on rePors subproject financing if above $100.00 * Liaise with Consulatve council * Appoint auditor * Submi anna reports to General Assemblyions * Appmrove sucu operaing rules of the secreoariat * Appoint the Executive Director * Decideon msbproj financing if above $100,00 * Lisise with Consulaive Counci * Appoint auditor * Subnit annual reports to General _ Assembly Page 100 Annex 6. Organzation, Starffing and Procedures of the NWF 7. Teucld Advisoiy Cemmiftee. In addition to the above, to maintain a continued open dialogue with the users and the suppliers of management and financial training services and to keep informed about current trends and issues, the Central Foundation would establish and use technical advisory committees in their principal substantive areas of activity. The committees would include leading Russian experts invited in their individual capacity, coming from enterprises and academic and non-acaderic training and research institutions. The work of the committees would be informal and their agenda highly flexible. Experience with the groups of Russian experts who helped to prepare the Management and Financial Training Project and in other meeting would be utilized 8. The CF has a Dowd of Diectors, comprising 18 members elected by the General Assembly (I seat is reserved for each of the 3 Regional Foundations and will be filled once they are accredited).' The Board's major functions would be to ensure the survival and development of the Foundation. Specific functions embrace strategy and planning, programs, general policies and guidelines, financing decisions for certain projects specified in the internal regulations. The Board would be assisted by external auditors, and would meet at least three times a year (more frequently if necessary). The Board would also rely on the advice of an external Consultative Council. 9. Each Regional Foundation would have its own independent charter and organizational procedures, but to be accredited, it would need to meet basic criteria set up by the Central Foundation. These requirements include common goals and scope, explicit rules concerning the constitution of board, election and voting procedures, transparent and objective criteria for competitive selection of projects, regular reporting and external auditing and ovaluation. All documentation should be of public access. D. AcTIvI AND FuNcTIONS 10. The NTF would finance several categories of projects within the three main programs covering management development, the financial sector, and public finance. These project types include: a. development of policy and standards; b. curricula and materials development; 20. The first Board of Directors, eleced on September 7, 1994, includes the following organizations: a. Government (9 representatives) 1. Ministy of Labor 6. State Committe of Higher Education 2. Ministy of Economy 7. State Committee on Ani-Monopoly Policy 3. Ministry of Finance 8. State Property Agency (GKI) 4. Office of the President 9. Place reserved for the Central Bank 5. Office of die Government b. Non-Governent (9 representatives) Financial Instttions 1. Incombank 6. All Russia Assoon of Privatized and Pdivate Enterprises 2. Bank -Stolishyi- Professional Associations 3. Association of Russian banks 7. Russian Association of Business Schools EnterDrises 8. Association of Accountants and Auditors 4. Chamber of Trade and Commerce 9. Place reserved for a professional association 5. Russia Union of Industialists and Entrpreneurs Anne 6. Or.gar4don, Staffng and Procedures of the N7F Page 101 c. training of instructors; 4. training of practitioners; and e. distance learning projects. 11. In order to undertake these projects, eligible beneficiaries (which include training establisbments, companies, banks, government bodies and professional associatiom) could apply for assistance. These applications would be reviewed on the basis of an objective selection process (see paras. 15-24 of this Annex). 12. In the initial four training foundation entities (CF and three RFs) a number of main and support activities would be undertaken. Some of these would be centralized for efficiency reasons, others decentralized to allow local control and accountability. Broadly speaking this would be as shown in Table A6.2. Relationships between the Central and the Regional Foundations would be such that the Regional Foundations, at the regional level, would prepare and approve annual work plans and annual budgets; select, approve and supervise subprojects; sign contracts and approve payments. The CF would review these matters and give 'no objections", prior to the Regional Foundations proceeding with the activity. In addition, the CF wou'd maintain a single Special Account, authorize payments, apply for replenishment of the Special Account to the Bank, and maintain accounts and audit expenditures. The regions would repay a part of the Loan proceeds they receive to the CF, according to the subsidiary financing agreements between the Central and the Regional Foundations. The CF would bear the sole responsibility of repaying the Loan amount, interests and commitment charge to the Borrower. 13. The interaction of the regions and the center should be one of network and close coordination. It would be a relationship based not so much on hierarchical control but more on specialization. In the key areas of program design and project formulation, it would be expected that the CF would specialize mostly in projects of national interest and scope which should, for efficiency reasons, be done only once for the Russian Federation. This includes materials development, training of multipliers, distance learning and mass media projects. Ivioreover, it would accredit and monitor the Regional Foundations, encourage the creation of additional foundations later on and be responsible for training and developing its own staff and that of the RFs during the early stages of development. In contrast, the three RFs would specialize mostly in projects which need to be adapted to local conditions, such as training of consultants, practitioners and training of trainers. In both cases, however, information exchange and coordination are essential as several types of projects would occur within one program and fiurthermore a central project in any area such as materials development may be implemented within the regions. Such coordination and information exchange would work in several ways: through accreditation and monitoring procedures; submission of annual plans and reports of the Regional Foundation to the Central Foundation; ad hoc external evaluation of regional projects by the center; cross-representation at Board level, through working relationships of the general managers and program officers and through the centralized data center. Page 102 Annex 6. Organizaion, Staffing and Procedures of the NTF Table A6.2: Relatoubips Between the Central and Regional Foundations of the NTF Undertaken In same way at Regional and Central Levd (with OPERATIONAL TASK close coordiation) Mostly Centralzed Strategy and policy formulation V Accreditation of Regional Foundations __ _ Resource mobilization __ Program design l Subproject proposal review Financing decisions V l Drawing up operational procedures V Staff recruitme an management Training of staff, review panels, Board members and applicants, i.e. proposal writers AdministrationVt_ _____ Information technology and communication r V Data center and information exchange If Legal __ Public Relations __ _ Accounts V Asset/Liability management* Planning and budgeting V Collections and disbursements* _ Financial consolidation for reporting i ft * Aspects of these functions may be subcontracted to a commercial bank for efficiency reasons. E. MANAGEMENT ANI) STAFF 14. The quality and efficiency of the staff of the Training Foundation will be crucial determinants of project success. The following points are relevant. Annm 6. Organfzcaion, Sqffig and Procedures of the N1F Page 103 a. Management. The Board of each NTF would appoinm a capable Executive Director, selected from a multiple list of candidates, who would be supported by competent staff chosen under his personal responsibility, but based on explicit job descriptions and profiles. b. Jnternal evaluation commiee. For purposes of decision about specific projects, the senior managers and program officers would meet as an Internal Evaluation Committee, chaired by the Executive Director. c. S and recritment. Each entity would recruit all staff through transparent, competitive processes described in the Operations Manual. The staff would combine senior staff in management posiWions ano -elatively junior staff capable of being trained in new approaches and techniques without being set in old ways. d. Nunbers. A basic objective is to make each RF entiy efficient, iean, and flexible. The RFs would use external panelists and local consultants to provide professionalism on project analysis and minimize the permanent staff payroll. Keeping staff numbers down would be financially imperative and would make the NTF more acceptable to existing institutions and potential members. There would be limits on spending for operations and for the payroll.21 The challenge would be to avoid creating a large, bloated new entity, but at the same time to make it substantial enough to handle a significant volume of project applications with professionalism, thoroughness, and objectivity. Initially, the suggested staffing numbers to be progressively phased-in during the first phase is suggested in Chart A6. 1. e. Staff development. Training would be a key feature of the NTF. It would be primarily performed through pre-service training for staff, coaching through experienced consultants, and on-the-job training through frequent interactions within the staff and between the staff and the consultants. In addition, the Foundation would also train consultants and clients in project preparation, analysis and evaluation techniques. (See Annex 14.) f. International advisors and consulants. A full technical assistant program to support the NTF's start-up and initial operations has been planned. A senior advisor directly subordinated to the Executive Director would help coordinate all external technical assistance to ensure maximum effectiveness and consistency. To ensure uniformity and cross-fertilization, the same consultants would, whenever possible, assist the Regional Foundations as well. In addition, national consultants would work directly with international advisors, in order to facilitate their understanding of the local reality, facilitate communication and language barriers, be developed as senior consultants and provide services to the foundations in the network. (See Annex 13.) 21. Ie% target would be to keep operating expenditures at or below 10% of outlays for SUbprojects, which compans favorably wid the most efficient foundations in the West. page 104 Annex 6. OtwAn on, Stfing and Putedam qf *e NiT Chart A6.1: Organizaton Chart-Positions of the Cetral Foudllom EXECUTIVE DIRECTOR OFFICE 11 . i Secretary Logistcs EXENA EGOALOEATOSADMINISTRATION _COordwor . ~ ~ - ' Receptionislr Tel& fax operator DEPARTMENT 11DEPARTMENT DEPARTMENl | COOXrRDINATREIONA . OPERATIONS ADMINISTRATION I L~~~~~~~~~~~~~~~~~~~~~.J~~~~~~~~~~~~~~~~~~~~ l nformabon Officer Pro gram Offieromn Accou-tart I S ^ B ~~~~~~Mg. _ 1 1 | ~~~~~~~~~Program Offcer | Compute Personal Assistnt AccountnglFinance | pecadta Program Officer Procurement I Banidn I SpeciNst I Program OffeLrI Public finance & Secretary PFot1p Secretary L Secretary Pawn I L I S .~~~~~~~OML Anney 6. Organia , &4Lng and Pocedures of the NF Page 103 g. Ad-Ie consuts. In addition to the permanent staff, the NTF would use professional review panels to provide external review of competitive project proposals. Use of such expertise would be a cornerstone of the review process and a key element in ensuring that objective and professional criteria are consistently applied to project financing decisions. h. Rera*on would be at competitive levels to attract qualified people on a full- time, exclusive contractual basis and to minimize staff turn-over in a high-inflation environment. F. PROJCCT CYCLE EVALUATION PROCESS 15. Ihe management and program officers would be responsible for the project cycle activities. Those span at least eight stages (the first five of which are summarized in the following diagram), and are elaborated in Chart A6.2. 16. Stage 1: Program Annowecemet a. Develop program guidelines, including description of eligible projects; b. Specify funding mechanism-grant or loan or both, with an indication of the merit and financial criteria used for review; C. Develop definitions of eligible institutions and organizations; and d. Develop timemble and set deadlines; these should be included in program announcement. 17. Stage 2: Issue RequWest for Project POp4als (Project ldnbtcation) a. Draw up list Af institutions or consultants to be invited to submit project proposals (in the case of restrictive bidding); b. Issut official announcement, using press and other media where appropriate, ensuring maximum transparency, and access to information by all intersted pardes; c. Respond to requests for information; and d. Offer training in proposal preparation to potential bidders. This would Initially be extensive and require detailed handbooks, guidelines, etc. but this function can be phased down as potential proposers gain experience with the system. Page 106 Ann= 6. Organization, Staffing and Procedures of the NTF Chart A6.2: Subproject Evaluation Process NTF provides PrTFlm ~ training to potential Pmgram ~~~~~~~~~~~~~bidders and annourncement | x >applicants 3 APPLICANT/ Retum to applicant -No ~~~BIDDER with explanationl submits proposal to Instruction No * ~~~NTF 4 * ntudo 7 No Program officer1. Receies and checks proposal |pAppraisal by Yes | - - ~~~independent 4 * Program Officer. Central or Regional |/ ~~~~~~~~~~peae ntemlEauto No-* NoN IfNoYes- roet No No Yes zM* Oi~~MUII Board Review Yes review World Bank piior No Annex 6. Org xaion, Staffing and Procedres ofthe NF Page 107 18. Sage 3: Project Pw-Appraol a. Receive project proposals in accordance with announced deadlines and record receipt. b. Check proposals received for compliance with NTF format and other conditions including application and register all applications received; and c. Examine applications and return those not in compliance to the clients with clear information about reasons. For example, proposals may be deemed unsuitable for review on different grounds: * The proposed activity is not responsive to requirements, as specified in the program guidelines; * The proposed activity is clearly not of sufficient quality; * The proposing organization is clearly incapable of delivering the project; or * The proposal is incomplete or does not comply with the formal requirements of the NTF. 19. Stage 4: Project Appraisal For this stage, the program officer should divide projects into different categories, according to the operational procedures. a. For smaller projects (i.e., those under US$50,000), two independent reviewers from the roster of approved reviewers would be selected for independent review appraisal of each project. If needed, a third independent reviewer could also be asked to present comments, at a later stage. b. The project officer would ensure that reviewers have no conflict of interest and receive clear guidelines (and training, initially) about ethical issues, the content of the review and the format in which they should send their comments. The reviewers should also receive specific instructions about postal deadlines and payment of fees. c. The project officer would formulate his recommendations to the internal review committee, which would then deliberate. d. For projects over US$50,000, the steps are similar, except that a panel review meeting would be convened to discuss the relative merits of a large batch of proposals under a same program. e. At the end of the deliberation, the review panelists would present a summary with their views, using a standard format. f. The evaluation officer would add any further comments and submit the proposats to the consideration of the Board of Directors, through the Executive Director. Page 108 Annex 6. Orgaaon, Sqfing and Procedures of the N7F g. At any time during appraisal, the program officer may contact the proposer to ask for additional information, clarification or undertake on-site visits or inspections. He can also commission experts to undertake such visits. In any case clear terms of reference should be specified beforehand, and detailed records of these visits should be attached to the project document. 20. In the case of loans, and simultaneously with technical appraisal, the program officer would submit the institution's financial data to the Director of Administration and Finance to process the information and decide on the ability to pay. Projects with no financial ability would not be submitted for further decision, and proponents would be informed of the reasons through a standard letter. 21. Stage 5: Projet Financing Dedsion a. The Internal Evaluation Committee (IEC) would review all subprojects. Any deemed unsuitable would be rejected and returned to the proposer with an explanation of the reasons. If the subproject is below $100,000, the IEC could approve the project. If above $100,000 the proposal would be sent to the Board of Directors with the recommendation of the IEC. Special considerations are as follows: * first, have the first stage procedures and criteria been applied during appraisal? * second, are there any broader issues of "fit" of the proposal with other NTF objectives? * third, are there any further benefits (externalities) of the proposal with other NTF activities? * fourth, has the ability to share costs or re-pay been convincingly demonstrated? b. Based on this analysis, the internal evaluation committee or the Board of Directors decides whether to approve or reject the financing. Loan and grants terms of conditions should be decided before hand, and any particular conditions should be incorporated to the written terms of reference. c. For projects t!at are approved, those which are greater than US$250,000 would be translated into English and sent by the Executive Director to the World Bank for prior review and approval. Those below this limit and those approved by the Bank you be passed on to the Finance and Administrative Manager for signing a legal, performance-based contract. d. Finally, formal, standard communication will be sent to all proposers, stating the reasons why the project has not been approved. The program officer would be responsible for giving this information while protecting the anonymity of the reviewers. This information should be as clear and detailed as possible to encourage rejected bids to be resubmitted if future occasions arise. 22. Stge 6: Projeed Impkmenti Am= IL OrpnkAgon, Sting and Procedres of the N7F Page 109 a. The administration manager would receive the approved proposals from the internal evaluation committee or Board (depending on project size) and would responsible for signing a grant or loan document with the beneficiary and providing regulations on disbursement and procurement. b. Once signed, the treasurer and finance manager receives the file with loan or grant information and would be responsible for disbursements and the r mitoring of procurement in accordance with the rules and regulations (alternaaLvely, if a suitable commercial bank can be identified, these tasks may be subcontracted). c. The program officer would be responsible for checking that the implementation is proceeding smoothly and should assist if necessary to clear any difficulties. 23. Ste 7: Project Superirion Once disbursed and operational, the supervision process requires the program officer to: a. Collect information (covering training, administrative and financial aspects) in a regular reporting format and assess performance; b. Visit projects according to a schedule of implementation approved by the Executive Director and taking into consideration size of subproject to check on implementation (such visits can also be delegated to outside consultants and be performed using standard terms of reference and forms); and C. Provide corrective advice if there are problems and keep record of such activities. bm finance department would be responsible for collecting the loan and dealing with repayment problems. Appropriate collection mechanisms, as well as legal and other sanctions and penalties, would be stated in the contract (alternatively, if a suitable commercial bank can be identified, these tasks may be subcontracted). 24. Stage 8: Tual Evaluaaon The program officer would need to undertake a final evaluation report to: a. Measure achievements against plans and targets; b. Quantify the impact by using adequate performance indicators; and c. Draw lessons from past projects to improve the design and appraisal of future projec. The program officer could commission evaluation studies of specific subprojects to collect data about project impact. The internal auditor should do an independent review of all such final evaluation reports on a sample basis. Page 110 Annex 6. OrganizaWon, Stqffhg and Procedures of the N7F G. FINANCIAL MANAGEMENT 25. Absolute financial integrity and professionalism are essential for the NTF. Some of the key aspects of its financial management would include the following. a. Sowd prinies. The NTF would have detailed financial policies based on specific principles, including that it would operate as a financial institution, would try to match assets to liabilities to the extent possible and would cover operating costs with its own income (based on commission fees). b. Endowment fld. In the long run the Foundation would need to establish an endowment on the basis of resources through donations and other sources. The Board of Directors should oversee the operations and management of the endowment fund, and would be advised by the Executive Director and the Manager of Administration and Finance. c. Alloaton of fieu. Initially the World Bank line of credit would be divided into two categories: one allocation for the CF and one for the three RFs. Each RF would have an incentive to commit and spend funds expeditiously. Delays could mean sacrificing funds to other regions that are more effective in funding subprojects. d. fanct ,.nsacdons. In order to minimize the amount of financial transactions within the four NTF entities, one Special Account would be created, under the supervision of the Executive Director of the CF. Coa RtcoV*ei 26. In its initial phase, most of the subprojects would be in the form of grants. Yet, during the initial phase, three types of cost-recovery would be possible by collecting: a. Commssion and other fees pid by the subprojed beneficarks; b. Pafpation fees from trainees participating in training activities sponsored by the Foundation; and c. Loan rexnbwsenenw and Interest paid on loans. Projects submitted by individual firms would also be reimbursable according to a schedule previously approved by the Board of Directors. TMe decision on whether to recover costs would be driven by the type of project. Development projects (e.g. standards or curricula development) would genera!ly qualify for grant financing. Some development activities, e.g. teaching materials and textbooks or even teacher training, might yield partial cost recovery. The income stream of the activity would need to be examined for such activities and decisions made accordingly. Other activities, such as delivery of training to commercial clients or participants, would-as a rnle-be subject to substantial cost recovery through loans or payback of participant fees. Anna 6. Oe?ad*on, &nSg and J oedmu if the NM Poge III EL QUAUJTrY CONRtOL Accwtabl 27. Mechanisms built into fth suctue of the project allow for accountability and quality control, summarized as follows. a. lhere would be a clear line of command flowing from the General Assembly to the Board of Directors to the Executive Director. Each of theso levels have clear responsibilities, mandates and authority over the lower level. b. There would be a clea definition of the system in th form of a network, with explicit criteria for membership and strict contols over program, procedures and actual disburseme, under the supervision of the Executi Director of the CF. The Executive Director enjoys considerable autonomy, but would be totally responsible and accountable for the operations of the Foundation c. There are two outside performance moniorng bodies, the exteral auditors and the Consultative Council, which would report dircdy and independenty to the Board of Directors and General Assembly. d. The Ministry of Finance retains veto power over specific allocation of resources considered as inconsistent with the projects agreemen. e. Members of the Boards and the Executive Drector are chosen through a set of criteria aumed at ensuring objecivity and fairness, and at avoiding direct political interferc or patronage. f. The Executive Director is chosen through mulple names and short-lists, to ensure a collegiate decision and responsibility of the Boart for this appointmn. He would be responsive at the Board as a whole, not to any individual member. g. Staff are selected on the bases of clear job description and following procedures which maximize transpareny and quality. Saqevbkn wd Mon*tortng 28. It would be a challenge to launch the NTF activides successfilly and a significant learning process would be required. Consequently, close supervision would be needed throughout to monitor the quality of the activities. From the World Bank perspective, there are a number of prime components in a supervision rategy: a. frequent supervision visits-probably every twotr months initally; b. prior review of the first five sub-project appraisls from each NTF entity, i.e. twenty in total; c. prior review of any subproject greaer than US$250,000; and Page 112 Am=z 6. Orgadizauion, Si and Prcedrm of du NIT d. conuc to expaud the membership base, strengthen the Consultative Council and interaction with the Board of Directors as we!l as key players in the General Assembly. (See also Annex 18.) THE RUSSIAN FEDERATION MANAGEMENT AND FINANCIAL TRAING PRojEcr ANNEx 7. NATONAL TRAIG FOuNDATION EVALUATION PLAN A. LEVES OF EVALUATION 1. Evaluation in the NTF is designed and implemented at four levels in order to best serve the interests of the multiple NTF shareholders: a. NTF impact (overall, program area, project results); b. NTF operations; c. "subject" area learnings; and d. individual subproject operations and results. 2. Except in the case of individual subproject evaluation, evaluation studies would be carried out by independent third-party experts contacted specifically for this purpose. Subproject evaluations are carried out in part by NTF program officers as a part of their monitoring role, and in part by sub-project management, using a self-evaluation approach. Leve Om - NT Ipd 3. Evaluation of NTF impact should be focused to serve three purposes: (i) to be accountable to NTF sponsors for faithfully addressing NTF objectives and mission; (ii) to guide decisions by the NTF Board and senior executive staff regarding overall NTF policies and goals in order to strengthen the NTF contribution to the establishment of a market economy; and (iii) to provide progran area leadership (program officers, advisers) with information needed to judge program effectiveness and strengthen program and project performance. NTF impact is defined as the impact, in aggregate, of each of the several programs mounted by each regional and the central NTF, NTF impact evaluation would be designed and implemented by program, where each major NTF program (i.e., management development, the financial sector) would be the unit of analysis. 4. TraItg Iact isdicators. Evaluation at this level assesses the extent to which the NTF program impacts and contributes to strategic business and economic development goals. Assessment of impact would include measurement and assessment of the following sorts of indicators: a. numbers of needed professional personnel trained by sub-projects (by categories and level of credentials); b. numbers of managers trained in market-economy skills; c. numbers of public officials trained; d. numbers of training professionals with upgraded credentials; e. numbers of trainers of trainers trained; f. the extent to which shortages of personnel in the above fields have been reduced; Page 114 Annex 7. N7F Evaluation Plan g. increased capacity for training needed personnel enabled by infrastructure achievemlents, such as training partnerships, financing of market-oriented programs, professional association programs established, etc.; h. nature and scope of curricula and curricular materials produced, revised, pilot-tested, disseminated, etc.; and - i. nature and scope of policies, standards, guidelines, etc. produced or revised to support training of needed personnel. 5. More detailed, specific, and relevant evaluation indicators would be developed for each program and project area prior to evaluation studies. These would be developed by the evaluation contractors in consultation with the pertinent advisers, local and foreign experts as necessary, and the NTF program officer responsible for the program/projects being evaluated, and other key shareholders (e.g., sub-project managers). 6. OveraUNlFIads dkatens. Impact evaluation also includes assessment of the extent to which the NTF has achieved its goals of strategic positioning. This assessment includes the following questions: a. To what extent has the NTF funded projects in the aimed for proportions in each of program areas? b. To what extent have subprojects achieved relevant and important outcomes; and to what extent have they achieved quality comparable to leading institutions in the country? C. To what extent is the NTF perceived among Russian training and development opinion leaders as a fair, open, and equitable agency that makes merit-based decisions and, in general, adheres to high standards of professionalism? d. To what extent has the NTF mobilized resources from key lenders, investors, and donors? 7. Data sources at this level are (i) aggregated project results, (ii) NTF operation, (iii) third party policy and impact evaluation studies, (iv) program officer reports, and (v) special "subject" area evaluation studies. Impact evaluation of the NT1F includes all regional results in aggregate; it is not differentiated by region. LUv Two: NW Operadons 8. Evaluation at the NTF operations level assesses the effectiveness and efficiency of foundation operations and organization. Conclusions would be drawn about the entire foundation in aggregate, and about each individual Regional Foundation site. Again, purposes are twofold: (i) accountability to NTF sponsors and shareholders, and (ii) improvement of NTF policies, procedures, and capacities. 9. Evaluation at this level should include systematic 'customer' feedback from those who are the recipients of NTF funds and services, and those whose role is provision of technical guidance and services, such as review panels and technical assistance councils. Ann 7. Z F EVuaudon Plan Page 115 10. Measurement and assessment at this level would Include: a. adherence to NTF guidelines, policies, and expectations for funding operations, fiscal integrity, accounting procedures, etc.; b. integrity, efficiency, and practicality of proposal review and selection procedures; c. internal and external communications; d. staff qualifications, selection, training, and other personnel procedures; and e. quality and efficiency of services provided by NTF. Level Three: Subject Areas 11. Evaluation at this level is primarily intended to produce learning that would shape future NTF objectives and strategies. Inquiry is across carefully formulated samples of funded projects with similar missions, target audiences, delivery strategies, and so forth. A program area evaluation might seek to learn, for example, what training strategies are most effective in decentralization of training expertise, or what curricular development projects have learned about linkage with professional groups and other customers. 12. Subject level evaluations are sporadic and highly targeted, being commissioned and designed as needs and NTF experiences dictate. They should be conducted by independent evaluation and subject-matter experts, but should involve subject-area project staff in instrument design, data collection, analysis, and interpretation. Subject area evaluations would typically incorporate staff development methods, employing workshops and seminars to formulate evaluation approaches, and interpret results. 13. Results from subject area evaluations are used to shape NTF program area goals and policies. These evaluation results should also be widely disseminated to inform the profession at large, and thus incorporate dissemination strategies into the evaluation design. Leve Four: Evaluaom of lnvidaal Fnded Projects 14. Subproject level evaluations serve both an accountability and a subproject improvement function. Subproject level evaluations are conducted from two perspectives. First, the NTF program officer conducts basic accountability evaluations for each project within his/her purview, to assure adherence to NTF guidelines and sound professional practices. Secondly, sub-subproject managers are responsible for designing and conducting subproject level evaluations according to NWT guidelines and supported fiscally from subproject proceeds; they may, at the discretion of subproject managers, employ external expertise. 15. For accountability purposes and to enable estimates of aggregate results across subprojects, subproject level evaluations designed and conducted under the guidance of sub-subproject managers must include mandatory NTF evaluation odestions. These questions are tailored by sub-subproject managers to be specific to their individual subprojects, but take the generic form of: a. To what extent have subproject terminal objectives been achieved? b. What progress has been made toward achievement of terminal objectives (enabling objective achievement)? Page 116 Annex 7. Nh Evmwadin Plan c. To what extent have target groups benefited, primary or secondarily, from subproject activities? d. What program-area learnings has the subproject achieved? 16. The remainder of the subproject level evaluation is driven by individual sub-subproject improvement needs. These evaluation questions would vary by individual subproject, but overall guidelines would recommend that certain dimensions be assessed, such as: a. "customer" feedback and impact on "customers"; b. effectiveness of managerial operations; c. progress on quality improvement efforts; d. results of and learnings from innovative methods, pilots, and trials; e. comparisons of alternative approaches; and f. follow-up and feedback from "graduates" or their employers (where appropriate). 17. Evaluation of individual subprojects from an accountability perspective is supplemented by regular site visits conducted by the NTF program officer. During these visits the program officer would interview subproject staff and clients (as appropriate), r.view subproject records, and conduct other inquiry as necessary to determine that NTF expectations, policies, guidelines, and procedures are being adhered to. Program officers would file a share a draft of their site visit report with subproject management prior to filing the report with the NTF, and would incorporate subproject management reactions into the final report as appropriate. B. SELF-EVALUATION PIOSOPHY 18. Individual subproject level evaluation policies and procedures embody a "self-evaluation' philosophy that views evaluation as an essential element of organizational learning and effectiveness. The NTF would provide guidelines for evaluation design and implementation, and would also make available training and other technical assistance to support funded subprojects in effective evaluation. However, designing, conducting, and using the subproject level evaluation are the responsibility of the subproject. Funding requests should provide budget specifically for evaluation, and a portion of funding proceeds should be used specifically to support subproject evaluation. 19. The overarching purpose of subproject-level evaluation is to inform decisions by subproject management that would guide continuous improvement of subproject quality and impact. This self-evaluation approach is based on principles of individual and organizational learning, and recognizes that evaluation for self-improvement must be differentiated from and balanced with evaluation that serves an auditing or external accounting function. Subproject level evaluation that is used, or is perceived to be used, as a "policing" function by the funding agency, suppresses the risk taking that is critical to learning and development, and otherwise invites bias in evaluation reporting. The Foundation approach to subproject self-evaluation is to: a. Require that effective evaluations be conducted; b. Clarify the key role that evaluation plays in subproject development and improvement; c. Provide support for evaluation, in the form of clear policies and guidelines, and training and technical assistance; Anna 7. 17 Emwlaaon Plan Page 117 d. Require evidence that evaluation has been conducted and used to i. make important subproject improvement decisions; ii. support the development of subproject staff; iii. incorporate customer needs and impa into subproject planning; and iv. demonste accountability to key stakeholder groups. 20. Only evaluation data on the mandated subproject-level evaluation questions must be reponed to the NTF; internal evaluation repots for subproject improvement are not intended nor required to be provided to the NTF. The subproject must report, however, on the nature and scope of internal subproject evaluation, and provide evidence that such evaluation results have been inrpreted and used. 21. EZwladi Advsi Boyrd. An evaluation advisory board comprised of independent evaluadon experts would be formed to advise NWT evaluation policies, procedures, and criteria and would sere to coordinate and guide subject area evaluation studies. The aim of the advisory board is to assure that NWT evaluation approaches are effective in furnhering the goals of the foundaion. THE RUSSIAN FEDERATION MANAGEMENT AND FINANCIAL TRAINNG PROJECT ANNEX 8. MoNwIoRING AND IMACT INDICATORS I1. The impact indicators for the project and measurements of its success have to be derived from the purpose of the project and its major objects. The purpose of the project is to contribute to the successi transition of the Russian economy from the command to the market by increasing the supply and qulity of necessary sklls. The purpose wIll be fulfilled by the attainment of three main objectives as outlined below. A. INCREAsiNG THE SUPPLY OF wELL TRAIED MANAGERs MD PROFzSONAts IN MANAGEMENT, ACCOUNTANCY, AND PUBLIC FINANCES Indicators Measruenwit 1. Develop suitable policy framework for training in the Specific proposals formulated and defined areas. submitted 2. Develop and have accepted quality standards for Standards produced and accepted training 3. Curricula, study guides and text-books prepared Number of subjects covered Number printed and sold 4. Self-development materials designed, produced tested Number of subjects covered and used. Number of packages produced 5. Trainers selected, full trained and used for training Number of trainers/instutors practitioners traiined 6. Number of practitioners (managers, accountants, Subject covered bankers, public finance professionals) Number of practitioners trained 7. Existing faculty members retrained in new Number retrained finance/accounting system Number of schools covered 8. Training Center for Finance/accounting training set, Number operational equipped and made operational Page 120 Anna & Monkorang and lact Indk*on B. PREPARING FOR PHASE 2 OF THE PROJECT AIMING AT EFFECWIVE IRVENRTON TO INCREASE KNoWLEDGE/SKILLS AND DEVELOPMENT OF CO ciVE ArUDES FoR OTnER MARKET-ORIENTED FIELDS. btt#aatos M I. Preparing two pilot projects for the area of small Project defined and implemented business development 2. Prepare two pilot projects for public administration, Project defined and implemented e.g. top executive program and preparation of regional Number of executives trained training 3. Preparation of pilot programs for market economics: Projects prepared training of journalists and design of system Number of journalists trained C. SEFNG UP AN EFECTVE OPERATION OF THE NATIONAL TRAING FOUNDATION AS TH MAIN INSU ONAL INSTRMUMET FOR TRAINNG ACrlVITIES/PROJECrS IN THE DSEFD ARES hndkators Measwvne* I. Effective organization and systems for NTF designed Organization chart, job descriptions, and out into practice operations manual produced 2. Effective project plhning and monitoring system in System designed and operational operations (computer-based) 3. Quality staff recruited, trained and highly motivated Performance review 4. All operations in compliance with strict ethical Assessment standards 5. Additional financial resources from donors tapped Amount of external funds raised 6. Clearing-house function for training resources and Data-base in operation activities operational Number of users 7. Inplementation Numbers of subprojects supported Financial outlays THE RUSSIAN FEDERATION MANAGEMENT AND FINANCIAL TRANIG PROIET ANNEx 9. SuBrPoJECT APPRAIAL CRITRI 1. lThe NTP would support subprojects that are (i) most responsive to NTP development goals and strategy, (ii) are most likely to produce results, (iii) are efficiently designed, and (iv) have the financial capacity to suppOrt proposed operations and pay back loans, where appropriate. These four major criteria are described in more detail in the following explanations and lists of sub-criteria. 2. The criteri in this amex would be furher defined prior to their application to a specific programlsubproject proposal review effort (i) specifically for each progran area (such as mauuzgement development), and (ii) by he NTF program officers worldng with the technical advisory board and other experts in their program areas. These draft review criteria would then be recommended to the advisers for their review, as a part of the program development process. IrW Screening of OveraU Proposal 3. Proposals must be effectively organized and clearly wntten so as to facilitate the review and award process. Proposers should follow NTF guidelines, and take other steps as deemed useful to assure that their proposal accurately represents their inenions, and directy responds to NTF needs. The initial reviewer would assess: a. Clai*y. The proposal should be clearly written, using direct and jargon-free language. b. Organization. Proposals should be effectively organized so that reviewers can clearly understand intent, while readily assessing the proposal against NTF guidelines and criteria. c. Internal consistency. The several parts of the proposal should clearly fit together in a logical and consistent manmer. d. Ecency. The proposal should be direct and straightforward, clarifying intentions and critical information in the least complex and most concise manner possible. No extraneous information should be provided. Analytia Framework Analytical FramEwork C. B. A. Criteria: Effifiney Effectiveess Relvae 0 _n 3~1 4- Supr;pff Page 122 Annex 9. Subproject Appraisal Criteria 4. The criteria work backwards from "needs' through objectives and outputs to inputs. The first criterion, relevance, addresses the extent to which the goals and objectives of subprojects would meet the overall objectives of the project and promise the most direct and worthwhile results in meeting the needs of transition to a market economy. Judgments of relevance would be based on an assessment of the immediacy of impact, and whether projects aim to provide urgently needed skills to high priority target groups. Effectiveness deals with the likelihood that the proposed project would work. Assessments of effectiveness would be based on analyses of clarity, the completeness and feasibility of the proposed design, degree of innovation, mechanisms for customer involvement, the qualifications of staff and consultants involved, strength of managetnent experience, institutional support and provision for leveraging results. The third criterion of efficiency assesses the extent to which projects are designed to produce intended results with the least expenditure of resources. Judgments of efficiency include assessment of training unit costs, utilization rates for equipment and physical facilities, security provisions, and integration of efforts with other projects to avoid duplication and increase cooperation. The sequence of analysis is highly important. For example, if subprojects are not deemed relevant it does not matter whether they would be effective or efficient. A. RELEVANCE-TH RELATIONsIP OF SUBPRoJECT OwECTIVES TO PRIOURTY REQUIREMENTS 5. The basic questions pertaining to relevance are: Should this project be done? Is it a priority? Would it help achieve goals of overriding importance? The goals and objectives of proposed subprojects nmst be relevant to identified NTF strategies and priorities. Subprojects receiving preference would be those that promise the most direct and useful results i meeting the needs of transition to a market economy. Judgments of relevance would be based on the extent to which the subproject's goals promise: a. Criticalfields. Production of demonstrably critical market skills. Subprojects mnust target specific skills and knowledge that are in clear demand for transition to a market economy. Sklls must be specifically defmed, and evidence should be provided to support the rationale and need for these skills. b. Poriy ty et groups and coverage. Meeting needs of target groups in priority fields (management development; finance; public administration). The NTF has identified specific target groups that most urgently require skills and knowledge, particularly practitioners in each field. Subprojects that promise quality service to the greatest numbers of these groups are preferred. c. T1meiness. Immediate impact on and utility for skill production. Preference is given to subprojects that can deliver the most immediate impact on critical skills production. Subproject cycles should be as brief as possible to assure timely results. Subprojects that have overly extended time-lines would not be considered as important as those that assure quicker delivery. d. Mulier effect. Creation of capacity for continued production of needed skills over longer term. While rapid transition creates urgent needs for immediate skill production, the longer term requires capacity to continuously provide needed sldlls. Subprojects that aim to create expanded capacity and continued skill production while meeting immediate needs are preferred. Anna 9. Subproject Apprisal Criteria Page 123 3. Lw CNVfTHE RELATIONSHIP OF OuruTs TO OBwECnvEs 6. The basic questions concerning effectiveness are: Can it be done? Would it be able in fact to adcieve its intended outputs? Does the proposed subproject provide all the inputs necessary? Are these organized into a coherent design? Subprojects must be capable of producing high impact and high quality results, and must be supported by sufficient resources to achieve those results, i.e. proposals must clearly and completely represent sufficient plans and resources to achieve objectives. Judgments of efficacy would be based on the following considerations: a. Avaiabiy of inpats. Would subproject inputs likely be available in the quantities and qualities required, as and when required? Are there any gaps in the proposed inputs? i. Trainers, instructors and subproject staffing. The qualifications and credentials of project staff must be relevant to, and sufficient for, the proposed activities. There must be assurances that identified staff are available for subproject duties, or that staff to be recruited can be readily located and recruited. ii. Teaching curricula and program content. Do modern, tested teaching programs already exist that could be used? Mii. Teaching materials. Do textbooks and other materials exist that are relevant to the Russian context? iv. Qualfied trainees. The bidder should be able to demonstrate that sufficient qualified trainees, with sufficient educational and professional background, work experience and language skldls and, if relevant, ability to pay would have incentives to participate in the proposed subproject. V. Project management. The management structure must be clearly defined to demonstrate adequate procedural guidance, constructive controls, and clear delineation of responsibility and authority. Managerial staff must be assured in appropriate number and qualification to lead the subproject successfully. Past experience of the institution in managing subprojects of the sort proposed should, where applicable, be demonstrated. vi. Erpert serices Oca and eternal). Are qualified experts available and wiling to work for agreed rates of compensation? vii. Support senvces. Where support services (.edrical, accounting, computer maintnance, for example) are critical to subproject operation, there must be assurance that sufficient and high quality services would be available. viii. Physical facilies and security. The subproject must assure adequate, consistent, and upto-date facilities, space, equipment, and technology to support subproject operations. The subproject proposal should define and descnbe appropriate procedures to safeguard the personnel, equipment, and information involved in the subproject. ix. Local cost financng and finanal resources. The potental contractor sholdd be able to demonstrate an ability to finance all necessary inputs, including a 15% share of total costs, and to pre-finance activities for subsequent reimbursement based on results. AVge 124 Annax 9. Subproject Appraisal Criteria b. Subproject design. Is the subproject well designed to achieve the intended outputs? i. Overall design. To what extent is the subproject design clear and well thought-through? The subproject design must contain carefully articulated activities and sub-activities that are based on knowledge of best practices in training, curriculum development, adult leaming, and so forth. Activities must be of sufficient scope, duration, and potency to meet subproject goals, and must be articulated in a logical work-flow. Evidence that plamned activities are capable of achieving objectives (such as pilot tests, past evaluations, research data) should be provided to support plans. ii. Linkage to customers. There must be evidence that subproject customers have been, and would continue to be, involved in subproject design, operation, and evaluation. Clear and effective linkages to customer groups, such as review panels, provisions for customer feedback, and so forth, should be part of the management plan. iii. Training strategy and mehods. Is the training strategy in the subproject the best way, under the circumstances, to achieve the training goals? To what extent would the subproject test, develop and disseminate effective tring methods? The present training profession tends to practice outdated and narrow methods (e.g., theoretical lectures, memorizing). Subprojects that aim to contribute professional knowledge and research about promising methods (such as action-oriented simulations, case methods, group learning, etc.) while producing immediate skill impacts are preferred. iv. Anticpation of obstacles. The subproject design should demonstrate that planners have considered and accounted for systemic obstacles to and, enhancers of impact. Subprojects should include, as appropriate, activities that would assure and support adoption, integration, and nurturing of new market-oriented practices in the workplace. V. Cutural adaptation. The subproject must be staffed, designed, and managed to assure consonance with the Russian context in general, and the specific cultural context of the subproject's customer group. Where subprojects aim to adapt foreign materials or methods, procedures to integrate and accommodate Russian needs and cultural particularities must be articulated and assured. vi. Evaluaton plan. To what extent are targets clearly identified and quantified with milestones established for their achievement? To what extent has a satisfactory plan been prepared for monitoring and evaluation of the subproject and provision made for corrections for deviations during execution? Subprojects must incorporate an adequate evaluation design that is ethical, useful, practical, and accurate. The evaluation plan should provide for the systematic collection and interpretation of information that subproject management can use to improve continuously subproject operations and impact. Ann 9. Subprojec Appraisal Citeria Page 125 c. Implementadon capacity. To what extent does the bidder have the ability to carry out the subproject as intended and produce quality results? i. Does the subproject management have sufftcient experience in implementing similar subprojects. ii. To what extent has the bidder identified adequately the staffing imnplications of subproject execution and has assigned the neesary technical and managerial capacity and expertise to carry out the subproject effectively? iii. To what extent is the detailed implementation schedule realistic and to what extent are the resources provided for its attainment? C. EFICIENCY-THE RELATIONSHIP OF INPUIS TO OUTrPUs 7. The efficiency criterion involves two basic alternative questions: (i) to what extent would the subproject produce the intended outputs at least (or reasonable) cost?; or (b) to what extent would the suIIroject, given a fixed level of costs, maximize outputs? Subprojects should produce the effecdve rsuts with the least possible resource expenditure. There should be assurance that the subproject pposers have planned for reasonable costs and considered needs for efficiency such as cost controls, linkages to other institutions that can achieve savings, methods of extending impact, and other means that can assure the greatest possible returns for a given expenditure. Judgments of efficiency would consider: a. Let-cost approah. To what extent have alternaive means to achieve program objectives that cost less been considered and adopted? i. Unit costs. Have proposed capital and recurrent unit costs (i.e. costs per trainee) been identified accurately? Are they reasonable? Do they conform to national and regional norms? To what extent have sales of economy been realized? H. Utilization rates. Have proposed utilization rates for staff, physical facilities and equipment been projected and are reasonable and within nonns? Use of space, capital equipment, consultants, and so forth, should be reasonably intensive and budgeted (e.g., sharing expensive equipment, renting rarely used equipment, outsourcing special services) so as to control costs and maximize returns. iii. Integration and coordination of efforts. To what extent has the subproject proposal been identified and taken into account work already done on the proposed topic? To what extent does the proposal make use of existing and newly developed training programs by other donors? There should be evidence that subproject planners have explored their context to identify similar or compatible efforts in order to form alliances and partnerships where feasible, and to avoid duplication, overlap, and interference of planned subproject outcomes with those of other efforts. iv. Administrative overheads. To what extent is the proportion of expenditures on overhead (administration and services) reawonable in relation to expenditures on actual development or delivery of training? Page 126 Annex 9. Subproject Appraisal Criteia v. Sustamnability and replicability. To what extent are the project recurrent costs of the subproject financially sustainable over time? Would the subproject generate sufficient revenue to becomne self-sufficient? To what extent is the subproject a model that could be afforded and replicated on a wider scale? b. Maximizing results. To what extent would the subproject, at given costs, multiply benefits? i. Multiplier effect. Training and other methods should be designed so as to maximize the audience and coverage for a given level of expenditures. Training of trainers is one method to have a "multiplier effect." Another is the use of distance learning methods. Distance learning technology can reduce travel costs in comparison with central-site training and have the potential to reach vast audiences with effective training programs. Subprojects incorporating distance learning methods would, therefore, be given priority in the approval process. .t. Dissemination and leveraging. Subproject proposals should provide clear indications of how the results would be disseminated to wider audiences. The subproject should, where possible and appropriate, incorporate means to extend results beyond immediate objectives. Inclusion of methods to extend and multiply subproject outcomes would increase likelihood of NTF support. D. FINANCLAL CONTmiUTION AmD CAPAC]TY2' 8. The institution housing the subproject should demonstrate capacity to maintain operation of high quality subprojects such as the one proposed. This would include the capacity to service debt, to raise funds and to provide accurate accounting. In cases where student fees and/or loan funds are insufficient to produce impacts, sources of additional funds must be adequately assured. Preference would be given to subprojects able to assure customer or other stakeholder group funding, such as matching grants and in-kind contributions. Judgments of financial capacity would be based on the following criteria, which would be variably applied to subproject proposals as determined by the extent to which the subproject entails a loan, a grant, or a combination of a loan and grant. a. AbiW to rey the loan. There should be evidence that the institution accommodating the subproject, and/or the revenues to be generated by the subproject, would be sufficient to repay the loan according to the terms specified. b. Accowing faciities. The subproject should assure that appropriate accounting facilities and procedures are available and would be used to track subproject accounts. 21. Applies mainly to loans for subprojects. Anna 9. Subproject Appraisal Criteria Page 127 c. Conbution of resosrces. The proposal should include identification of speciflc financial or in-kind contributions to support the proposed subprojet. The contributions should comprise at least 15% of the total subproject budget. d. Addional criteria. To be added. E. SUMMARY EVALUATION 9. Taking into account all the above factors, should the subproject be financed and under what conditions? a. Risks. What are the key subproject risks and what has been or should be done to minimize them? After these actions, to what extent are the risks withim manageable levels? b. Prioty. To what extent is the subproject a priority, sound and implementable investment for scarce national resources? C. Conditions. On what conditions should be subproject be ec d for financing? F. PRoPOSAL RgEVw PRoCURE 10. The proposal review process would be impartal, fair, and accurate. NTF management would ensure that reviewers selected have no conflict of interest with subprojects being evaluated. 11. External reviewers would present their written comments on a standard form which would reflect the general crteria outlined in the previous sections (A-D) as they apply to the specific subproject. External reviewers would be asked to present any additional comments beyond those preseuted in the form, and list the positive and negative characteristics of each subproject. It would not be the role of the reviewers to recommend, classify or rate any individual subprojects. 12. When reviewers meet as a panel, a single appraisal form would be submitted to reflect the consensus of the group. Individual reviewers should be encouaged to present additional or dissenting views, if they consider it important. 13. Assessment of financial capacity (when applicable) and analysis of costs would be conducted directly by the NTF staff. G. SCORING PROCEDURE 14. Scoring procedures would be the same for decisions submitted to the Intenal Evaluation Committee (EC) or to the Board. Two general situations would arise, depending on the type of subproject. Page 128 Annex 9. Subprojec Appraisal Critena 15. Type A Subproject or NTF-ln&ed Subprojeds. These are subprojects in which the NTF would make the detailed specifications, and institutions would bid on a competitive basis. Over 80% of the subprojects in the first phase would be in this category. For some subprojects, or elements within subprojects, the NTF would have developed basic cost parameters. Subprojects would be competing with each other, and typically only one proposal would be chosen. If there is more than one proposal, a rating would be required. These proposals would be decided upon by the Internal Evaluation Committee, unless their value is over US$100,000, in which case the IEC decision would need to be reviewed and ratified by the Board. 16. Type B Subproject or User-Iniwated Subprojects. These are subprojects in which the NTF would present basic criteria for financing and individual finms or consortia would present requests for funding, according to previously established guidelines. Cost parameters would be known to the NTF. Subprojects would be evaluated on their own merits, and typically several subprojects may be chosen. If there are enough resources at a given evaluation session, all subprojects may be approved if they meet NTF requirements. If resources are limited, a scoring of individual proposals would be required. These proposals would be decided by the Board of Directors, regardless of the amount requested. 17. The decision on individual subprojects would consider the following preliminary steps: a. All proposal submitted to external reviewers would be submitted to the respective decision body. b. For each proposal, a detailed price analysis would be undertaken by the Foundation staff. Unit costs should be compatible with the Foundation's parameters. Total subproject costs should also not exceed the Foundation's maximum allocation for that specific set of proposals. Subprojects which exceed the maximum amount of resources available or which exceed the Foundation amount of resources available or which exceed the Foundation parameters would be rejected. 18. Decisions would take into consideration an overall quality evaluation. In the cases in which there is no competition among individual proposals (type B) and when resources are sufficient, there is no need for a scoring, and the Board would only approve or reject a subproject. The judgment would take into consideration the comments of the external reviewers and the overall evaluation of the decision-making body. Reasons for rejection should be stated, as they would be communicated to the proponents. 19. When proposals are competing with each other and only one proposal would be financed, (type A subprojects) an overall quality score would be required. Decision-makers would give their individual overall ratings on a scale based on the same criteria used by the reviewers, and reflected in the standard evaluation form. This would be a single score, based on their overall judgement of the quality of the subproject. The subproject with the highest mean score should be approved. 20. When proposals are not competng with each other, but resources are limited, the decision-making body would first rate individual subprojects as above. The final decision on which subprojects to approve would take into consideration other aspects beyond individual scores, such as a hierarchy of priorities, cost considerations, number of subprojects to be approved, as defined by the Board on the basis of the lot of proposals being considered. TIlI RUSSAN FEDERATION MANAGEMENT AND FNANCIAL TRAING PRtoECr ANNEx 10. PROJEc ) LEvEl OF CoSr RECovERY A. GROSS REPAYMENT POTETA BY SUPROGRAMt Swiss Grtat Total Foreign Rpaynme Loax Amunt AmouNt Conmirifion Repaymen Poteal (UmS Milions) (US$ illons) (US$ mons) Potential (%) (USS mos) 1. Mangemet Development a. Policy Framework 0.5 0.5 0 0.0 b. Etrpris Based Managenent 4.6 4.6 30 1.4 c. Learing Matras 4.1 4.1 10 0.4 d. Management Consulting 1.1 1.1 40 0.4 e. Networking 2. Financial Sector 6 w _ _ _ 8 2A _ L._ a. Training Maeas 2.0 0.2 2.2 10 0.2 b. Infrastuur 1.9 0.4 2.3 10 0.2 c. Training Practitioners 4.5 0.5 5.0 40 2.0 d. Trainig Facuty 1.0 1.0 0 0.0 e. Dedoping Curicula 1.1 1.1 0 0.0 f. fanking Accountacy 3.2 3.2 S0 1.6 g. Banking Operation 12 -_J_s__ 3. Public Finmce 3.7 3.7 0 0.0 4. Phawe 2 Pilot Proects 1.8 1.8 10 0.2 S. National Traning FoPdatim 4.2 4.2 S 0.2 Contugencies 2.0 2.0 10 0.2 TOTAL 40.0 1.1 41.1 18.5 7.6 tFues iag y nyot tota due to rounding. Page 130 An= f1. Proeced L.w& of Coil Recovey IL Pum -U"SS,000000LOW, -nI 1. Wa Usam N. v .sa l. Rat. Rh'_ M~~~ lady - -~~~~Ud -: - - - -- Pt.cddt T _ii TIM0 510 51.125.000 6. Stu2 S1112.913 4.815 U9S 5475343 SmarEauds t_vmu 300 S22W S67.500 W51 Sl91 ia769 .s S4t4 $StJ19 ReiewCtc, COOuN SA $10 5200.000 4.0 134 1442M0 3.250 S16 584.500 COMPAqdbwedCam Cord. 1.000 530 130.000 850 VA6 521,67 450 S20 $12.675 Tcaoceb2 7.5110 S10 S5000 6.3 Sp9 M5411S 4A75 S7 511.662 Can so 7.0 $7 $s5O0 6. 56 S37.931 4.67$ $5 S.SI6 CwM Ram.-A.u.aag. A.*i,fg..Mi m Slm. 51.119 I 56s5.lfs675 _awr_bf chjatUsgE u Talus 250 S1.100 t 2.750.000 2.125 S935 W1K6S 7 $715 S1.161.875 Mw4Id Zampu T1W _ 3.000 ISO S450.000 2. 5128 5325.125 1.950 S9S S190.125 kms. But - D bmm1Ru Taju_ 3.000 S10 S300.=0 2.0 SW S216.750 t.950 s65 $126.750 SmIckAMOMh Ta1u 6.OD S50 S3.000 5.100 S13 S216.75D 3900 S33 S126750 Bak Moos T _l SA0N S50 $250.00 4.210 $1S4 10625 320 S3u S105.6U Gus huue*UaIgsu v f SCe*sg12.50.125 $1.711.125 ohm &t mus.2bs1h,.a d&a $5A00.- -S4.06.1 00 52,366.0 m_ uD .wlud TDd I10w0 117s5 11.925n000 9.350 5149 W 11390.13 7,150 $114 S813313 S m du S% It=0 m7 S 150,00D 80 S64 SS41,S75 6M00 S49 5316M7 ?aft rams iqu FMs 1000 5275 S275.000 5 S234 $196,6 650 S179 S116.1SS aIoSVos Sa 7.0 S275 S1I.925.000 5.90 S4 S1390,83 44M0 S179 SS13.313 Ta-IDICU.VA- Tar_s 750 50 5450.00 638 SS0 53.125 488 $390 S190.125 i%wRomw-MwwmMDnd*nW 5325.00 53.84313 3232tS13 s Rem mdw Admf. e2P1o ePtm S225.000 S191L250 $146250 NmslTbkftag11 S2S5.810 S2.99 S185.77m P Ws S260.000 S221.0 S169.O0 w wm r @s770.S10 655189 S501S.27 ?TAL&3WDVU0OUtIIONE 1; 5. A120818 01 AN 540.80.00 "M0 AW00 140.80.80 RWIUN3ONWANR"Bt 291.% 21.37% 12.79% 'Figures may nsa toW due to rou g. Am lQb. hQJECEd Uvk Of CON ReVoY paoe 131 Thu Two-USS8,w,ooo Lon' IWUUMAWRA4S_U) 3UW4U) dd-S- - - - - P,a00dcum T,v1 17.000 5300 $3,00.000 14.410 S21 S3.684.7$0 31.050 St"f 52.134.750 Sg1Exi Rt dv C " x2o S1 4 530.000 170 S 5130.00 130 55 576.0o R*w CM. Capl 200 551 s57.00D 3.100 57 5122.825 100 Ws5 57,82 C _inpuZar.bueS Cow= C9 1,000 530 53.0W0 on0 26 521,A5 65d 10 5)2.6S T_x6cob eab 17.0 510 5170.000 14,450 59 5t22 11.050 57 571t Can. Sas 1.N0 5t1 5170.000 14.10 59 5122,S 1,00 57 571.2 Gaa &seAagabAa*Ifm _ ad5m 5.820.000 54*15 52,850- cdutEs.n Tam 4.000 S.500 56W. I0O 3.400 S.275 A35N000 2,00 S975 S2WAII0 Ml4.l3 maum5Om Tulm 7.0 $25 S.1 0 50 SW1 SIA 4,510 St 573 X7S hWr* 3uk .ll 33g 0Rh f _ S.00 5300 Sl.S00A0 4.210 S231 S.83710 3.210 s59 S63.750 s=k*Acaft T.Iu 4000 StOO MA00N00 3.4a0 s85 SU9A00 2,600 5W5 SW6=,0 aku = t u 3.000 5Ws5 S225.00 2.0 S6u S213 90. 549 S9WA6G Gas, RuaeBab wba 9875.000 S7,t3.63 54.372.18 - ;binDwdqmm ?.mt_ 8A0 S200 SI30. 6.800 5170 Sl,31X0 53200 S130 566.D0 SeacttMg sm 8.0SIIIIO,C00 5110 SU0.0 6.80 594 S558O0 5350 sn S5i0 P31.yFuinwwkudQ y FePN 5D Soo S0.000 425 568 WM900 325 512 5s160o hl.inehaaServI.. sm 8.I00 $41 S360.N00 6500 538 5260.10O 520D S29 S5112100 ?aiaheC.miiaau Twba. s 5100 0.000SID 425 595f S2.7 325 545 5 147.811 G'vn&waeMaqumDw ir 53.230.00 52.333,671 51364.675 -- Sudnm~~~~~~~~~~~~~~~~~A S.A SawUp-Ttalnhng 7.500 5100 S750.00 6,375 so Su41 4,875 56 S316 Su-poD.L $I500 51 S112.00 6375 513 531.21 * 4,875 510 S5733 &aa4p-T.K 50.000 0 50 S0 0 SO SO _W*qm w . 1400 S5 S210.00 100 So Sw.725 9.100 S30 51.2 zumv= =3aD.L 14.000 15 $210000 11.90 513 S 5175.5 9.100 t10 S8.725 _aTpas-?.M. M50A0 0 So SO 0 SO SO A.dsimySawu 150 5460 567.10 125 535 548.169 #4 525 528,59 aIut _a-Sowasb S1,440.0 S95.375 V57"5 Tn1 40 5200 S900W0I 3.8, s51o S56020 2.925 S130 V380W2=0 T?wuyOOias 3* S120 M10.N IZ25 510 S13000 975 Vs 57650 Staff lm 2000 51S M0I 11,000 513 S2t6.5 13.000 S10 S126.150 o 4wn _ea- A s SJtk Smmo SF.W.0 5997.050 5 10 1_ Sa 8.000 S3 S24A. 6.800 S3 517.30 5.200 S2 S10.140 s _wmi6sm3w 55W S5,9 S250 425 43 S53,s63 325 533 510,56 *wa&smu* .MiaIau.UiNSW* 549.00 513.403 S20.703 TOTAL RIVUNJ Of ALL ' CT57 3800 m9.6 AMMMOII LOWMEN.3A 2 s55.0000 M55.0A000 558,000t0 R3ff O N LOAN .1- M32 37256% 27.04% 13.81s R3M3t ON TOMAL LOAN -15A83 1 2 34.17% 24.72% 134% 'Rwaus I * lei b ad.. abyIdaaq iU U* adI _ -lb dWap b 65% eAt0u3 OftM tdl a "Ten_Is ..1 _fa ~ isu m THED RUSM4 FIDAITON MANAGMENT AND FINANaAL TRAINNG PROncr ANNEX 11. NIF CASM FLwW PtojWrONSo ..puaIh. ?a.I 2 1w) wd S yw6 .YWl ..,6 iwO I .ul . ,Lam __ us tDaVl ,I-MA I ah6wu.ustt ) I.0. _ t . ANIedulmm .301 t2M40D 2.4 6.D 9.400 9w 2 utJ' Fa -I 2A7.400 37.AM 9370 028.4010 T w$7. 000* 9M.0 90.750 t 36.0l S0l5 0*t 901s 74 a2 54.4S3 4.32 3,2 _tf*iovmp 7.607.500 740.750 0,71.125 1.561.30 t.171.t2S 70WM,73 .525 4 450 MMM 31005 392.t5 C _AvaS wS* 2f2t00.0 200M 250.00 ".0 30 3WMD 7.00 200.0 00D MM1m m _ b l.ua6 faIC _*V"eGy 2PiD.00 mm3 06.00 300 400 30.4C 2D.00 10.00 0 0 0 1... an No Cut itm c 1.4114 35. VK74.73 32U4.114 32C0 20.0 It7D0 14. 43 saw' bmhvmmtca Pa40cw 7A400 0.k40 S.00 0. m.m00 sR0D 30.0D 3D.00C Am ado0 am=0 I 0w6.oIineaSo=mPFIuo6 Loom n im6 (OS) 236759 037.22 41.00 5700 I) CAmI LOW 9304 CtERAflflG AC? wino 1430)40 3404719 3 003 223 291070 I 649762 0 0020710 0090DO 616I490 707 03 todi WH OIJDt CaM m:T F _ *u . 0moCm: OpevU abs oduclt,. t 7Ffd4 phrn I- hb_ Op. sddoh.d NI? .hpO lw boe £ I Te*AdcAbmA g do A Ih &mwotowe. sh e ?ebrn*i0rn 1.1 001%4,456va b ane to de K Tb pbm I.£ P. C. IS Cam DI C 2c 24 _d __ __O__ _ a_ed to_ d a_ d_ _ __ . _Is_ ,4 M 2 __________________________ boj th42 116ds4h _ 1 u p 05 *p _. 27Mup.t Dc -5pe Sot. 00.00D 211.CtD 200.00 £3.00 £3.0 801.0 00.0,02 Fba.SadSeix 1.206, 00D 0 25A. 70.00 70.0 00 1M.00 50.002 29 ,t P _: 3 .00 20.000 30,0 05.0 05.00 1 to.0 0. 3 3 22(S vWWLWsVmOoA4M 2L000 4 001 OO 42000 250 2500 20000 )00KOD 321 33 3 tS e0iElqm m OIedP dtLo 3,4W00 57.40 265.400 5A0.40 617.408 511.4D 47.40 367.40 367.40 MA9 S9.403 _0 Iumwszly... U4f*dMMLmaftm 780 S00W 21.00 C.M03 3o AWT Qm2~ruP,Nsu _ ......2.0A , 0 11.40 1 _ S7t4 471F Mm01 CM =59 _ ,,4,j 31 39 39~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~3 4 Oc WPWOWFRaOAERAITNOAcM'tT5 , Wk41X 20A0 MA3.401 - 6 b 2 1 j0.4, 7-40 --*- f j0,37* 40 4£ 49~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~4 42PSQ4M ?DFO 1. OW MGr ACIl SIIS $.P206.111 2.156.963 1.%3 SU3 M5.125 2", MUG54 .,,j ..... 2 42 M4 C^NErCASBFIO PW US ROI 0N0 AC V - 1.206.370 UM 3.100.913 OI3 7_.891.AM t7 7.6.1it7 " "ma 44 46 PA _ _ MIt - - - -o ittllC _ 2,F -Sz -_ 40 4 CWILAIUI? PAOAt _… 4.D…W f = &wD IL 47 3 tLM cu QMc FWW fM OTtB An= rA AC- t,Z&3t4r 2.9S9t It§.93 If3M .t9t33 35745 . 112ti 9 2D2,454 "M3 _rA Page 134 Amnex 1Q. NwT cash nlow Projecions ASsUMPwONS AND EXmATIoNs UNDERLYNG THE PA nATION OF TIE STATEMENr OF CASH FLOWS FROM OPERATION Line 2 Applcon Fees. The application fees is based on an estimated number of applications as outlined in the NTF project report dated October 1993. The number of applications were reduced to reflect the activities of phase one of the project. Line 3 Users' Fees. Users' fees is estimated to be $179,760 and is distributed over the life cycle of phase one based on Table 6.7 'Estimated Loan Disbursement Schedule." Lines 4, 5 Revenues from Tining Materials, Training rogtams, and Consulting Services. Sale and 6 of produced items, fees from training programs, and consulting services for all other activities of phase one are based on the estimates provided by the negotiating team during negotiations. The revenue covers the life cycle of phase one and assumes continued performance of traming activities in years 5 to 10. It is also assumed there will be a demand on these activities during the entire period. Line 9 Interest Income on Net Cash From Operahon. Interest is calculated at the rate of 8% on net cash flow from operations of the previous year plus 8% on the average net cash flow from operating activities in the current year (Line 44). Line 10 Interest Income on Paid-In-Capital. Interest is calculated at the rate of 8% on cash contributions from 15 NTF founders at the rate of $2000 each and on funds received from private enterprises at the rate of $10,000 each (35 members in year 1, 32 members in year 2, and 30 members in year 3). Line 11 Interest inome on Excess of Al Proceeds from iton-Operating Activities Over All Disbursements. Inte is calculated at the rate of 8% on the access of the average of all proceeds from financing activities over the average of all disbursements on non- operating activities. Line 15 Cash Outfows - Foundatin Operations. As explained in the body of the Annex, the cost of running the Central and Regional Foundations in year 1 to 4 is assumed to be included in the cost of setting up the foundations. The cost of operating the foundations in year 5 to 10 are assumed to be included in the additional investments for phase two of the training project. Line 32 Cash Flow - Subprograms - Operatons. This is the total cost of running the training activities from year 5 to year 10. The cost of conducting training activities for the first four years is part of the initial investment in setting up the subprograms as explained in the body of the Annex. Ana 11. NTF Cash Flow Projealons Page 135 Line 3S Interest Expense on Disbursed Porion of the Loan. Interest expense is calculated at the rate of 8% on the accumnulated disbursement portion of the loan as of the beginning of the year plus 8% on the average disbursement of the loan for the current year decreased by the payback amounts starting in year 5. Line 36 Interest Expense on Undisbursed Loan Balance. Interest expense is calculated at the rate of 3/4 of 1% on the balance of the undisbursed portion of the loan at the end of each year within the life cycle of the loan. Line 40 Cash Outlow from Operating Activities. This is the total of row 32 and 38. Line 42 Net Cash Flow from Operating Actvides. This is row 13 minus row 40. THE RUSSUIN FEDERATION MANAGEMENT AND FiNANCIAL TRAINIG PROJECT ANNEX 12. DETAILD BASELI N RoJMr COSINGt (US$ million) MANAGEMENT DEVELOPMh I TEIE FOANCAL SECrOR ~~~I I TOT~~~~~~IAL PBDOJW PUU FILOT U 9 SE2 MAtINGL CosR" Cv4ai Cods Technucat Assinc F&orign Expens 0.7 1.7 13 0.4 0.1 0.4 0.1 OA 0.1 0.3 0.4 0.4 0.7 0.3 0.6 7.8 Local Bxpers 0.1 1.2 1.0 0.2 0.2 1.4 0.02 1.2 0.03 OA 0.4 0.4 0.S 0.4 2.4 9.9 Overeas Fetowships 0.07 1.4 0.3 0.2 0.07 0.5 0.3 0.6 0.6 0.3 0.4 0.4 0.6 0.3 0.2 6.1 SnbwsaredakasaW ce 09 4.3 2.6 0.8 0.4 2.3 03 2.2 0.7 1.0 1.2 t.2 1.8 1.0 3.2 23.7 Bquipnt 0.0 03 0.4 0.1 0.3 0.1 1.8 0.7 0.0 0.1 O.S OS 0.2 0.2 0.5 6.1 P"rductionandDlstibution 0.1 0.1 0.7 0.1 0.4 0.0 0.0 0.2 0.01 0.0 0.1 0.1 0.2 0.04 0.2 2.3 Tran Deivery 0.1 0.9 t.3 03 0.3 0.4 0.7 3.1 0.5 0.4 1.8 1.8 1.8 1.1 1.2 15.6 TOWe fMm Cogs 1.1 5.6 5.0 1.3 1.4 2.8 2.8 62 1.2 1.4 3.6 3.6 4.5 2.3 5.1 47.7 Rewo Cosb operadog CosIs 0.10 0.10 O.to 0.03 0!03 0.04 0.04 0.10 0.03 0.04 0.10 0.10 0.10 0.05 0 121 Tad dlsweucosts 0.10 0.10 0.10 0.03 0.03 0.04 0.04 0.10 0.03 0.04 0.10 0.10 0.10 0.05 0.25 I.2 TAL BASEL ECOMS I.2 S.7 5.1 1.37 1.A 2.8 2.6 6.3 1.3 A.4 3.7 3.? 4. 23 5.3 49.0 NW. C _dqmieu 2.40 TOtAL MI COWS SIA Fpt= my toW do to ro4-h. THE RUSSIAN FEDERATION MANAGEMENT AND FINANCIL TRAINIG PROJECr ANNEX 13. SUMMARY OF TECNCAL AsSisrANcE INPUTS Naeonal lTwining Foundaion 1. Technical assistance would be provided to support project implementation. It includes both technical assistance and training to foundation staff, managers, ad hoc experts and board members as well as training and technical assistance to potential proposers. Training would be offered in project preparation, appraisal and evaluation. Ttaining and technical assistance would also be offered in the area of altemative delivery media, particularly individualized instruction, computer-based and other media- based distance learning approaches. In addition, direct consulting and advice services would be provided to the staff of the central Regional Foundations. Even though essential, technical assistance needs to be commensurate with the local absorption capacity. It would also need to be designed in a way to avoid advisors being involved in executive matters or providing conflicting advice. The current estimate follows. srTMATED TECNCAL ASSrSANE REQUEErMs (NTh Staff/Month* Total (US$) General Advisor 24 480,000 Finance/Procurement Advisor 06 120,000 plementation/Training Advisor 09 180,000 Other Advisors 12 240,000 Local Advisors 48 134,400 Total 1,154,400 * All advisors costs estimated at US$20,000 per staff/month including travel, local travel and per diem when applicable. Local advisors costs estimated at US$2,800 per month. Notes following this section present details on the technical annex presents details on the technical assistance component including an addition US$600,000 for technical support for distance education activities to be undertakn under the various components. 2. The technical assistance component includes training and other direct advisory services to the Foundation and its clients. It involves external and local consultants working directly with the staff of the central and Regional Foundations, as well as training and other technical support activities in the area of distance education. Page 140 Annex 13. Summary of Technical Assistance Inputs 3. The consulting services would be provided as follows: a. A general advisor, whose major task would be to advise on the general strategies of the Foundation and to provide general managerial advice to the Executive Director of the Central and Regional Foundations. The senior advisor would be hired for a period of 24 months, including 18 consecutive months and the remaining 6 months being phased out progressively. The senior advisor would also help coordinate the activities of all other external advisors. He would report to the Executive Director and submit quarterly reports to the Board of Directors and the World Bank. b. The finance and procurement advisor would advise the Administration and Fiance Manager on matters pertaining to finance and procurement. He would also be involved in training staff and clients of the foundation on project implementation issues. He would be hired for a period of 6 months and report directly to the Administration and Finance Manager. c. An implementation and training advisor, to be hired before the actual installation of the NTF, and with two major functions. The first would be to advise the Executive Director on all implementation details, help prepare routines, manuals, procedures and forms. The second function would be to organize the initial training activities for the staff and clients, including detailed training programs and materials, training delivery, preparing terms of reference for production of tramiing materials and for hiring training and participating in the supervision and delivery of the initial training activities. The implementation and training advisor would be hired for a period of nine months, starting as soon as possible. d. Three foreign program advisors, one in management training, one in accounting/finance training, and a third one in bank training to advise the respective program managers of the central and Regional Foundations all aspects related to planning and implementation of the respective subcomponents. The advisor on management would be hired for 30 months, the advisor for accounting and finance for a period of 12 months, and the advisor on banking for 18 months. All three would report to the general advisor. e. Two national assistant program experts, who would work closely with the three foreign program advisors, with the purpose c. assisting the foreign program advisors in technical, institutional matters, facilitating communication with local staff and other experts, while at the same time developing their own consulting skills. These experts would be expect to continue providing advice to the Foundation after the foreign program advisors finish their contracts. They would be hired for 24 months each and would report directly to the respective technical advisors or the general advisor. f. Other advisors. Up to 12 months of foreign assistance has been retained for hiring advisors on an as-needed basis. It is expected that the Central Foundation would need advisory services in areas such as fund-raising and for establishing its own clearinghouse. 'p i~~~~~~~~; I lii- p E~~~~~~~~~~~~~~~~~~~~ }~~~l j }lii iii fuj i ii wSS Jl I fii,' iin j III I l 111111!Rb Eti a@ 2*§0 " 5 t ~~~i3 :I |e R | | I R| ThE RussmAN FEDERATION MANAGEMENT AND FINANCIAL TRANNG PROJECT ANNMx 14. NATIONAL TRAINING FOUNDATION STAFF DEVELOPMENT PLANS 1. All NTP staff would receive training in overall NTP goals, policies and procedures as well as in NTP evaluation concepts and methods, before and during their several NTF jobs. All NTF staff would also receive general orientation to their specific roles and responsibilities. This training would be organized by the NTF director. New NTF directors would receive training from World Bank concerning its policies and procedures, and in general exemplary foundation and training practices during overseas visits. 2. Training for program officers and other professional roles (such as financial analysts) would be organized by the NTF director, and would include visits to exemplary foundations, visits to and training from World Bank professionals, local training workshops, and on-site technical assistance from foreign and local experts as needed to effectively carry out their roles and responsibilities. 3. Several related staff development efforts are necessary in order to support NTF evaluation operations. These are summrized in the chart below (NTF Staff Development Summary), then described in more detail in the following sections. Training recipients are listed separately only to clarify their training needs. They may be grouped during training with other training recipients to add efficiency and improve cross-functional awareness and cooperation. NTF Evaluation Taning Summar 7hulnee Growp Topics Metods Potenta Proposers * NTF goals and mission * self instructional package (audio * program/project goals tape, manual, annotated * proposal procedures proposal example) * proposal requirements and * regional workshops by NTF rating criteria staff (design techniques) Winning Proposers * project design * initial regional workshop * project evaluation philosophy, (design & start-up) procedures, requirements * manual, job aids * project management * regional workshop (data * project accountability collection, reports, evaluation procedures usage) Page 144 Annex 14. NTF Staff Developmem Plan NTF Evuation Tmining Summary (Continued) Thanee Group Topics Methods N Proposal Review * proposal criteria * workshop & simulation Panel Members * review procedures * manual & job-aids * NTF evaluation approach * rating sununaries & analysis * program area needs and goals data N7 Program Officers * NTP evaluation approach * initial workshop * evaluation fundamentals * follow-up seminars * site visit procedures * manual & job-aids * monitoring and reporting * video/audio tapes procedures * visits abroad * how to manage the review panels and process NTF Staff * NTF evaluation approach * workshop * site visit procedures * manual * exemplary foundation operations * visits abroad NTF Board * exemplary training approaches * workshop seminar - NTF evaluation approach * visits abroad - NTF goals and mission * fundamentals of foundation leadership AdWsors * NTP goals and mission * workshop/seminar * NTF evaluation approach * train-the-trainer * innovative methods and research * briefings by foreign and local in program areas experts * needs data in NTF region * how to be a project manager w.ikshop Subject Group Evaluation * NTF evaluation approach * initial workshop Teams * subject evaluation procedures * subject seminars * subject-specific evaluation issues * manuals & job-aids Subproject Stff Involved * NTF evaluation approach * initial workshop in Subject Evaluation * subject evaluation procedures * subject seminars Projects Potential Subject * NTF evaluation approach * NTF sponsored seminars and Evaluators and Other * general evaluation concepts workshops Evaluation Professionals * exemplary evaluation practices Anne 14. NiF Staff Development Plans Page 145 P1tentla Proposers 4. Brief training would be provided to potential proposers so that they can construct complete and responsive proposals that include a reasonable evaluation overview. Subsequent, more in depth and detailed training would then be provided to winning proposal project staff, to enable them to design complete and fully effective project management and evaluation plans for their proposed project after it has been revised and fully approved. 5. Initial training to potential proposers is best provided through self-learning packages, to reduce NTF costs, and to make training readily available and more convenient to the broadest possible population of proposers. A "hotline" could be employed in addition so that proposers with questions can quickly resolve their concerns with a telephone call. 6. The NTF would make available (on request) a self instructional package to all potential proposers. This package would be as brief and as simple as possible, and should include (i) an introductory video (or audio) tape overviewing NTF goals, mission, and procedures, (ii) a listing and explanation of proposal review criteria, (iii) a flow-chart of tbe proposal preparation and funding cycle, and (iv) an exemplary proposal annotated to demonstrate the criteria. In addition, this package would include one or more annexes-to be requested by the proposer-explaining the goals and objectives, and any special proposal review criteria, for the program(s) or project(s) in which they are interested on competing. Wn7ning Ptoposers (new subproject managers) 7. Funded project staff would receive training in three stages. a. The project manager and an assistant responsible for evaluation would attend an initial 3-day workshop in their region where they would learn in detail about NTF project design, evaluation, management, and accounting procedures. At this workshop, participants not only learn how to design evaluation, but actually produce evaluation design materials during the training, so as to maximize feedback from NTF training leaders (preferably program officers) and commitment to plans. These workshops would be offered regionally, and should be organized and scheduled (if possible) such that like-project types are trained together. Participants would fund their travel and other attendance costs from their project loans or awards, but other than this, there would be no charge to receive the training. Attendance at such workshops is not mandatory, but would be recommended, as it clearly would help projects be more successful, and thus improve their chances of receiving renewed funding. Trainers for these workshops would be selected from among NTF advisers and program officers, who would first have been trained by foreign experts. b. The project staff would receive (at the first workshop) a manual and pertinent job-aids explaining and demonstrating in detail the NTF evaluation approach, design criteria, report formats, and so forth. The manual would include annotated examples of all documents, checklists for instrumentation and data collection criteria, and other materials supporting exemplary evaluation practices. This ^Age 146 Aner 14. NTF Staff Developmen Plas manual would serve as a detailed reference guide to NTP sub-project managers and staff for project evaluation. C. A second 2-day workshop would be conducted for project staff to give them the skills and guided practice needed to support their implemenation and usage of effective evaluation. This follow-up would be timed to allow project staff to have gained some experience with their projects, yet not too late so as to reduce their opportunity to redirect the project and its evaluation; about 1-2 years after project start-up would be workable for most projects. Topics addressed in this session would include instrumentation and data collection, data analysis, interpretation, reporting, and using evaluation information to improve project effectiveness. N17 Pioseal Rvew Pand Members 8. Proposal review panelists would attend a 2-day workshop intended to provide them with the skills needed to reliably and consistently reach effective decisions on proposal approvals. This training would include: a. review and discussion of NTF goals and strategies b. review and discussion of the review guidelines and criteria c. simulations, in which assessors evaluate, then discuss and compare results, for mock proposals (training of this type is crucial to developing inter-rater reliability) 9. Following the workshop, reviewers would receive a "Reviewer's Mamnal" that summarizes the training content, explains the criteria, and contains worked examples of each review docm t (rting forms and checkists). An appendix to the manual would include 2 mock proposals with ascid review forms completed. This appendix would be annotated to explain the rationale for the rating decisions reflected in the mock proposal review forms. This manual is intended for self-study prior to proposal review panel meeting, and should be used again by each reviewer as a guide during the proposal rating review process. Amiexes to these manuals would be customized for particular NTF program and project areas, as prepared by special working groups constituted of NTF program officers and advisers. 10. In addition to the workshop and manual, the assessors should receive reports to reflect and _mmarive their individual rating performance analyzed both over time from the same rater, and as compared to other raters. These reports should be followed by facilitated small group discussion of trends, issues, and problems. This sort of training is crucial to developing intra-rater reliability. NTFPigem Officer 11. Program officers will serve as the primary ongoing contact with sub-projects after they have been funded. As such, program officers need to understand NTF evaluation approaches and procedures, and must be able to assess whether subprojects are proceeding effectively. Program officers would receive the following training: An=ex 14. NfTf Staff Developmen Plans Page 147 a. An initial three-day workshop in the NTF evaluation approach and essendal evaluation concepts, principles, and methods. b. visits to overseas training foundations c. follow-up seminars in special evaluation topics (such as interview skills) as needs and NTF experience indicate d. A 2-day workshop in site visitation procedures and report formnas. e. A manual and associated job-aids, checklists, etc., that were covered in the workshops to serve as a reference guide to program officers when they make site visits. This manual would include exemplary (annotated) reports and other NIF forms. Subject Goup Ealuaos 12. Subject evaluators are chosen because they have established evaluation expertise. Thus they need only training in NTF approaches in general and subject-evaluation purposes in particular, plus training in the specific issues and concerns that drive the subject evaluation in which they would be involved. This training includes: a. an initial 'qualifying' 1-day workshop in NTF evaluation principles b. a 2-day subject-specific evaluation preparation seminar to be conducted prior to each subject evaluation effort. c. a subject-evaluators guidebook that summarizes workshop content and contains exemplary, annotated reports and documents, deists, and other matrials needed for effective subject evaluation. SUProject Staff Involved in Subject Evauaton 13. Selected project staff would attend a. the 2-day subject-specific session anended by subject evaluators, and b. a 3-day follow-up workshop led by the subject evaluator team to discuss evaluation findings, interpret results, and formulate conlusions and recommendations drawing from the subject evaluation. Reports would be prepared for two primary audiences: i. NTF leadership, with implications for NTF policy; these are privat reports to the NTF only. ii. training leaders in the same field as the subject projept, wih implications for professional practice. These are public reports for broad dissemination, and may be published as books or monographs. Page 148 Annex 14. N7T7 Staff Development Plans NTF Professional and Selected Sppoat Stiff 14. All NTF staff need awareness and understanding of NTF evaluation policies, philosophy, and procedures. This would be imparted at a regularly scheduled 1-day workshop. Potendtal Sjet Evaluators and Other Evaluaton raing Leaders 15. The NTF should engage in a progranmnatic effort to develop evaluation expertise and resources in parallel with, and as a part of, NTF supported projects. The purpose of this training is to develop a pool of evaluation expertise and resource persons whose services could be used to support NTF evaluations. These evaluation resource persons would design and conduct subject level evaluations, provide training to funded project staff, and provide other training and technical assistance as deemed necessary by NTF to support the development of effective evaluation within NTF, among funded projects, and throughout the Russian training profession in general. 16. This training would most often take the form of workshops and seminars, but might also include higher level symposia, targeted research and dissemination, and publication of manuals, case studies, and so forth. NTF Board 17. flie NTF board should receive a several workshops and seminars as needed to thoroughly acquaint them with NTF goals, strategy, policies, and procedures. They should also receive special training to make them aware of the NTF evaluation approach, and of the most exemplary training models and approaches available. This latter topic could be best achieved from overseas visits to exemplary training institutions. Advisers 18. Advisers should attend an initial workshop to thoroughly orient them to NTF evaluation approaches, goals, strategies, and needs. Selected advisers with special training skills and interests would be selected to become trained as workshop leaders for subproject nanager training (as explained above). This training of advisers as trainers would be conducted by foreign experts, who would design and conduct, as a pilot and as a train-the-trainer session, one or more of the initial subproject managers' workshops. THE RUSSIAN FEDERATION MANAGEMENT AND FINANciAL TRAIIG PRoCr ANNEX 15. SummARY OF PROGRAMS, SUUPROGRAMS, AND SuInwEROS 1. PROGRAM: MANAGEMEN DEVELOPMENT (FAtmated Totl: 56 Subprojects) 1) Subprogram: Policy Framework and Quality Stdards (US$1.2 mlUion) Subprojects (5) - Study of supply/demand - Proposals for change in legislation - Assessment of current quality standards - Design of certification procedures 2) Subprogram Enterprse-based Management Development (US5. 7 mon) Subprojects (25) - Initial seminars on post-privatization management - Enterprise-specific management development programs - Internships in foreign companies - Training guides - Strengthening local training institutions/finns 3) Subprogramn Learning Materials, Trainers, Teachers (US$5.1 mon) Subprojects (17) - Develop 17 sets of materials in priority subjects - Retrain 100 trainers in each subject - Further training of master teachers - Development of distance learning packages 4) Subprogram: Management Consulting (US$1.4 milion) Subprojects (5) - Training of management consultants - Internships in foreign firms - Training managers of consulting firms - Production of professional consulting guides - Advisory services to consulting firms 5) Subprogram. Networldng and Informaton (US$1.4 miion) Subprojects (4) - Assistance for development of voluntary professional association - Development of databases, clearinghouse Page 150 Annex 15. Swnmary of Programt Subprograms, and Subpjecs II. PROGRAM: THE FINANCIAL SECTOR Accounting, Auditing and Finance (Estimad Total: 22 Subprojects) 1) Subprogram: Development of Training Resources (US$2.8 mOilon) Subprojects (7) - Adaptation of computer-based training programs - Adaptation of textbooks - Development of Russian case studies - Training in modem delivery methods 2) Subprogram: Buing Trainng lfrastucure (US$2.8 milon) Subprojects (8) - Creation of eight information technology centers - Training in automated accounting systems - Introduction of computer-based self-teaching - Development of distance teaching programs 3) Subprogram: Training Human Resources (US$6.3 mnlion) Subprojects (12) - Developing trainers - Training practitioners in modem accounting methods - Awareness programs for senior AAF executives 4) Subprogram Developing Faculty (US$1.3 millon) Subprojects (5) - Staff training through modular programs, sandwich courses and distance teaching 5) Subprogram: Revision of Curricula (US$1.4 mon) Subprojects (3) - Development of standards - Development of syllabi and teaching content Banking Tnaning (Dsimated Totl: 23 Subprojects) 1) Subprogra Banking Operations (US$3.7 illon) Subprojects (19) - Training of chief executives in bank management - Training in credit management - Training in international banking - Training in investment banking 2) Banking Accouancy (US$3.7 mon) Subprojects (4) Anw IS. Swnmay of Program, Subprograms, and Subprojects Page 151 IM. PROGRAM: PUBLIC FINANCE (timated Total: 9Subprojects) 1) Subprogram Be and Tax Policy (US$1.6 mWion) Subprojects (3) - Staff training in budget, budget policy - Staff training in taxation policy and tax systems - Training of trainers 2) Subprogrm TuWdng of Tasury Offld s (US$3.0 mUIlon) Subprojects (6) - Staff training in budget execution - Training in application of computerized systems - Training of trainers - Development of distance leaming techniques IV. P}AS 2 PILOT PROJECTS (Estimated Total: 7 Pilot ProJects) (US$ 2.3 millon) 1) Small Biness (2) 2) Pblic Admtbdon (2) 3) Mo*et Econemls (3) Totals: 3 Progrm 14 SubpraS 120 Subprojects 7 Phase 2 Pilot Projects ..... ....... . ... .. ....... . ... 0~~~~~~~~~~~~~~~~~~r re 01~~~~~~~~~~~~~~ ... ....... ..................... .. ..... .. . . ... 0* 0 4X1 ! # -g1 1 ---------------.. .-...... ..... ................. ........-...... .... . E ...................... ....... ... . ...... g .. ..... ........... .............. ...... ............................. ........................... ..... : A t-tm---------.........-. ........ Jb _ a I a~~~~~~~~~1 1I x$ 4.,~ ~~~~Ini nI NI sI I-xit si-i : : }E J_ Z, m~~~~~~~~~~~~~~~~~~~~~m l l ~~~~~0 m § I 3.1t aI1S 'C. 0~~~~~~ ~~~ ~~~ 2 - ~~~~~~ I 1 Page 156 S1nae iI6b, e s,-J''- I' : National Training Foundation Startup schedule QIr 1 I994 Qtr 2, 1994 _Qr J, 1994 I ar 4, 1994 1 Qr 1 . 1995 I atr 2. 1995 01 _ID_ Task Name_ _Ja_________________ Ji Tun Jul]7AugJSep o4J No JanrebM AprLM J F un Jul 1 PROJECT PREPARATION - 2 Min of Labor Working Party. - project preparation 3 Preparatory work on implementaton ptans and protect systems - _ __ ___ _= _= _= _ ____ tech_asIsisance__ ____________ 4 World Bank Slall Appraisal Mission 11] 6 Decide on final composJit of founders * 06 Sep '94 6 Prepare NTF lnternalwownregutlon(1sidrafl) I Founde orfiial meerng 07 Sip'94 a First rmeeDg of the Board of Diretors * 07 Sep 94 9 'ROJECT STARTUP 10 NTF REGISTERED AS LEGAL ENTITY + 29 Sep '94 11 Search tar candidaes tor Executive hd(*or 12 APPOINTMENT OF EXECUTIVE DIRECTOR 07 54p 94 113 Loan negoiation 14 Agreent of Terms of Payback and Special Accounts 21 Oct'94 1t PROJECT PREPARATION FACILITY StGNED Is Executve Diredors slarts wdng 01 Nov '94 17 Intl bW ein of ED by consuant i 11t offce premises 19 ObIn lemporary office for NTF stat team 20 Work of startup team In temporary offices P3ag 1 N~~~~~~~~~~~~~~~.. . ... 4: CD~~~~~~~~~~~~~~~~~~9 c ..... ... ..... .. ..... .. .. .. .= _IL .~~~~~~~ In ~ ~ ~ ~ ~ ~ I .So.~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~5;. I~~~~~~~~~~~~~~~~ -~ ~ ~ ~~~~~~~~~~i (U~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~C -~~~~~~~~~~~~~~~~~~~ 1L.~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~C Iu.c,) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~C ~~~~~~~~~~~~~~~~~L - - -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~F I I I~~~~~~~~~~~~~~~n I C r m tF 5 ff- I r ;, j 1~~~ [ k 'I | I S~~~~~~~~~~~ 17 - --''-'''''"''''"- b i di |e l Th _ _ ttr 'n~~~~~T~j CL .. ........ .. . '4~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~Z Ca~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~C 2* ifi~~~~~~h CL0 CL a. -0 U.... .... .............. ....... . THE RuSsIAN FEDERATION MANAGEMENT AND FINANCIAL TRAINING PROJEcr ANNEX 17. DETAiLED PROCUREMENT PLAN AND SCHEDULE . . Category # Contracts Loan Funds NBF Total ______________________ ____________ (US$xOOO) (US$xOOO) (US$X000) Equlment & MateI_ _ 40 4,300 900 5,200 (ICB) 4 1,500 400 1900 (DC) 3 400 100 500 (IS) 6 600 100 700 (LCB) 5 700 100 800 (LS) 20 800 100 900 (IS - PPF) 2 300 100 400 Subproject Tech. Services 70 24,300 9,900 34,200 Fellowships 15 6,100 0 6,100 RTF Consultant Services 15 2,700 300 3,000 RTF Services (Small Value) SO 1,800 100 1,900 operations Exp. na 200 1,000 .rgwn 'hl ''0 40,000. ''W l1$0 3IE Page 162 Annex 17. Detailed Procurement Plan and Schedule Category of Lot / Estimated Proc. Wor Activities Schedule (Dates) Procuremnent Itet n Nature of Package Cost ($ x 000) Method Issue of Docs. SBidtProp. Recdt Contract Signat tart Date I Issue of Docs. Bid/Prop. Recd. Contract Signat. Begin Delivery Foundation Expertsants . 11/15/94 - 1/10195 3/15/95 - 5/01195 - Staff 1 Program Consultants and Experts 2,700 Reg Short Lis 6/15/97 8/10/97 10/20/97 12105/97 _ I I ~~~~~~~~~~~~~11 0(95 - 3/1 0/95 - 5/05/95 - 6/25/95 2 Staffing and Operations Services 1,800 Reg. Short List 61/1598 8/10/98 10/2098 - 12/05/98 Subproject Consultancies 1110/95 - 3/10/95 - 5/05/95 - 6120/95 - and Fellowships 3 Management Development 12,400 Reg. Slior List 6/15(98 8/10/98 10/20/98 12/05198 4 Financal Sectr 12,500 Reg. Shn Lists 1/10/95 - 3/10/95 - 5(05/95 - 6/20/95 - 4 Financial Sector 12,500 Reg. Short Lists ~~6/15/98 8/10/98 10/20/98 12/05/98 2/20(95 - 4/10/95 - 5/25/95 - 6/25/95 - 5 Public Finance 3,700 Reg. Short Lists 7/1/98 9(10/98 11/20598 12128(98 6 Pilot Projects 1,700 Reg. Short Lists 8/1/98 9531/95 7/25/95 912/5/9 ':i.u. :, . \j., v .',... ''' ' ', E~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~819 9/09 IIl9 12 5E9 Annex 17. Detailed Procurement Plan and Schedule Page 163 Category of Lot Esdimated Proc. Major Acdvities Schedule (Dates) Pro|uremnent i Nature of Package Cost ($xOOO0 Mediod Is s .. __e Do Foundatiot Equip. & . & R Oc u2 10/15/94 - 11/30/94 1/15195 - 2/20/95 - Materials l Cntal& Rgina Ofic.Euit 290 IS 6/15/95 8/10/95 10/20/95 12/05/95 11/20/94 - 1/10)95 - 3/15/95 St520/95 2 Foundation Training Equipment 200 LS 6115/195 U/10195 l020195 120/0595 4I 11595 9 5/2/95 - 7125195 - 9/12959 3 Foundation Training Equipment 250 DC 8/1/95 9/10195 7/1t95 8/15/95 10/15/94 - 11130194 - 1/15t95 2/20/95 - 4 Fboundation Office Operations 900 IS6/15/97 8/10/97 10120197 12105197 Subproject EqApip. and S Accounting Technology Centers 1.500 ICB 1/15/95 6t3/10/95 - 5/15/95 6/25/95 Materia_s 4/15/97 6/20/97 8/20/97 10/5/97 6 Subprojects Equipment I Materials 150 DC 3/15/95 4t20/95 6/20/95 8/5/95 7 Subprojec Equipment / Materials 610 is 12/10/94 1/31/95 - 3/30/95 5/10195 8/15/96 9/30/96 11t25/96 12128196 I ~ ~ ~ ~ ~ 5 09 109 8/25/95 - I10J05/95- 8 Subprojects Equipment / Materials 700 LCB 5/10/95 6/30/95 - 7J95 8/15/97 9 Subprojects Equipment / Materials 600 LS 12/10/94 2/05/95- 4/20/95 5 5/25/95 Subprojects Equipmen / Materials 600 IS9(20/97 j 10/15/9'7 11/10/97 12/15/97 _ 1J1 v ~~~~. :,' . aTr; THE RUSSIAN FEDERATION MANAGEMENT AND FINANCIAL TRA1 IG PROJEcr ANNEX 18. SUPERVISON PLANt PWS-30 sw (of which 20 sw on mission; 5 in headquart review; 5 at resint missio) Mission Dates Nature of Mission Requrd Personnd sw/ea. Toal sw 1. January 1995 Project Launch Task Manager 3 Foundation Specialist 3 Procurement Specialist 3 Disbursemnt Specialist 3 12 sw 2. April 1995 Supervision Task Manager 3 Foundation Specialist 3 Management Traiing Specialist 2 8 Sw FY96-4o sw (of whikh 24 w on mssion; 8 in headpate t0ie; & at *addnt miMis) 3. July 1995 Supervision Task Manager 2 Foundation Speciist 2 Accountancy/Finance Specialist 2 6 sw 4. October 1995 Supervision Foundation Specialist 2 Procurement Specialist 2 Public Finance Specialist 2 6 sw 5. January 1996 Supervision Foundation Specialist 2 Disbursement Specialist 2 Management Training Specialist 2 6 sw 6. April 1996 Supervision Task Manager 2 Foundation Specialist 2 Procurement Specialist 2 6 sw P 7-39 sw (of which 24 sw on mission; 7 in headquars rteview; 8 at resident misin) 7. July 1996 Midterm Review Task Manager 3 Foundation Specialist 3 Management Training Specialist 3 Accountancy/Finance Specialist 3 Public Finance Specialist 2 14 sw 8. December 1996 Supervision Foundation Specialist 2 Procurement Specialist 2 4 sw 9. March 1997 Supervision 3 Specialists 2 sw/ea. 6 sw sw = staff week t Note: Loca Russian specialist would be recruited and trained to work at Moscow Resident Mission to handle day-to-day supervision and procurement issues (8 sw equivalent per annum). THE RUSSIAN FEDERATION MANAGEMENT AND FINANCIAL TRAINNG PROJECI ANNEX 19. DOCUMENTS IN PRoJECr FILES 1. Staff Appraisal Report 2. Project Prospectus 3. "Russian National Training Agency: An Organizational Proposal" submitted to the World Bank by John H. Moore (April 1992). 4. "Russian National Training Agency: Organizational Alternatives" by John H. Moore (April 1992). 5. "Russian National Training Agency: Merit Review-A Decision Mechanism" by John H. Moore (April 1992). 6. "Survey on Business Schools in Russia" submitted to the World Bank by International Business School (September 15, 1992)(In English and in Russian). 7. "Investment in Training for Transition to the Market' submitted to the World Bank by R. Kapelusbnikov, 1. Albegova, T. Leonova, and R. Yemtsov (September 28, 1993) (In English and in Russian). 8. "Structural Economic Reform: Its Impact on the Development of Management Potential" by G. Melikian (1993). 9. "Russian Television and the Market Economy" submitted to the World Bank by Igor Malashenko (May 7, 1993). 10. "Economic Training Books in Russia: Present State" submitted to the World Bank by Boris Bagarlyatsky, Nachala Press (May 21, 1993). 11. "Investment in Training for Transition to the Market" submitted to the World Bank by R. Kapelushnikov, I. Albegova, T. Leonova, and R. Yemtsov (September 1993). 12. "Potential Loan Demand for Management Training in Russia" submitted to the World Bank by International Business School, Moscow (November 1993). 13. "The Role of Employers' Organizations in the transition to a Market Economy in Russia" submitted to the World Bank by Dr. Alexander Posadsky (April 1994). fBRD 25948 RUSSIAN FEDERATION MANAGEMENT AND FINANCIAL TRAINING PROJECT f Central Found-ion aof th^ RTt * Regionat Foundoair, of tie RTF Oblast, kri, or republic boundories Autonomous oblst, okrug, or republic boundo,ies 84 International boundories 8 80 83 St! Petersbuwg 75 79 (10 9 8 2 87 20 6 19 23 4 16 MOS5COW .67 4w1 l23 o 12 14 7 21 1522 Nissny ,.. 76 30 7 18 8°Navgovad2s 56 N 31 28 25 66 2 32 2426 58 55 0 81 86 33 36 40 35 48 41 3 1 1647 78 45 6 049 363 62 69 8 443 7 270 42 60 __200_A_O_60 0 The boundaries. color-so denominations ond any other intonnotion Mrm , ~~~~~~~~~~~~~~~~~~~shrosnr on tt,is map do niot imnply, on the part of The Word Bank OJOOre. any judgmsen' on the Il status of any territory, or any endorsement or acceptance of such boundaries. 1.Arkhangelskoia oblast 1 7. Orlovskaio oblast 33. Voronezhskaia oblost 48. Rostovskoia oblast 63. Omskaia oblest 75. Taimyrskii (Dolgono-NenetskMil 2. Nenetskii autonomous okrug 18. Riazonskaio oblast 34. Astrakhanskoio oblast 49. Stouropolskii krai 64. Tomskoia oblast cutonomous okrug 3. Republic of Karefio 19. Smolenskaia oblast 35. Republic of Kalmykic 50. Korochai-Cherkess Republic 65. Tyumenskaia oblost 76. Evenkiiskii autonomous okrug 4. Republic of Komi 20. Tverskoio oblast 36. Penzenskaia oblost 51. Republic of Bashkortostan 66. Khanty-Mansiiskii 77. Republic of Tuva 5. Murmanskoic obloas 21. Tul'skair oblast 37. Somarskai oblast 52. Cheliabinskaia oblosi autonomous okrug 78. Amurskaio oblost 6. Vologodekoia oblost 22. Vladimirskoio oblost 38. Saratovskoia oblost 53. Kurganskaia oblost 67. Yamal-Nenets 79. Kamchatskaio oblost 7. St. Petersburg 23. laroslovskaic oblast 39. Republic of Tatorstan 54. Orenburgskoio oblost autonomous okrug 80. Koriokii autonomous okrug 8. Leningradskaia oblast 24. Republic of Chuvash 40. Ulianovskoio oblast 55. Permskoia oblast 68. Republic of Buryatiia 81. Khaborovskii krai 9. Novgorogskaia oblost 25. Kirovskaia oblcst 41. Volgogradskaia oblost 56, Komi-Permyotskoiia 69. Chitinskoia oblast 82. Jewish autonomous oblast 10. Pskovskana oblast 26. Republic of Marii-£l 42. Republic of Dogeston autonomous okrug 70. Aginskii Buryatskii 83. tAagadonskaia oblast 11. Brianskala oblast 27. Mordovian Republic 43. Chechen Republic and 57. Sverdlovskoia oblost autonomous okrug 84. Chukotskii autonomous okrug 12. Ivonovskoia oblast 28. Nizhniy Novgorod oblast ingush Republic 58. Udmurt Republic 71. Irkulskaio oblast 85. Primorskii kroi 13. Kaluzhskaia oblast 29. Belgorodskata oblast 44. Kabordino-Bolkar Republic 59. Altoiskti krai 72. Ust'-Ordynskii Buryalskii 86. Sakhtilinskoic oblost 14. Kosdromskoai ablast 30. Kurskoia oblast 45. Krasnodarskii krai 60. Republic of Altoi autonomous okrug 87. Republic of Sokha (Yokutia) 15. Moskovskoia oblast 31. Lipetskaia oblast 46. Republic of Adygeya 61. Kemerovskaia ablost 73. Krasnoiorskii krai 88. Koliningrodskoia obloSt 16. Moscow City 32. Tambovskaia oblost 47. North.Osetien Republic 62. Novosibirskoia oblast 74. Republic of Khakasio 89. Ingush Republic .'INE 1994