The World Bank Georgia Human Capital Program (P175455) Technical Assessment Georgia Human Capital Program-for-Results: Technical Assessment Date of this draft: February 17, 2022 Table of Contents A. Program Description 2 B. Program Strategic Relevance and Technical Soundness 4 Program Strategic Relevance 4 Program Technical Soundness 8 Institutional Arrangements 30 C. Program Expenditure Framework 34 D. Program Results Framework, including DLIs 42 Results Areas and Theory of Change 42 E. Program Economic Evaluation 70 Annex 1: All Human Development Expenditures of the Government 91 Annex 2: HCP Program Expenditure Framework 97 Annex 3: ERCs classified by Schools supported by ERC 102 Annex 4: School Digital Action Plan: SELFIE Georgian Pilot Data from Sample of Schools 104 Page 1 of 153 Official Use The World Bank Georgia Human Capital Program (P175455) Technical Assessment A. Program Description Project Development Objective The PDO is “to make the human capital delivery system in Georgia more efficient and inclusive�. The human capital delivery system refers to the financial and administration system of selected education, health, TSA and employment programs as defined in the Program scope. Progress toward achieving the PDO will be measured through six key results indicators: Number of schools that meet minimum quality standards. Percentage of annual OOP spending on health care that is allocated to medicines. Percentage of TSA benefits going to households in the poorest quintile. Share of vacancies filled through the Worknet job-matching portal. Reduction in the rural urban gap in UNE scores. Average number of outpatient consultations per person per annum at PHC providers Scope of the Program The Government program for Human Capital for 2022-2031 is comprised of a specified set of Education, Health and Social Protection services that provide for the development of human capital and its effective engagement in the development of the country. In the 2021 budget, the three human development sectors together accounted for about 7.3 billion GEL (US$ 2.3 billion equivalent) out of a total budget of 18.4 billion GEL (nearly US$ 6 billion equivalent). The three human development sectors are managed by the Government of Georgia through various entities under two ministries – the Ministry of Education and Science (MOES) and the Ministry of Internally Displaced Persons from the Occupied Territories, Labor, Health and Social Affairs (MOILHSA). A specified set of human development sub- programs for the period 2022-2027 constitutes the Government Human Capital program, as summarized in Table 1 and further explained in the section on Program Expenditure Framework. The sub-programs were selected on the basis of an agreement on a set of crucial reforms needed in each of the three Page 2 of 153 Official Use The World Bank Georgia Human Capital Program (P175455) Technical Assessment human development sectors, to bring about transformative change in Georgia’s human capital within the Program duration of six years. Not all human capital sub-sectors or the accompanying reforms are included in the present Program Boundary for reasons of focus – reforms associated with pensions, higher education, most of vocational education and early childhood development. It is possible that successful implementation of the currently defined activities may lead to future expansion of the Government Human Capital Program. Table A1: The Government Program Item Government of Georgia (GOG) program for human Program supported by PforR capital 2022-2031 Ten-Year Plan (2022-2031) for the Development of Education and Science and Ten-Year Plan (2022- Government Program for Title 2031) of the Ministry of Internally Displaced Persons human capital (2022-2027) from the Occupied Territories, Labor, Health and Social Affairs of Georgia Education, Health and Social Protection services that Making human capital provide for the development of human capital and its delivery system in Georgia Objective more efficient and inclusive. effective engagement in the development of the country Duration 2021-2031 2022 -2027 Page 3 of 153 Official Use The World Bank Georgia Human Capital Program (P175455) Technical Assessment Efficiency for better value for money of human capital expenditures Results areas are the same Inclusion for quality of human capital expenditures Result Areas for all beneficiaries Connectivity of digital systems across all areas of human capital provision All areas of the country Geographical where human capital services Scope All areas of the country where human capital services are provided by the are provided by the Government of Georgia Government of Georgia Financial US$ 5,988 in period 2002-2027 + additional amount US$ 5,988 million 2022-2027 Envelope in 2028-2031 B. Program Strategic Relevance and Technical Soundness Program Strategic Relevance Page 4 of 153 Official Use The World Bank Georgia Human Capital Program (P175455) Technical Assessment Georgia has committed itself to strengthen its human capital base as a springboard to diversify its economy and create better quality jobs. High quality human capital will help Georgia avoid the middle- income trap and advance its long-anticipated integration into the European Union. While the government is committed to this goal, as evidenced by increased levels of public spending, concurrently it is not getting the value for money spent on the human capital sectors. Through the proposed PforR operation the government seeks to implement the transformational reforms needed to improve human capital outcomes and redirect the economy. Georgia is a small and open economy with a population of 3.7 million and a per capita gross domestic product (GDP) of US$4,785. Georgia’s economy grew at a 5 percent annual rate in the decade prior to the shock of the global pandemic. Growth has been supported by capital deepening and reforms in the governance and the business environment, but the quality of human capital remains an obstacle to sustained high economic growth1. Lack of continuity of transformational and deep cross cutting reforms in the human capital sectors is a factor holding prosperity back in Georgia. While Georgia has made significant sectoral progress through own government led programs supported by donors, they have suffered for lack of continuity when funding was phased out. Moreover, at times there has been a lack of an overarching push to sustain the policy direction in each ministry over a longer period which would allow the results to emerge. Concurrently, under a tight fiscal regime, the human development sectors have at times been competing for the same resources, instead of focusing on improving expenditure efficiency and efficacy in service delivery. Georgia since January 2021 has formally initiated the path toward applying for EU membership by 2024. This implies a significant effort towards improving governance, rule of law, inclusion and equity underpinned by greater fiscal efficiency and discipline. Concurrently, Georgia’s renewed focus on human capital will be required to both transform its institutions – private and public – towards the EU while improving human capital outcomes and expenditure efficiency to counter the demographic decline, - and to place its overall education, social and health services, standards and norms more aligned to EU 1 Survive, Learn, Thrive: Strategic Human Capital Investments to Unlock Georgia’s Potential; World Bank 2020. Page 5 of 153 Official Use The World Bank Georgia Human Capital Program (P175455) Technical Assessment institutional frameworks and best practices: this in turn will result in greater fiscal efficiency, better quality and equity and inclusion. There is potential for Georgia to significantly improve its human capital as measured by the Human Capital Index (HCI). A Georgian child born today, will only be 57% as productive as an adult as she could be with a quality education and enjoying full health (HCI value of 0.57) leaving significant room for improvements. This is evidenced by scores in international assessments (PISA 2018 score nearly 100 points behind the OECD average, which indicates that Georgian 15 year-olds perform on average at a level of 12 year-olds; or Georgian 18 year-olds at level of OECD 15 year-olds). It is seen in a Human Capital Index (HCI) value of 0.57). In order to help realize this potential, the Program is designed to bring about a revamped approach towards human capital. Economic activity crucial to the Georgian economy such as the tourism sector, high value agricultural exports such as wine and fruits and nuts, and remittances from migrant workers are human capital intensive. Technology-led modernization of the financial and agriculture sector holds great promise for Georgia as engines driving Georgia’s future shared prosperity. Emerging areas such as software development by tech startups require a throughput of high-quality human capital that begins with digital literacy in elementary school and a modernized curriculum throughout different stages of education. Overall Education Spending Georgia’s education spending, as a share of GDP, has grown but remains below regional and EU averages. Georgia spent 3.9 percent of its GDP on education in 2019, compared with five percent in the EU27 (Figure B1). The OECD recommends five percent. In addition, Georgia’s relatively low allocation needs to reach a larger percentage of the population compared to aspirational peers. The school-age population in Georgia represents about 26 percent of the total population, compared with 21 percent in the EU27, on average, suggesting that Georgia has more pressure to invest in services (such as education) for its youth population. Education spending has been growing (from around 3 percent of GDP at the start of the previous decade) and compared to the overall resources available to the public sector, Georgia is not under spending. In fact, Georgia allocates 13 percent of the total government Page 6 of 153 Official Use The World Bank Georgia Human Capital Program (P175455) Technical Assessment expenditure to education, similar to that of EU27 (12 percent), and aspirational countries such as Estonia (13 percent) and the Czechia (14 percent) (Figure B1). Figure B1. Education spending: Georgia vs comparators Latest available In percent of GDP In percent of government expenditure Source: Forthcoming Georgia Country Economic Memorandum (2022) Georgia’s overall spending on health is larger than in many countries in the ECA region, although it remains below the European average. In 2018 (i.e., pre-COVID-19) Georgia spent a little more than 7 percent of GDP on health (Figure B2). This places Georgia among the non-EU countries that spend most on health in the region, although spending remains below the EU average (9 percent). When measured in absolute terms (in US$ PPP), however, spending remains relatively low compared to many countries in the region, which may affect the ability to purchase pharmaceuticals or equipment that are priced in the international market. Figure B2: Georgia’s health spending is larger than in many countries in the region Page 7 of 153 Official Use The World Bank Georgia Human Capital Program (P175455) Technical Assessment 12% 5000 10% 4000 Share of GDP 8% PPP int. $ 3000 6% 2000 4% 2% 1000 0% 0 AZE TUR CYP MNE MLT EU TKM BIH RUS MKD HUN UKR ALB LTU SVK LVA BGR CZE ECA* KGZ MDA GEO SVN EST SRB UZB BLR ARM KAZ ROU POL HRV TJK Current health expenditures (% of GDP) Source: WHO (Global Health Observatory and Global Health Expenditure Database). Data for 2018. Averages are population-weighted. ECA*: LIC & MIC ECA (excl. Russia). Overall Social Protection Spending In 2018, Georgia spent about 6.2 percent of GDP on social protection, which remains below the average social protection spending in Europe and Central Asia (around 8.7 percent of GDP). Specifically, Georgia spent 2.1 percent of GDP in 2018 on social assistance (including targeted social assistance and categorical non-contributory benefits), above the average for Eastern Europe and Central Asia, and 3.6 percent of GDP percent on the (non-contributory) universal social pension. Spending on labor market programs was negligible and very limited compared to other countries. Spending on the contributory pension pillar was not included as it was introduced in 2019. It should be noted that social protection spending jumped to almost 10 percent of GDP in 2020 because of the COVID-19 pandemic (Figure B3). Social assistance programs were expanded, and some new benefits were introduced (see below), but it is as yet unclear how much of the higher spending will remain in the medium to long term. Figure B3: Social protection spending remains below the ECA average Page 8 of 153 Official Use Official Use 0% 5% 10% 15% 20% UKR - 2017 Assessment MNE - 2012 The World Bank ROU - 2017 POL - 2012 Social Insurance SRB - 2019 UZB - 2017 BGR - 2017 HRV - 2014 MKD - 2019 Social Assistance Source: World Bank SPEED database. BLR - 2017 HUN - 2016 Georgia Human Capital Program (P175455) KGZ - 2017 EST - 2017 LVA - 2017 CZE - 2017 ALB - 2019 Labor Market Programs RUS - 2016 TUR - 2016 MDA - 2017 LTU - 2016 Page 9 of 153 GEO - 2018 GEO - 2020 Social Care Services KSV - 2017 ARM - 2017 AZE - 2014 KAZ - 2017 TJK - 2017 SVK - 2017 General Subsidies BIH - 2017 Technical The World Bank Georgia Human Capital Program (P175455) Technical Assessment Program Technical Soundness Comparison of Georgia’s human capital performance with a group of Eastern European and Central Asian countries provides a useful context to examine the Government’s policy response.2 The Human Capital Index (HCI) for Georgia indicates that a child born in Georgia today will only be 57 percent as productive with respect to his or her full education and health potential. The HCI captures basic education and health elements of Human Capital that have a clear impact on people’s productivity. As can be seen from Figure B4, while Georgia’s level of Human Capital is aligned with the one of countries with similar levels of development, it remains well below the ECA average for low- and middle-income countries (63 percent), and well below the EU average (74 percent). Moreover, if elements that include the performance of higher education and NCD are included productivity drops even further – to 40 percent. Figure B4: Georgia’s Human Capital Index remains below ECA’s average 2 The comparative view of Georgia’s Human Capital and by sectors within human capital is drawn from analytical work for the forthcoming Georgia Human Capital Review, to be published by the World Bank, with financial support from the European Union. The HCI data are from the World Bank’s 2020 report “The Human Capital Index 2020 Update: Human Capital in The Time of COVID-19�. Page 10 of 153 Official Use The World Bank Georgia Human Capital Program (P175455) Technical Assessment 80% EU Average 75% 2020 Human Capital Index 70% 65% LIC & MIC ECA Average 60% 55% 50% 45% 40% GEO KGZ ALB LTU KOS LVA TJK MKD AZE ARM POL SRB MLT CZE UZB HRV CYP MDA RUS BLR BIH ROU BGR KAZ MNE TUR EST HUN SVN UKR SVK HCI Average of Development Peers Source: World Bank HCI 2020 Update ;Averages are population weighted. LIC & MIC do not include Russia. Average of development peers is obtained by using a kernel regression. The technical soundness of the Program can be judged from an examination of the reforms supported under each sector: in the education sector, the reforms related to the purpose, content and mode of delivery of education. The purpose needs to be aligned towards the provision of life-long learning skills also termed as 21st century competencies; the content needs to be re-aligned so as to be more relevant to future as well as current jobs and to develop self-reliance and entrepreneurship skills; the digital capabilities of teachers and indeed of the entire digital ecosystem for education needs to be thoroughly modernized. In the health sector, the policy solutions call for efficient functioning of all levels of health care, beginning with primary health care. It is critical that efforts are made to increase efficiency and obtain better value for money from current spending. To achieve this, the reforms need to include a revitalized PHC system that is regionally balanced, well-articulated with affordable, secondary and tertiary care; efficiently managed hospital and pharmaceutical contracts that incentivize quality while reducing cost and patients’ out-of-pocket health spending, and the use of digital technologies to help provide quality medical attention and wellness care. For social protection, the requirement is to upgrade the social protection delivery system, the employment services and job-relevant skills provision to best prepare the workforce to better job Page 11 of 153 Official Use The World Bank Georgia Human Capital Program (P175455) Technical Assessment opportunities. The Program will support reforms to strengthen the capacity of the social protection system to better identify poor families with children and to respond to negative shocks and sudden changes in vulnerability and new forms of poverty, thus to make it a more inclusive and resilient system that helps the needy; it will support reforms to connect people to job opportunities through effective employment services and skill training as institutional arrangements for providing skills and connecting job-seekers with jobs are nascent in Georgia. Education Sector Analysis of technical soundness The Comparative analysis of Georgia’s education sector performance shows low achievement in relation to neighboring countries that share historic characteristics with Georgia. Georgia achieves relatively high enrollment rates: at 106 percent, the Gross Basic Enrollment rate (Grades 1-9) is close to the EU average (110 percent), although attendance is much lower for pre-primary education and also drops at the Upper Secondary level (85 percent net enrolment at Grades 10-12). The quality of education, when years of education is combined with information from standardized tests of learning, differs much from the European average. Students in Georgia rage close to 12.9 years of schooling, but they receive on average only 8.9 years of schooling when adjusted for quality3. Georgia has the second lowest reading score in the region in the 2018 PISA test scores, which captures reading ability of 15 year-old students (Figure B5, left panel), and 65 percent of its 15 year-old students are functionally illiterate, meaning that they cannot correctly process and understand a simple age-appropriate text. Moreover, differences between the best and worst performers are significant, with the bottom 25 percent performing 70 PISA points below the top 25 percent. Quality also remains an issue in higher education (Figure B5, right panel), where in spite of high tertiary education attainment among the population, the quality of the tertiary education remains far below the average of the EU and of many other ECA countries. 3 Filmer, Deon, Halsey Rogers, Noam Angrist, and Shwetlena Sabarwal. 2020. “Learning-Adjusted Years of Schooling (LAYS): Defining a New Macro Measure of Education.� Economics of Education Review 77. https://doi.org/10.1016/j.econedurev.2020.101971. Page 12 of 153 Official Use The World Bank Georgia Human Capital Program (P175455) Technical Assessment Redressing past education performance gaps becomes more complicated as society’s demands from education have changed drastically. In Georgia, as in a number of other countries, education provision has remained rooted in an industrial model. In the industrial model of education, designed for the factory mode of production, the teacher’s job was limited to providing a range of facts to students. While facts still form the basis of knowledge, mere transmission of facts is not sufficient preparedness for increasing complexity, readiness for climate change, and resilience to face the next pandemic. Education policy experts speak of 21st century competencies for a world where limitless knowledge can be accessed anywhere pressing a few buttons on a smartphone, but where human needs of partnership and communication are unchanged. Critical thinking, problem-solving, collaboration and creativity have been valued throughout history, but until recently, it was not possible to systematically provide such competencies in a general education system. In the modern age, the risk is that individuals who do not learn these competencies will be rendered irrelevant; nations which are unable to retool their education systems for the modern age will simply be bypassed by history4,5. Figure B5: Low and unequal learning outcomes Quality and equity of learning outcomes Tertiary education 600 EU Average 2018 PISA score 500 400 300 KOS EU GEO ALB LTU AZE MKD ECA* SRB KAZ MNE MDA BIH BLR RUS TUR UKR Mean score (reading) Bottom quartile mean score Top quartile mean score 4 Educating for the 21st Century: Perspectives, Policies and Practices from Around the World, Suzanne Choo er al (Eds), Springer, 2017 5 24 of 28 recommendations made by The OECD Review of Evaluation and Assessment in Georgia, 2019, are in the Program. Page 13 of 153 Official Use The World Bank Georgia Human Capital Program (P175455) Technical Assessment Source: Demirgüç-Kunt and Torre (2020); OECD (2019b). Note: Averages are population-weighted. ECA*: LIC & MIC ECA (excl. Russia). The curriculum is the driver of all education – it sets the destination of the learning journey and providers the milestone markers and landmarks. The defining rules of the game are framed by the curriculum6. According to Nuno Crato, highly successful Education Minister of Portugal: “The curriculum needs to be ambitious, demanding and set clear objectives. These objectives must be sequenced, setting solid foundations for students’ progress… Everything needs to be coherent around curricular goals.� Georgia set forth on just such an ambitious curriculum for Primary Education in 2020, tasking teachers and school communities to build school curriculums that are centered on a very challenging construct of “complex assignments�. From 2021, this approach is being extended to Secondary Education schools in Georgia, with close engagement of experts from Estonia. The MOES gap analysis of the current curriculum notes “the absence of the targeted and functional purpose of the upper secondary level. the transition to the upper secondary level does not bring along any changes in terms of their study experience.� This is not adequate for an education system for the 21st century. The Government has set forth an even more ambitious plan for Secondary education to address the main weaknesses. Classroom and online implementation of the new curriculum is a complex endeavor that includes a large number of elements – standards for schools and a mechanism to enforce those standards with provision of resources and support. Tied to the enforcement of standards is the school re-authorization process mandated by the Law on Education for the year 2026-2027. Along with the standards, comes a process of support to schools through specialist teams that will work with schools – this is very important because the new curriculum is very demanding and teachers and school leaders need a close form of support. A key feature of the implementation is the development of a school curriculum and a teacher individual development plan. The school curriculum is a way for a school community to adapt the national curriculum to the particular local needs of it students. A teacher individual development plan is a live online system for support of teachers that is managed by the teacher herself who undertakes a self-assessment and identifies her training needs, the means to meet their needs, discussions with mentors about the accuracy of her self-assessment and the management of her progress. 6 Audacious Education Purposes: How Governments Transform the Goals of Education Systems, Fernando Reimers (Eds.), Springer Open, 2020; Improving a Country’s Education: PISA 2018 Results in 10 Countries, Springer Open, 2021. Page 14 of 153 Official Use The World Bank Georgia Human Capital Program (P175455) Technical Assessment The progressive fragmentation of the financing model of general education, under which more than USD 250m each year is spent in Georgia’s Program Budget, has generated serious problems regarding efficiency, equity and transparency. With a view to eradicate corruption in the school management system, since 2006, Georgian reformers removed local governments from the budget process, which allowed the central government to fund schools directly from the national budget. The Ministry of Education chose vouchers as the simple financing model with just three values that include urban, rural, and mountain schools. Over time, this model evolved into a very complex system, quite different from standard school voucher initiatives, with 60% of schools essentially receiving discrete payments. The system of school financing in operation today in Georgia is no longer a voucher model, though it is continued to be called as one. The complex and opaque financing system is not able to adequately support the needs of different schools across the country. The theoretical benefits of a standard voucher system, namely transparency and equity are missing and needs reform7. The EMIS system reveals a stark picture regarding the extant fiscal in efficiencies, and the problem of poor quality of learning in small schools exacerbates the negative shock to the system. In 2020, the system registered 2,308 schools in Georgia, of which 2067 are public schools. Of these 2,067 schools, there are 1,296 schools (more than 60%) that are officially regarded as small schools with an average enrollment of 71 students, for whom the voucher formula does not apply. These 60% of schools enrolled 92,012 students, which is less than 20% of the total student enrollment. If we look at the total financial transfers made to schools, a more interesting picture emerges. In 2020, the small schools received transfers amounting to more than 300 m GEL or 40% of the overall transfers. While it is inevitable that smaller schools would be more expensive, it is not inevitable that the skew would be as extreme as exists in Georgia, when compared internationally, including former state-planned economies in Eastern Europe. A very important manifestation of the education financing problem and the inertia of maintaining schools of all grades together is the issue of low quality of Secondary schooling. In order for a school to be able to offer a good quality learning experience, where Secondary students can experiment and 7 Review of Education Financing Model I – Evolution of the Voucher Formula 2006-2021; and Review of Education Financing Model II – Policy Options for Reform – Working Papers as part of South Caucasus Education ASA – P174980 Page 15 of 153 Official Use The World Bank Georgia Human Capital Program (P175455) Technical Assessment figure out their interests and strengths and indeed their vocation, there needs to be a critical mass of students. Scale is also needed to have adequate infrastructure like laboratory and IT equipment. Most importantly, scale is needed because teaching and learning is a community endeavor. Students needs the stimulus of competition and the support of friendships, much of learning is a team-effort for students as well as for teachers8. The required scale is not achieved by a staggering proportion of Georgian schools. Schools in Georgia are not differentiated into Elementary/Middle and Secondary. Of the approximately 2300 schools in Georgia, there happen to be 400 schools that do not have any children in Grades 10 to 12 (but they still have subject teachers). There are another 400 schools with less than 20 children in Grades 10 to 12. The Government plans to invest in providing the adequate scale for High Schools with sufficient quantity and quality of didactic inputs, including teachers with necessary qualifications, training and motivation. Previous neglect in formulating clear objectives for secondary education, combined with high stakes university entrance examination led to serious quality and equity problems the Program seeks to resolve. A recent review of the upper secondary education curriculum (Grades 10 through 12), carried out by experts from Estonia, remarked on the “absence of a targeted and functional purpose of the upper secondary level�.9 Though students prepare for the Unified National Examinations (UNE) that are needed to enter universities, the study notes a lack of engagement of students in their school classes. A clear manifestation of this is the neglect of school classes by students and the high outlays on private tuition classes to prepare for those examinations10. Even as early as 10th grade, many students do not find it worthwhile to attend classes at school. Reform in High Schools will go hand in hand with reform in university entrance, so the UNE score is not the singular metric used, as is the case today. Figure B6 shows a figure about estimated days of skipped classes in the previous two weeks as answered by 15 year-old students taking the PISA examination. Georgia figures 2nd amongst 80 countries that took part 8 School Size and Student’s Achievement: Empirical Evidences from PISA Survey Data, Francesca Giambona and Mariano Porcu, Socio-Economic Planning Sciences, 2018 9 Policy Document: “Promoting quality education for all children through improving the National Curriculum for grades 11-12� in partnership with the Estonian Education and Youth Board and UNICEF, February 2021, MOES. 10 The uses and misuses of centralized high stakes examinations – Assessment Policy and Practice in Georgia, Sophia Gorgodze and Lela Chakhaia, Assessment in Education: Principles, Policy & Practice, 2021 Page 16 of 153 Official Use The World Bank Georgia Human Capital Program (P175455) Technical Assessment in PISA 2018. The estimates from PISA data indicated that Grade 10 students in Georgia miss 54 days in the school year, by the time they reach Grade 12 (comparable data not available) this number would have gone up, and it is the less well-off students who cannot afford the tuition who suffer the most. Reforming the education structure as part of the Program will culminate in a process of re-authorization required for all schools. All schools in Georgia will need to complete a re-authorization process that depends on the quality of the learning environment including physical characteristics as well as teaching quality and scale. The re-configuration of the schooling system requires a mapping of the future supply of schooling places with demand. Safe school transport for older children will help to build scale, but needs to consider the particular geographic and climatic conditions of Georgia. In particular, some roads may not be accessible in winter months and in-person education would need to be supplemented by distance learning with adequate broadband connection. Consultations with local communities would require time as students, parents and other stakeholders learn about the various alternatives and the consequences of choices they would need to make. It is possible that at a future date, local government capabilities would be strengthened enough to take up responsibility to manage the school system in the area. Until such date, MOES plans to strengthen the role of ERCs in each municipality with equipment, staffing and working arrangements that would allow support for local school networks. System restructuring is expected to be a gradual process, with the financing model revised as knowledge is gathered about the possible need gaps generated as the supply configuration is altered. Figure B6: PISA 2018: Days of Instruction Lost due to students not attending school Page 17 of 153 Official Use The World Bank Georgia Human Capital Program (P175455) Technical Assessment Source: OECD-PISA 2018 The Program includes a commitment to reform the financial model to enable school communities to access the resources they need: (i) The model would adhere to criteria of transparency and equity; (ii) The model would include adequate financing for payment of teacher salaries, financing of utilities including energy, lighting, ICT connectivity and general maintenance expenditures, teacher professional development and other education quality and student wellness related resources aligned to the new competency based curriculum; (iii) The financing of energy expenditures would incentivize emission reducing and energy efficient modalities; (iv) The financing would provide for adequate staffing, IT equipment and operational expenses for an enhanced role of Educational Resource Centers, including the role of ERCs to optimize school networks and to support teacher individual development plans, school principal instructional leadership, and formative assessment of students; (v) The financing would incentivize the formation of school clusters at the Upper Secondary Level (Grades 10 to 12), precluding Page 18 of 153 Official Use The World Bank Georgia Human Capital Program (P175455) Technical Assessment this level being offered in schools where the total number of Grade 10-12 students is lower than 100 for city schools and lower than 50 for rural or mountainous schools; (vi) allow for the provision of planned provision of vocational education services in selected Upper Secondary education schools. The reforms are expected to lower the performance gap shown below, between urban and rural students in Science, Mathematics and Foreign Languages (see Figure B7). Figure B7: Urban-Rural Performance Gap in Unified National Examination Source: NAEC. [Error bars show width on one standard deviation of student scores on each side from the mean] Page 19 of 153 Official Use The World Bank Georgia Human Capital Program (P175455) Technical Assessment Ensuring quality of pre-school services is also an important policy consideration. Georgia's decentralized preschool system currently faces a number of challenges, including: lack of data on children enrolled in or left out of the system, absence of a measurement system to assess the quality of service delivery, and lack of feedback mechanisms from service users (children and their parents) on the effectiveness and efficiency of preschool services. The Law on Early and Preschool Education (effective since September 2016) requires municipalities to provide oversight and monitoring of preschools in the implementation of national standards including education, nutrition and food safety, sanitation and hygiene, and physical environment. Finally, transformational change in education in the 21st century needs to have a strong digital component, which means providing IT infrastructure and setting in place conditions for use through school level action. The key instrument to implement this policy are School Digital Education Action Plans, to be implemented using the European Union Joint Research Center’s SELFIE (Self-reflection on Effective Learning by Fostering the use of Innovative Educational Technologies) tool, which has been initiated in Georgia with support from the Estonian Government. [Smart, Digitally Enhanced Learning Ecosystems: Bottlenecks to Sustainability in Georgia, Eka Jeladze and Kai Pata, Sustainability, 2018.] Health Sector Analysis of technical soundness Figure B8: Georgians face among the highest incidence of NCD in ECA Page 20 of 153 Official Use The World Bank Georgia Human Capital Program (P175455) Technical Assessment Disability-adjusted life-years 50,000 (NCD DALY per 100,000) 40,000 EU Average (DALY) 30,000 20,000 10,000 0 CYP TUR TKM AZE MNE MLT BIH RUS HUN SVN MKD GEO UKR LVA BGR ALB ARM SVK CZE LTU KGZ UZB MDA KAZ EST BLR SRB ROU TJK POL HRV NCD DALYs Average of Development Peers Source: World Development Indicators and Global Burden of Disease, 2019. Note: Averages are population-weighted. Average of development peers is obtained by using a kernel regression. From a comparative perspective, Georgians face among the highest incidence of NCD later in life in ECA, which is affecting productivity, life expectancy and healthy aging. The prevalence of NCD in Georgia is way above the EU average and the average of its development peers (Figure B8). Such high rate has negative impacts on people and society all along their life cycle. During their working age, NCD affect workers’ productivity; and many people are obliged to work less or retire earlier than they would have liked because of NCD. This phenomenon shrinks the working population in a country that is already rapidly aging. A high incidence of NCD is also among the main causes for low life expectancy, especially for men, and also affects heathy aging and the quality of living of many elderly. Finally, while prevention of NCD is relatively cost effective, providing comprehensive treatment is expensive and adds an unnecessary fiscal burden on the health system. There are strong inequities in health outcomes, with many poor and vulnerable households facing the dire choice of falling further into destitution to pay for health care, or not being able to seek treatment. Inequities in health outcomes start early in life .and cumulate all along people’s life cycle. A significant feature reflecting such inequities is the dire choice that many poor and vulnerable households face between falling further into destitution to pay for health care, or not being able to seek treatment. Impoverishing out-of-pocket (OOP) health expenditures are the highest in Europe and Central Asia, and an estimated 5.7 percent of the population is pushed below the poverty line each year because of health expenditures (Figure B11). Such a harsh tradeoff translated into inequities in health outcomes all along people’s life cycle. Page 21 of 153 Official Use The World Bank Georgia Human Capital Program (P175455) Technical Assessment Universal health coverage means that all people have access to the health services they need, when and where they need them, without financial hardship. It includes the full range of essential health services, from health promotion to prevention, treatment, rehabilitation, and palliative care. Georgia introduced its universal health coverage program (UHCP) in 2013 and has made significant progress in improving access to health services so far. Georgians, particularly those relatively less well-off, has benefited from the UHCP that has improved access to health services and reduced the likelihood of impoverishment or catastrophic out-of-pocket (OOP) spending on health care. The UHCP extended publicly financed entitlement to health care coverage to the entire population. The benefits package covers a range of primary and secondary care services and limited essential drugs. The health financing reforms introduced since 2013 and backed up by significant increases in public health spending. Since the implementation of the UHCP, the public health spending has risen sharply (from 5.5 percent to 10.3 percent of total government spending between 2012 and 2018). While the public spending in Georgia comprised just 19.4 percent of total health spending in 2012, one of the lowest in the European region, it is 39.4 percent in 2018. Despite the increase in public health spending, the share of OOP health spending remains higher than in many other comparable countries (Figures B9 and B10). Figure B9: Public Health spending as % of General Government Spending, 2018 Page 22 of 153 Official Use The World Bank Georgia Human Capital Program (P175455) Technical Assessment Source: World Development Indicators, 2021 Although public resources allocated to health is about average for Georgia’s income level, the share of OOP health spending remains higher than in many other comparable countries, resulting in poor financial protection. For a country whose public health spending is 2.8 percent of its GDP, similar to Georgia, 54.7 OOP health spending is too high. Despite rising allocation of government’s budget to health, more public health spending is not translated into lower OOP and, thus, better financial protection. OOP is likely to distort people’s incentive towards overuse of costly services that are free under the UHCP, such as inpatient emergency services. Page 23 of 153 Official Use The World Bank Georgia Human Capital Program (P175455) Technical Assessment Figure B10: OOP health spending vs. public health spending, 2018 Source: World Development Indicators, 2021 Figure B11: Impoverishing health expenditures are among the highest in the region Page 24 of 153 Official Use The World Bank Georgia Human Capital Program (P175455) Technical Assessment 6% Share of population pushed 5% below the poverty line 4% (3.20$-a-day) 3% 2% 1% 0% AZE CYP TUR MNE BIH KOS MLT RUS UKR MKD SVN SVK LTU BGR UZB ALB SRB LVA KGZ GEO BLR EST MDA ARM KAZ HRV POL ROU TJK Source: World Bank Health Equity and Financial Protection Indicators. Averages are population- weighted. Data is for the latest available year for each country. The UHCP has consistently overspent its budgeted amount in recent years, which raises concerns about the sustainability of the program. To maintain the progress made under the UHCP, and further deepen coverage and financial protection, the government needs to ensure the financial sustainability of the UHCP. This requires efforts to increase efficiency and obtain better value for money from health spending. First, purchasing mechanisms that are good at managing costs effectively and incentivizing patients and providers should be adopted. The purchasing mechanism is the way in which public funds are used to deliver health services. It includes: (i) managing revenues and expenditures; (ii) contracting; (iii) paying providers and setting the right incentives; and (iv) monitoring provider performance, service and quality. As the public purchaser of health services under UHCP, the National Health Agency (NHA) potentially has the power to purchase services strategically and manage costs effectively. In practice, however, the NHA is more of a passive purchaser (WHO 2016). The detailed and complex payment system for hospitals with different tariff setting and copayment rules for different types of hospital care enables providers to game the system, which leads to increased administrative costs and constrains NHA’s ability to control costs. The NHA’s main instruments for ensuring services are delivered appropriately are prior Page 25 of 153 Official Use The World Bank Georgia Human Capital Program (P175455) Technical Assessment authorization and claims management. Both processes are a drain on the UHCP’s administrative capacity because of the extensive paperwork involved. In practice, all claims from hospitals are reimbursed. In recent years, the NHA has also taken steps to standardize tariff-setting rules (for example, for critical and intensive care), which have already led to cost savings. Second, ensuring value-for-money of pharmaceutical spending is another key reform area. Current reimbursement and purchasing systems contribute to inefficient spending. While medicines provided under MOILHSA’s vertical programs are centrally procured, medicines for inpatient care are purchased directly by hospitals, likely with large variations in prices. In the absence of price controls and uniform reimbursement rates, the UHCP’s outpatient benefit also pays for medicines at variable prices. Adopting a reference pricing system can improve the efficiency of spending on pharmaceuticals in the public sector. Reference pricing uses either: price information from other countries to derive a benchmark for the purposes of setting or negotiating prices at home (external); or prices of pharmaceutical products that are therapeutically similar including generics, biosimilar products, and closely substitutable products which contain different active ingredients that produce similar health effects (internal).11,12 While 23 out of 31 countries uses external reference pricing as the main systematic criterion when setting the price of a medicine, others including Belgium, Finland and Germany use it as supportive information.13 Similarly, 17 out of 27 EU countries and Canada, Japan and Switzerland use internal reference pricing to set the price of medicines. Figure B12: Overview of external reference pricing across Europe, 2018 11 External reference price methodologies differ widely across countries–for example, variations in product and country reference baskets, and different algorithms used to determine the reference price for a given product. External reference price information is also usually combined with other policy considerations before prices are set (Ruggeri and Nolte 2013). 12 WHO Guideline on Country Pharmaceutical Pricing Policies 13 Rémuzat, C., D. Urbinati, O. Mzoughi, E. El Hammi, W. Belgaied and M. Toumi. 2015. Overview of external reference pricing systems in Europe. Journal of Market Access and Health Policy. Page 26 of 153 Official Use The World Bank Georgia Human Capital Program (P175455) Technical Assessment In addition to the reference pricing, many OECD countries and European Union (EU) member states now use managed entry agreements (MEAs), when adding new medicines to the health benefit packages. MEAs have been widely implemented in the last decade as an alternative to tendering due to patent protection preventing competitive bidding between producers. MEAs, if implemented smart, have potential advantages as a longer term, more sustainable framework that distributes risk between the payer and the producer to further their mutual goal of facilitating patient access to new medicines. MEAs can be designed to distribute financial risks, risks relating to the outcome or performance of the treatment not being as expected in real life, or a combination of both financial and performance uncertainty. Financial schemes aim to minimize the financial risks to the payer in making a positive reimbursement decision and publicly financing the new medicine and in reducing its cost. Of all agreement types, these are internationally by far most widely used because of technical simplicity in contract preparation, low administrative burden of monitoring agreement performance. Performance based agreements require substantial administrative capacity and/or advanced clinical IT systems in monitoring implementation as therapy effects of all treated patients have to be monitored and recorded. Over the medium term, the government needs to consider expanding outpatient drug coverage in the UHC Program. Expanding the UHC drug benefit would help to defray OOP costs of medicines incurred by Page 27 of 153 Official Use The World Bank Georgia Human Capital Program (P175455) Technical Assessment individuals and improve financial protection. In practice, a gradual expansion of the drug benefit needs to be considered, starting with a limited subset of patients and products, for example, high burden diseases, vulnerable groups, or high-cost medicines. Given that CVD ranks at the top for both burden of diseases and mortality, and the relatively low cost of treating hypertension, one option would be to start by expanding the benefit for first line antihypertensive drugs. Depending on the fiscal space available, the drugs could be provided at a higher reimbursement rate, or free of charge, to a broad majority or all patients with CVD. To ensure that there is minimal leakage or gaming of the benefit, a track-and-trace system that enables electronic prescriptions needs to be in place to monitor prescribing, dispensing, and adherence. Third, Georgia’s health service delivery system needs to be reduced reliance on acute hospital and emergency care by reorienting towards greater emphasis on primary care and improving coordination between primary and hospital care. Such a reorientation would ensure that, for most chronic diseases, the onset of the disease is prevented, or secondary prevention is provided early on; and treatment, where needed, is provided early, thus avoiding the more acute and catastrophically expensive hospital treatments that are needed later in the disease. Coordination of care across providers reduces duplication of tests, and a strong primary care function based on a longstanding doctor-patient relationship ensures continuity of care and better patient case management. Experience from countries such as Germany, Denmark, segments of the United States, and Turkey have shown that a strong primary health care system is critical in achieving good health outcomes. One of the key elements of this reorientation of service delivery is enhancing quality and efficiency of primary care. The rural doctor program needs to be integrated into the UHC Program in order to reduce fragmentation of primary care. The incentives facing primary care providers need to be strengthened so that primary care providers take more responsibility for patient care, particularly for patients with multiple, chronic conditions. This includes adjusting the capitation rate for risk or patient needs and introducing a performance-based component to reimbursement for primary care. Reorienting service delivery away from hospital services and strengthening primary health care will certainly improve outcomes and efficiency in the long term, but this reorientation of service delivery would require additional investments in the health sector that cannot be achieved through efficiency savings alone. Primary care utilization rates are increasing but remain low, exacerbated by a shallow outpatient drug benefit under the UHCP. As long as access to outpatient drugs remains limited, patients will face strong incentives to go directly to hospitals and/or emergency services when ill. Therefore, another key element of the reorientation towards primary care is expanding access to essential drugs in primary Page 28 of 153 Official Use The World Bank Georgia Human Capital Program (P175455) Technical Assessment care. This involves not only expanding the outpatient drug benefit in the UHC Program but also increasing the availability, affordability and prescribing of generic medicines. Social Protection Sector Analysis of technical soundness of reform Social protection, in the form of targeted social assistance and employment promotion programs, plays a key role in helping households build and protect human capital. The Targeted Social Assistance (TSA) cash transfer program in Georgia provides poor and vulnerable households with children with the financial stability to invest in human capital services such as health clinics, schools and nutrient food. The TSA serves as a platform to identify other vulnerabilities (child malnutrition, school drop- outs, cases of domestic violence for example) to be addressed through other services and benefits beyond the TSA cash transfer. In Georgia, the TSA central targeting scoring system is used not only for the administration of programs centrally managed – like the TSA – but it is also used by municipalities to assign health fee exemptions, scholarship and education fee waivers, free food meals, child protection services, energy discounts and other poverty targeted social benefits. The newly established SESA14 reporting to MOILHSA is tasked to carry out labor intermediation, implement employment promotion programs and organizing the implementation of job-seekers’ vocational training and retraining courses to enhance their employability and/or prospects for self-employment. By facilitating vulnerable people employment and youth transition into the labor market, SESA’s services ensure that investments in human capital are fully utilized. Georgia has undertaken many innovations in its social protection delivery system, including in the development of its Social Registry and Single Window front office model for citizen interface. In Georgia, the processes for determining potential eligibility are supported by the Unified Database of Socially Vulnerable Households (hereafter the social registry) and a Single Window approach by which households can apply for TSA (and other benefits and services) at local Social Window offices operated by the SSA. The social registry includes information for about a third of the population, precisely 334,823 households (equivalent to 31.4 percent of households in the population) as per first quarter 2021. Two key features make the social registry a particularly useful tool for social policy. First, it operates as an 14 The State Employment Support Agency (SESA) under the Ministry of IDPLHSA became operational on 1 January 2020. Page 29 of 153 Official Use The World Bank Georgia Human Capital Program (P175455) Technical Assessment Integrated Social Registry (for benefits paid from national budget), serving not just the TSA but numerous other poverty-targeted benefits and services at central and local government levels including subsidies under the Universal Health Care (UHC) system, free food meals, social energy tariffs, scholarships, special benefits for poor disabled, foster care, transport subsidies, for example. Second, the social registry operates on an “on-demand basis� and continuous access is facilitated by the extensive network of local and regional SSA offices; it provides dynamic inclusion, such that anyone can apply at any time at the SSA offices. Not all countries’ have achieved both of these social policy roles with their national social registries, and these are important strengths for Georgia’s registry15. The technical assessment has highlighted some key areas for reform in line with the government’s priorities and lessons learned from international experience. These include the targeting reform to improve equity and efficiency of spending, the digitalization of the social assistance application, registration and eligibility verification processes for higher responsiveness to poverty and vulnerability changes, the scale up and adaptation of SESA employment services and programs to best serve vulnerable jobseekers and match labor market demand, and the upgrade of the information systems to create a more holistic view of the needs of the poor and other vulnerable persons, the coverage of both central and local social protection and employment services by regions and municipalities against poverty, unemployment and economic activities/ labor demand. Another aim of the improved information systems will be to provide support for reducing exclusion and inclusion errors. The targeting model of the TSA needs to be updated to improve the human capital Program’s ability to target the poor, hence more equitable outcomes. The PMT scoring formula is used to determine eligibility to the TSA, the government’s flagship anti-poverty program, to the Universal Health Care benefits as well as a poverty targeted benefits and social care services provided by municipalities over the country. The targeting system played a huge role also to manage the pandemic crisis. In 2020, as much as 27 mln USD emergency transfers were delivered through PMT-based poverty targeting, including nearly 5 mln USD spent on coverage of tuition fees. Improvements in the quality of the PMT scoring formula will therefore result in more accessible human capital to those who, due to poverty and vulnerability, have the least opportunity to invest in human capital through their own resources. 15 Lindert, Kathy; Karippacheril, Tina George; Rodriguez Caillava, Inés; Nishikawa Chavez, Kenichi. 2020. Sourcebook on the Foundations of Social Protection Delivery Systems. Washington, DC: World Bank. Page 30 of 153 Official Use The World Bank Georgia Human Capital Program (P175455) Technical Assessment Nonetheless, the accuracy in identifying the poor has worsened over time and is bound to further deteriorate due to structural issues with the underlying Proxy Means Testing formula and the Needs Index computation16.The World Bank has assisted MOILHSA to re-estimate the PMT scoring formula based on more recent household survey data (GEOSTAT HIES 2018), a sounder treatment of nominal variables, and more accurate incorporation of observable components of welfare for the estimate of the same. The updated PMT scoring formula based on the new approach is expected to increase the number of beneficiaries from the poorest quintile by nearly 12%. The World Bank technical assessment17 proposes a simplification of the equivalence scales and a revision of the cohabitation coefficient. It also challenges the accuracy of the current subsistence minimum consumption basket and calls for its update based on more reliable assumptions on consumption patterns. The registration, eligibility verification and enrolment processes of poverty-targeted benefits could be better streamlined and digitized for timely, efficient and adaptive social protection delivery. Major bottlenecks to timely delivery lie in the application, registration, eligibility verification and enrolment procedures. The application process, for instance, has to be initiated only by an in-person visit of the household to the nearest SSA office (of which there are 69 in the country). Declaration forms are on paper. Data operators enter the information into the social registry using software applications for the front office. Only for half of the variables needed to generate the PMT score, the social registry pulls information from other government administrative systems using the unique national ID and 16 In addition to that, the PMT score is slow in reflecting sudden drops in income, since it reflects the monthly income of the household in the second month prior to application. For instance, if a household lost all of its income starting from mid-February, the full extent of this drop would be reflected in the monthly income in March and this would be captured by the PMT score if the application were made, at earliest, on the 1st of May. The registration and certification process would then be expected to last from May to August, and the first payment would be received at the end of October according to the current processes 17 Honorati, Carraro, and Sormani. 2021. Needs Index in Georgia: Assessment and Proposed Revisions. The World Bank. The existing needs index used to adjust the estimated welfare for household size dates back to before the recent economic growth of Georgia and misrepresents consumption patterns of people in nowadays Georgia. It does not accurately represent observed economies of scale in households’ welfare or standards of living. It hinges on a very granular partition of household members that, in the effort to increase precision, ends up introducing sources of discontinuity across many age thresholds, with negative consequences on predictability and transparency. Page 31 of 153 Official Use The World Bank Georgia Human Capital Program (P175455) Technical Assessment interoperability capabilities for data exchange. The local Social Agent then pulls up that information, manually collects other information on remaining variables and conducts a home visit to the applicants’ home to gather further information for the complete Family Declaration18. The social registry applies business rules programmed into the back-office software to calculate the PMT scores combining information from the Family Declaration and information curated from other government administrative systems. The registration process determines if the household is eligible for the TSA program, the child allowances and to other local benefits which all apply different eligibility thresholds. Households whose score qualifies for TSA are enrolled; when a household is deemed eligible, the SSA formalizes the enrollment decision and calculates the benefits that would be paid to that household depending on their PMT score and household composition (with additional benefits for children). The enrollment decision is then transmitted through the territorial SSA office to the Social Agent in the local SSA office, who then notifies the household of the decision. The Georgian regulation on registration to the social registry19 mandates a maximum of 90 days between application and registration into the social registry, which is done contextually with the attribution of the PMT score once all verification cross check have been completed. The enrolment shall be completed within 2 months after the PMT score is assigned. The list of enrolled beneficiaries is shared on a monthly basis with Liberty Bank, on the 10th day of each month; Liberty Bank in turn proceeds to pay the transfers by the 15th day of the month. For instance, a household that applies in January, is registered and assigned the PMT score in April and will receive the first payment between the 15th of May and by the 15th of June. Interviews conducted by the WB team with local experts and consultants suggest that effective times are aligned with the regulation. Overall, then, from the moment a household applies to enrolment it takes about five months. Once the score is generated, it is not updated for 4 years once the recertification process will be conducted. This implies that changes in income, assets and other household vulnerabilities captured in the scoring formula are not reflected in an updated score despite the social registry has the potential to automatically update certain household characteristics on a regular basis. In this sense the social registry is currently inadequate for an adaptive social protection system: it does not automatically and regularly update important information from other government databases for faster eligibility and recertification checks. 18 WB assessment found out through interviews with SSA agents that three visits by social agents and beneficiaries to submit and verify a large amount of information, even when this could be retrieved directly by the SSA through the Civil Registry, as is the case for example with birth certificates 19 Govt. Decree N 126 of 2010 and its amendments. Page 32 of 153 Official Use The World Bank Georgia Human Capital Program (P175455) Technical Assessment The Covid-19 pandemic has revealed the urgency of transitioning to an adaptive model social assistance, where households and children in need are rapidly identified in the face of constantly evolving circumstances. The Covid-19 pandemics has highlighted the relevance of the TSA delivery system in supporting resilience to systemic shocks (Figure B13). The TSA and its scoring system was the main eligibility criterion of a large number of emergency related benefits 26.5 million GEL). However, the Covid-19 experience has also uncovered the limits of the use of PMT score as it is currently designed. Both the computation and the verification protocols, indeed, made it an incomplete instrument in the need to support the households that were the most affected by the mobility restrictions, such as informal workers and the urban self-employed. For these vulnerable categories, special benefits had to be introduced through the creation of an ad hoc government portal and a special benefit for those who were able to prove a reduction in their informally generated income. Figure B13. TSA expenditure and total coverage Page 33 of 153 Official Use The World Bank Georgia Human Capital Program (P175455) Technical Assessment Source: MoHILSA/SSA. While the Social Registry serves as an integrated and dynamic “gateway� for households to apply for potential inclusion in social programs, Georgia does not have an integrated SP reporting system to coordinate and track beneficiaries and benefits across programs and across central and local government. The existing social registry, - the main administrative tool of central administration (SSA) to deliver social assistance - is currently not connected with local (municipal) information systems and other government systems. Currently, data sharing on social protection delivery between municipalities and the central government’s SSA is systematized only in a limited number of major cities. For smaller municipalities there is no unified data sharing protocol to allow a mapping of SP delivery across the country. The decentralization of many social protection benefits and programs coupled with the lack of an integrated reporting system for social protection and human capital services more broadly makes it difficult to form a comprehensive picture of overall coverage, identify possible exclusion gaps and overlaps. For instance, the social registry provides a list of households who receive TSA and child allowances, but not a combined picture of whether these same households also receive what types of local benefits, IDP and disability allowances etc. Integrated reporting systems –or “Integrated Page 34 of 153 Official Use The World Bank Georgia Human Capital Program (P175455) Technical Assessment Beneficiary Registries�20 – would allow for monitoring and coordination of “who receives what benefits,� and for identifying intended or unintended duplications across programs. This informational gap is all the more relevant as municipalities have high autonomy and spend about 14 percent of their budget on their own social protection and healthcare programs, furthermore with high geographical variation and therefore very different gaps in social protection and healthcare benefits across towns and regions. The municipality of Tbilisi alone, for example, allocated an additional GEL 136 million to social protection expenditure in 2021 against GEL 287 million for 271 municipalities analyzed by UNICEF (2020), despite Tbilisi being the administrative region with the lowest poverty rate in the country. The Government partly addressed this through the geographically targeted benefits to residents of high mountainous settlements, but this does not fix the underlying need to fine-tune central government’s policy to the differing needs of different parts of the country. The geographical coverage of ALMPs and SESA is very unequally distributed. SESA, the agency in charge of implementing employment and labor programs, was established in 2019 with the mandate of developing employment services, taking over responsibilities from SSA Labor Department. It is, however, strongly underfunded, understaffed (51 employees for the full country in 2021), and does not fully cover poorer and remote regions. Three regions lack a SESA regional office (Racha-Lechkhumi and Kvemo Svaneti, Samtskhe-Javakheti, and Mtskheta-Mtianeti) and even where present, the scale of SESA’s activities is very limited and the provision is not equally distributed across regions (short-term vocational training, for instance, was only provided in the four regions of Tbilisi, Kutaisi, Zugdidi and Sachkhere in 2020). The employment service model is not sufficiently tailored to the needs of vulnerable jobseekers (TSA beneficiaries, IDPs, women, youth, PWD among others). The new employment model piloted with support of EU introduced a profiling tool to categorize the hard to serve, though existing job search assistance, intermediations services and counseling are not implemented to adapt to the higher needs of “hard to serve� jobseekers. Low-skilled and long term unemployed are identified through a profiling tool that classifies registrants in the employment service centers into four groups, of which “Group 3� 20 Lindert, Kathy, Tina George Karippacheril, Inés Rodríguez Caillava, and Kenichi Nishikawa Chávez, eds. 2020. Sourcebook on the Foundations of Social Protection Delivery Systems. Washington, DC: World Bank. doi:10.1596/978-1-4648-1577-5. Page 35 of 153 Official Use The World Bank Georgia Human Capital Program (P175455) Technical Assessment comprises the low skilled and long term unemployed and group 4 comprises people living with disabilities. The scope and scale of ALMPs is very small to help promote human capital utilization among vulnerable jobseekers (TSA, IDPs, youth, women, PWD and other vulnerable persons). ALMPs managed by SESA include programs for employment promotion (subsidized employment for vulnerable21, job matching through Worknet, counseling, job search assistance, job fairs) and programs for jobseekers’ skills development (provision of vocational guidance and career planning, development of jobseekers’ core competencies, internships22 and vouchers for short term professional training/retraining/upskilling with providers accredited with MOES23)24. Evidence on the effectiveness of existing ALMPs is limited, due to 21 Under wage subsidy programs, vulnerable beneficiaries (youth, those with PMT score less than 100,000, IDPs, PWD) are provided skills assessments, intermediation services, and wage subsidies for employment. The subsidies cover 50% of job remuneration up to GEL 560 per month, for up to four months, with the requirement that contract is extended by the employer for at least six months after the end of the subsidy program. 22 Internships are three months long and require the provision of an internship stipend of GEL 200 per month. Employers participating in the scheme are mandated to hire all “successful� interns for a minimum period of six months after the successful completion of the internship. 23 Short term vocational training is provided by reimbursement of the fee of short-term vocational education training courses for job seekers in occupations which are in demand on the labor market of Georgia. Trainings offer a good mix of theoretical and practical classes, on average, over 12 weeks (112- 136 hours). Upon completion of the courses, successful trainees receive certificates which are formally recognized by the state. Eligibility is universal, but preference is given to special categories (e.g. people with special educational needs, members of ethnic minorities, etc) and those with low PMT score (less than 100,000) or low wage (less than twice the minimum wage). 24 Some employment services and ALMPs are provided at the local level with limited coordination or information exchange with the central authority. The Republic of Adjara, for instance, has allocated GEL 445,000 for 2021 to the Adjara Employment Agency (AEA). The municipality of Tbilisi runs a vocational training program that covered about 4,000 people in the 2020-2021, with a budget of GEL 660,000 in 2020 and a budget of GEL 180,000 in 2021. While the autonomous initiative of local authorities can obviate to the shortcomings of the centrally administered services, however, it might come with the unintended consequence of lowering the pressure for a scale up of the services delivered by SESA in Page 36 of 153 Official Use The World Bank Georgia Human Capital Program (P175455) Technical Assessment the low coverage of SESA’s employment services. The total number of beneficiaries receiving ALMPs is 4,300 in 2021 (as of end of October 2021): 1,585 received individualized counseling and job search assistance services, 1,754 were matched/referred to employers, 757 participated in short term professional training, 136 were trained in core competencies training, 59 participated in internships25 and 7 in subsidized employment (wage subsidies). In terms of job placement: of the 1,806 beneficiaries of job intermediation services in 2020, only 343 eventually found a placement, and among them only 69 were placed with the employer they were referred to. Overall, 809 registered unemployed with SESA found a job through the Worknet job matching portal and employment services. Employer’s participation in ALMPs is low: As of October 2021, only 702 employers were registered with Worknet posting a total of 8,321 vacancies. Currently SESA does not provide nor refer to programs promoting self-employment and livelihood support specifically targeted to vulnerable and poor jobseekers in rural areas. The geographical coverage of training providers is very narrow adding a further access constraint to rural beneficiaries. Until 2019, the training and retraining supported by SESA was separated from formal VET provided by MOES: an adult that wanted to undertake formal VET training would join the same classes with young students, leading to high drop-out rates among adults. Since 2019, a new system of short-term training and retraining was introduced by MOES along with competence standards, quality assurance mechanisms and an accreditation system for training providers. The capacity of the short- term vocational training includes about 212 vocational training or retraining courses for about 3,000 trainees per year delivered by about 100 legal entities26 , half of which are private companies and NGOs, and VET colleges is the other half. However, the geographical coverage is very narrow: in 2020, it was limited to four regions (Tbilisi, Kutaisi, Zugdidi and Sachkhere). Before the pandemics, 70% was the poorer regions. These data can only be obtained with difficulty and are not stored centrally. SESA is therefore not able to base its strategic decisions on a complete picture of the geographically dispersed provision of employment services. 25 Internship programs have been limited, with 19 beneficiaries in 2020 and 59 between January and October 2021. In 2018, however, the programs showed relatively good success as 79 of the 188 beneficiaries were subsequently employed 26 5 institutions of higher education, 5 NGOs, 40 VET colleges and private companies such as iHilton/PS retraining courses that are funded by the government. Page 37 of 153 Official Use The World Bank Georgia Human Capital Program (P175455) Technical Assessment employment rate of short-term training courses, while 62% was the employment rate among VET diploma courses students. The quality (earnings) and job tenure are not tracked. The potential of SESA to activate TSA beneficiaries into the labor market is a strategic priority for the GoG. Within TSA beneficiary households, around 145,000 individuals are estimated to be “activable� (work-abled, inactive not in school without caring duties) based on administrative data, which represents around 15 percent of the working age population currently out of the labor force27. Evidence of TSA’s disincentives on labor participation is weak28. Income disregards were introduced in 2019, exempting TSA beneficiaries whose labor earnings increase within a certain threshold from the re- assessment of their PMT score to build in incentives to work. In 2017 the GoG has also mandated the registration with the job portal Worknet of work-able TSA beneficiaries. More than disincentives embedded, there is lack of tailored employment services and support for livelihood and income generating activities to support sustainable graduation of TSA. The Worknet jobs portal is assessed as a basic platform, offering only basic information and involving few employers and vacancies compared to the number of jobseekers. Georgia has made significant progress over the past couple years in developing concrete tools and approaches for promoting activation of social assistance beneficiaries. One key tool is the development of a management information system known as “WORKNET.GOV.GE� used by job counselors in SESA local offices. This portal allows employers to advertise vacancies, and job seekers to register their skills and experience to support job search. The system is free of charge and can be accessed online or via kiosks in the SESA regional offices. However, Worknet is assessed not to be user-friendly due to complex registration 27 “Inputs for the Development of a Strategy for Activating Targeted Social Assistance Beneficiaries in the Republic of Georgia�, Gonzales 2018. 28 An impact evaluation of TSA’s impact on labor supply commissioned by UNICEF (2020) finds that TSA eligibility significantly discourages labor market participation among TSA beneficiaries who rank at the margin of eligibility. The study estimates that TSA beneficiaries around the threshold forego nearly one sixth of their household’s net income in order to avoid losing TSA eligibility status. However, the perspective of losing eligibility status is very high for households whose score is close to the threshold and declining for the majority of TSA beneficiaries who are well within the threshold. Using a different approach, Carraro et al (2020) indeed find little evidence of discouragement in a quasi-experimental analysis of 2015 data, assessing the effects of an exogenous change in eligibility criteria on employment. Page 38 of 153 Official Use The World Bank Georgia Human Capital Program (P175455) Technical Assessment procedures for both jobseekers and employers. The taxonomy of occupations and skills used in Worknet29 is not disaggregated enough to aid job matching, which is performed manually by SESA staff. Finally, a limited number of vacancies is collected also due to limited outreach efforts to employers. About 300,000 job seekers have registered in Worknet since its launch (not all of them active with regularly updated profiles); in 2020 only 3,000 vacancies were posted (mostly due to the pandemic crisis) and 908 jobseekers were successfully matched to jobs. In 2019, 10,000 vacancies were posted and more than 1,000 were successfully matched. About 700 employers are cooperating with WORKNET, with 235 registered with webpages. Attracting a larger number of jobseekers (beyond the lower skilled and TSA work-able beneficiaries who are mandated by law to register in Worknet) and integration with other job portals will make the portal more attractive to the employers who might use it to post new vacancies and ultimately lead to better job matching. One of the main labor market problem facing Georgia is the underutilization of human capital. It has two facets: high unemployment, also among highly educated workers, and low-productivity employment of highly educated workers30. There is a large number of workers with tertiary education, but jobs requiring tertiary education are few. Overeducation and over-skilling reflect scarcity of productive job opportunities in Georgia. Many highly educated workers are trapped in low-skilled jobs. Consequently, the returns to education are low in Georgia, about 7 percent for each additional year of education 31, significantly lower than in other economies of Europe and Central Asia. This represents a loss from the perspective of both individuals, and the society as a whole. Moreover, the scarcity of jobs requiring higher education leads to the crowding out of less educated workers, often pushing them into unemployment. Skills shortage is one of the biggest problems affecting Georgian employers32. For many employers inadequately educated workface is a major constraint to their operations. Employers find it difficult to hire workers with the right skills. It is not only technical, job specific skills that workers lack, but also 29 At ISCO and NACE 3 digit levels. 30 Rutkowski and Honorati , 2021. 31 Patrinos and Montenegro 2015. 32 WB Enterprise Survey 2019. Page 39 of 153 Official Use The World Bank Georgia Human Capital Program (P175455) Technical Assessment higher-order cognitive skills (such as critical thinking and problem solving), and socio-behavioral skills (such as leadership and initiative). The skills shortage has an adverse impact on firms’ performance and limits their growth prospects. To address the skills-shortage challenge a new Skills Agency has been established in 2021 with the mandate to define vocational education standards through the collaboration and shared duties among the public and private sectors. Specifically, the agency is tasked to foster the provision of job-relevant skills by upscaling and diversifying vocational trainings among public providers and to stimulate the private training provision (with a focus on job creating sectors) closely with sectoral employer organizations; this includes the need to upgrade the selection, delivery methods and curricula of short- term training and to tailor them to the needs of vulnerable groups. SESA lacks a methodology to identify skills in shortage and for medium term forecasting. The reporting LMIS (labor market management information system) in the Ministry of Economy provides only aggregate statistics on labor market trends, including on wages and employment by regions, economic sectors. The information is not disaggregated enough to aid SESA perform its job matching services. To identify the skills in demand SESA relies on information from jobs.ge, ss.ge and information provided by employers to SESA by email, in addition to Worknet. Currently neither the ministry of economy not SESA perform analysis for skills forecasting and to identify skills in shortage (Budget support from the European Commission is supporting this area). Limited coordination mechanisms exist between employers and the VET system to adapt vocational training courses to the skills needed by employers leading to poor job relevance of the vocational training system. To address the identified challenges, the Program comprises a set of key reforms and excludes some other critical SP reform because they did not satisfy the four defining criteria: (1) transformative potential and strategic relevance to the GoG; (2) their readiness for implementation and ability to deliver impact within the lifetime of the project; (3) ability to convey human capital investment towards those with the highest returns (in particular the children and youths); and (4) reliance of proposed reforms on solid technical foundation. The above criteria are the rationale why other critical SP reforms such the disability assessment and the potential introduction of an unemployment insurance scheme currently considered as part of the Georgia Social Code development are not considered within the boundaries of the Program. The reform of social case management through a revision of the role of social agents in providing referrals carries high potential for development, but it would require extensive Page 40 of 153 Official Use The World Bank Georgia Human Capital Program (P175455) Technical Assessment additional analysis and is not the most immediate bottleneck to human capital growth in the short to medium term. Reforms of social protection for IDPs and refugees, which is a high priority due to the large number of applicants and the political economy of the country, have been excluded due to the difficulty to ensure the delivery of results during the Program. The pension system reform is not urgent: the GoG has recently introduced a contributory pension scheme and revised the indexation rules by introducing elements of automation in the definition of benefits. More will probably need to be done as the changes in the indexation rules kick in, but the required changes are not a key bottleneck to human capital in Georgia and are rather incremental than transformational. Page 41 of 153 Official Use The World Bank Georgia Human Capital Program (P175455) Technical Assessment Institutional Arrangements Adequate institutional and implementation arrangements are in place to implement the PforR Program. The Technical Assessment reviewed the institutional arrangements to achieve the DLRs. The assessment concluded that the basic institutional and implementation arrangements for multisectoral and multilevel programming are in place for the long-established departments and agencies involved from the MOF, MOES and MOILHSA. The new agency SESA belonging to MOILHSA requires capacity improvements, investment in information systems and stronger incentives. The recommendations are incorporated into the PAP (Annex 6). To ensure smooth and regular information flows across units and ministries, the Program will require close partnership between the MOF which leads the Program, together with MOILHSA and MOES. Other ministries such as the Ministry of Regional Development would have a smaller role for some activities. With the Program following the Government’s Budget and implementation procedures, there is not a need for a full-scale Project Management Unit (PMU) as is often the case with IPF (Investment Project Financing). Rather, there would be a Program Coordination Unit (PCU) established under and headed by the IFI Division withing the MOF Debt Department. This PCU will have focal points from the line ministries. It is proposed that at the initial stage of the Program the focal points be the key personnel from the existing PMUs managing the WB’s financed project under MOES (PMU for I2Q) and MOILHSA (PMU for Covid-19). For oversight of the functioning of the PCU, there would a high-level steering committee headed by the Deputy Minister in charge of the IFI Division in the Ministry of Finance, together with Deputy Ministers from the line Ministries. Implementation processes: Each sectoral ministry (MOES and MOILHSA) will have the ultimate responsibility for the financial management and of the activities they implement, as in the existing government arrangement. The main processes involved would be (i) maintaining the existing FM and Procurement procedures and regulations for the program agreed with the Bank as being suitable for PforR financing modality; (ii) Facilitating the fund flow from the Bank trough the existing channels to the ultimate beneficiaries; (iii) meeting the reporting requirements regarding the Results on which the Program is based, according to verification protocols which will be specified for each Result; (iv) arranging the program auditing (both internal and external audit) and (v) Acting as the contact representative with the WB on the program implementation. Page 42 of 153 Official Use The World Bank Georgia Human Capital Program (P175455) Technical Assessment Budgeting and Planning: The budget of the program is included in the budget of MOES and MOILHSA for their respective activities. The program budget and planning is prepared and approved on an annual basis following the budget procedures for ministries in GoG. There are line items (six digit or eight digit codes from the Program Budget of the Government of Georgia) that would constitute the program expenditure framework. These would be used to define the boundaries of the required audit function. The financial terms are to be discussed further, as well as details such as the allocation in the first year and the extent to which disbursements would be advances or reimbursements. Results monitoring for this PforR operation relies on three core elements – investments, inter- operability and continuity. The Georgian Government has recent and ongoing investments in digitization in relevant government functions that will support better results monitoring for the entire Government Program, much beyond the PforR Program – the recent advances and the plans to be implemented are outlined in the forthcoming “Long-Term National Strategy for the Development of Digital Economy and Information Society and Its Implementation Plan.� This plan includes the creating of the new Information Technology Agency within MOILHSA and EMIS Action Plan. Program investments in connectivity including schools, primary health care centers, educational resource centers, learning management systems and electronic medical records, are examples of interventions that will support the generation of data that goes into the monitoring and evaluation of results. Inter-operability of systems, through the adhesion to open-source standards ensures “future-readiness� of the digitized systems and dashboards being created. Indeed as data becomes ubiquitous and streams of “Big Data� are generated, the Program includes the foresight of using Artificial Intelligence (AI) to process the data, a measure that is bound to become increasingly common. Continuity in the monitoring of results is provided by the Program including continued participation of Georgia in International Large-scale Student Assessment (ILSA) studies and the availability of Household Survey data from Geostat, the National Statistical Agency of Georgia. Monitoring and Evaluation (M&E) capacity at MOES and MOILHSA is well-developed and the WB has previous experience relying on the M&E arrangements under the I2Q and COVID Emergency projects. Intermediate and final outcomes related to social assistance efficiency can be monitored relying on the TSA database and can be complemented by the use of the Household Survey (IHS), implemented by GEOSTAT, which captures receipt of social benefits including TSA in a nationally representative survey run every year. Strengthening the cooperation between SSA and GEOSTAT, in particular to ensure that Page 43 of 153 Official Use The World Bank Georgia Human Capital Program (P175455) Technical Assessment HIES survey measures the variables entering the eligibility TSA score, would be a valuable improvement to the current monitoring arrangement and has been highlighted by SSA. Intermediate and final outcomes related to the improved inclusion and labor force participation among vulnerable groups will be monitored through SESA monitoring system and the development of an Integrated Reporting System for Social Protection (IRSSP) supported by the project. Connectivity between central and local social registry is challenging (local registries being mainly paper based) and almost non-existent. Connecting the local and central information systems is an outcome of the Program and would constitute a key improvement to the monitoring of social assistance delivery, and the analysis of its efficiency. SESA’s capacity to perform quality M&E reporting is satisfactory, but the amount of data it has access to is very limited. In its first two years of existence, SESA has accurately reported number of beneficiaries per program, number of firms engaged in ALMPs, number of worknet registrants that were referred through the portal for the delivery of emergency transfers. However, the low capacity of worknet is a concern and is an activity supported by the Program in its own right. Limited staffing at SESA made it fall short of its mandate of using Worknet as an overarching labor market monitoring tool. Substantial investments in human resources will therefore be needed to accurately categorize and regularly report beneficiaries of ALMPs under the Program. The Integrated Reporting System for Social Protection will require adequate governance arrangements for the management of the IT systems (under MOILHSA’s IT Agency) and the use of IT system for business intelligence reporting and analysis, which will be led by a newly established M&E unit within SSA. The main responsibility of the M&E unit will be to monitor the progress in the implementation of strategies, policies, projects. The main focus is on defining and understanding indicators (including base line and targets, data sources (internal, external, surveys), and prepare regularly progress reports. Other important areas are open data and reports and statistics for other governmental or international organization. The M&E unit should be the single exit point of social protection statistics. The M&E unit is recommended to include statisticians able to use advanced Excel formulas and statistical packages (like SAS, SPSS, Stata) and statistical language like R and Python. The M&E unit will be the main interlocutor and beneficiary for/of surveys gathering business request and receiving raw data from surveys. The M&E Unit will also estimate indicators based on the raw data and data from administrative database). The M&E Unit will have also to be involved in forecasting and counterfactual analysis to support policymaking. The M&E Unit will have to work very closely with IT Unit especially the IRSSP team (who will manage the large volume of individual data). The M&E Unit will prepare requests for aggregated Page 44 of 153 Official Use The World Bank Georgia Human Capital Program (P175455) Technical Assessment data from IRSSP and will be the main user of advance tools like report generators for extracting regular statistics for IRSSP. The institutional arrangement of a light footprint PCU housed at the Ministry of Finance with trouble- shooting of problems will help set-up a permanent results culture in Government. There has been a previous intent for results-based budgeting in the Ministry of Finance, since Program Based budgeting was introduced in 2012. While the Program Based budgeting does have numerous advantages (for instance, it supports collaboration between hierarchically diverse government agencies working on a particular program), in the case of Georgia, it appears that the results part was not entirely consistent with supporting high performance. For example, the Program 32 02 11 (“My First Computer�) which had a budget of GEL31.35 million (approximately USD 10 million) in the school year 2020-2021 (further increased to GEL43 million the next year) simply states that all first graders will be provided with the computer equipment, but there is no attendant action to make sure that those computers are used adequately to enhance learning33. In the case of the PforR, with a comprehensive view of Human Capital development, issues like this come in sharp focus. With results tied together in a package of reforms for a long-term learning objective (in the case of Program 32 02 11), it is expected that there will be significant institutional learning for a results-oriented functioning of government agencies. Borrower Commitment The commitment to the Program comes from the highest level of government and goes through to the frontline of delivery because of the perceived importance of human capital reform. The Government is currently finalizing a set of ten year sector strategies for 2021-2030 for all the key sectors of the 33 The idea of providing new computers to 1st graders is a clever one as students get to keep their computers when they move to the next grade, with every new entering cohort of 1st graders getting new computers. The budget document does not mention that the computers are netbooks called “bukis� which by definition are meant to work on the net, typically through wifi. But most Georgian schools do not have adequate Wifi for all students, thus severely limiting even the theoretical usability of the bukis, leave alone the actual practice in classrooms (Source: “Smart, Digitally Enhanced Learning Ecosystems: Bottlenecks to Sustainability in Georgia�, Eka Jeladze and Kai Pata, Sustainability, 2018, 10, 2672; doi:10.3390/su10082672). Page 45 of 153 Official Use The World Bank Georgia Human Capital Program (P175455) Technical Assessment economy, including the education, health and social protection sectors. The Program design is aligned with these sector strategies as the Government looks to Bank support on technical and institutional matters in addition to the Bank financing to help operationalize the strategies. The main guiding principle for selection of the Program elements is the imperative for transformative human capital reform. Georgia has a good track record of human capital, which has been supported by the Bank through technical assistance and advisory services, IPF and DPL loans. The proposed Program is the first PforR operation, which the Government has specifically selected as it seeks to find higher value for money from its human capital investments. The commitment is strengthened for all stakeholders, because of the policy linkages to the application for EU membership, which enjoys broad political support in Georgia. Page 46 of 153 Official Use The World Bank Georgia Human Capital Program (P175455) Technical Assessment Figure B14:PROPOSED IMPLEMENTATION ARRANGEMENT Page 47 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment The implementation arrangement diagram intends to show the structure following two principles – the blue lines are the regular function of the government – using government methods for financial management (including flow of funds, reporting, auditing) and procurement. There is no transactional level engagement with the Bank as there is in IPF projects. Rather there is a lean mechanism for monitoring of results shown by the orange lines – with a small project coordination unit (that also would include a part of the existing IPF PMUs in MOILHSA and MOES). The PCU function rotates around following up on results with the line ministry agencies and reporting back to the Deputy Minister of Finance for IFIs about the Program results. Page 48 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment C. Program Expenditure Framework The fiscal outlook that provides the setting for the Program Expenditure Framework has been impacted by the COVID-19 crisis, but recovery appears to be active as of October, 2021, with GDP growth rate for 2021 projected at 8%. According to the ECA Economic report “The fiscal deficit widened by 27 percent y/y in the first seven months of 2021, as rising public expenditures—including additional COVID-19 response measures—offset a 15 percent y/y increase in revenues. By end- July, the fiscal deficit had reached about 4 percent of annual GDP out of planned 7.6 percent, while public debt fell to 53 percent of GDP from 62 percent as of end-2020. The fiscal deficit is expected to remain elevated at around 7.6 percent of GDP in 2021 before gradually declining as revenues recover and emergency spending subsides. The deficit is projected to narrow to about 3 percent of GDP by 2023 in line with the fiscal rule�. The Bank’s Georgia Monthly Economic Update for October, 2021 provides further updated context: “Public debt fell to 53 percent of GDP in August, as compared to 62 percent as of end-2020. The first draft of the 2022 Budget law was submitted to the parliament in October. It envisages a deficit of 4.8 percent of GDP as continued robust revenues (mostly VAT and profit tax) and cuts in COVID-19 related spending are expected to offset spending increases from pensions, in line with the new pension indexation rule, and higher teacher salaries.� (“Competition and Firm Recovery Post-COVID-19� Europe and Central Asia Economic Update (Fall); and Georgia Monthly Economic Report for October, 2021). Program Based Budgeting, implemented in Georgia since 2012 provides a high level of budget clarity and predictability, with a clearly defined Human Capital Program Boundary explained in this Expenditure Framework. The budget defines a program of activities together with assigned expenditures and related results. Georgia meets the five requirements of an adequate expenditure framework that would allow a smooth functioning of a PforR Program: (i) the fiscal context of the Program in terms of the fiscal outlook and Program impact on the outlook; (ii) financial sustainability and predictability of resource availability; (iii) allows the alignment of policy objectives, priorities and results with the budget allocation and execution; (iv) creates mechanisms and incentives supporting an efficient service delivery and value for money; Page 49 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment (v) provides an affordance to vary assessment according to the size and complexity (the Program does this through a standard chart of accounts with 2, 4 and 6 digit classification of sub-programs and activities within each sub-program). Table C1 details the composition of the USD 5,988 million Program Boundary for the Program Period 2022-2027. The Program is defined by the selection of specific human capital sub-programs as defined by the budget codes. These sub-program are the ones that provide fiscal resources for implementing agencies to implement the Program. Detailed description of each budget code is provided in Annex 1, and Annex 2 provides data on spending in available expenditure categories. Column E in Table C1 indicates the mean annual amount that has been spent (allocated in case of 2021) in the three year period 2019-2021 for each sub-program code. With continued stable allocations being a Government priority, Column F indicates the six year total of the planned Program outlay in million GEL and million USD. Table C1 : Expenditure Framework including past record and planned expenditures for Program Column A Column B Column C Column D Column E Column F Annual Program Expenditu Boundary re Main Total Implementi (2019- Expenditures ng Agency 2021) 2022-2027 Budget (GEL (USD Theme Expenditure Categories Code (GEL m.) m.) m.) Results Area 1: Efficiency for quality improvements General Education Transfers to schools (vouchers) 32 02 01 MOES 799.79 4,799 1,600 Page 50 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment Column A Column B Column C Column D Column E Column F Annual Program Expenditu Boundary re Main Total Implementi (2019- Expenditures ng Agency 2021) 2022-2027 Budget (GEL (USD Theme Expenditure Categories Code (GEL m.) m.) m.) Protection of the general health of the Labor expenses, medicine, medical MOILHSA/N population supply, consumables 27 03 01 HA 828.67 4,972 1,657 Targeted Social Assistance MOILHSA/S (TSA) Cash transfers 27 02 02 SA 784 4,704 1,568 SSA current Labor expenses, management of MOILHSA expenditures social programs 27 01 04 /SSA 17.4 104 35 Software upgrade, Piloting design, SSA Training, increase of non-financial MOILHSA digitization assets 27 01 10* /IT Agency 0 18 6 Results Area 2: Inclusion 32 01 02 MOES NAEC 5.80 Strengthening Rehabilitation of schools for extra 32 02 13 TPDC 10.90 High Schools students; Unified National 01 15.00 and reform of Examinations Labor expenses for 32 02 01† 16.72 University experts supporting curriculum Entrance reform; 32 02 02 13.15 32 04 01 55.64 780X 260 Page 51 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment Column A Column B Column C Column D Column E Column F Annual Program Expenditu Boundary re Main Total Implementi (2019- Expenditures ng Agency 2021) 2022-2027 Budget (GEL (USD Theme Expenditure Categories Code (GEL m.) m.) m.) 32 07 01 Provision of medical services to the population in Labor expenses, medicine, medical MOILHSA/N priority areas supply, consumables 27 03 03 HA 197.77 1,187 396 Rehabilitation and equipping of medical institutions Equipment and hardware 27 04 MOILHSA 6.25‡ 38 13 SESA administrative Labor expenses, cars and office expenditures equipment 27 01 08 SESA 0.79 14 5 27 05 01 † SESA ALMP Training, internships, job matching, delivery placement 27 05 03† SESA 13.28 80 27 Results Area 3: Connectivity Page 52 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment Column A Column B Column C Column D Column E Column F Annual Program Expenditu Boundary re Main Total Implementi (2019- Expenditures ng Agency 2021) 2022-2027 Budget (GEL (USD Theme Expenditure Categories Code (GEL m.) m.) m.) 32 01 04 12.74 Digital Purchase of hardware including 32 02 11 35.49 Education computers for teachers, students, (funded) internet 32 13 MOES 14.98 750T 250 Digital Labor expenses and training for 32 02 15 Education (to school community; developers and be funded) hosting 32 02 16 MOES 15* 5 Digital Medicine, Admin Labor expenses, hardware and systems software, developers and hosting 27 03 02 MOILHSA 102.43 450] 150 Soft. devt, training, hardware and MOILHSA SSA IRSSP licenses 27 01 10* /IT 0 18 6 MOILHSA SESA Worknet Worknet 27 01 10 /IT 0 36 12 Total 2930.8 17,965 5,988 Notes: * No current allocation; † Part; ‡ Spending in 2019; X about 10% increase from 3 year mean; T Represents 100% increase from 3 year mean; ]Represents about 1/3rd decrease from 3 year mean; Page 53 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment Three flagship sub-programs in education, health and social protection account for 81% of the projected Program outlay of USD 5,988 million in the period 2022-2027. The Program budget for 2022-2027 summarized in Table C1 includes allocation primarily for reforms of existing activities, which constitute the major outlays of the Program. The three main budget codes are related to the Provision of general education (32 02 01), Targeted Social Assistance (27 02 02) and Protection of the general health of the population (27 03 01). These three programs in recent years have had annual budgets of a little over GEL 800 million (USD 267 million) for each of the three programs; or a total annual outlay of a little over GEL 2400 million (USD 800 million). It is very important to understand that the Program does not envisage substantive changes to these amounts. Indeed, efficiency is a key result area of the reforms envisaged under the Program. The same magnitude of expenditures (with adjustments for future inflation) is projected in the future six years of the Program, for a total amount around (USD 800 million x 6 = USD 4800 million). This figure of USD 4,800 million constitutes 81% of the projected Program expenditures (see Table C2 below). Reforms supported under the Program are expected to enhance the value-added or throughput of human capital benefits derived from the Program, so that higher quality and equity will be made possible without increases in budget outlay in the three programs (last column of Table C2). Table C2: Sub-Programs ranked by Relative Size Sub-Program Expenditure Cumulative Additional 2022-2027 Spending Spending in USD Percentage to 2019- million 2021 projected in USD million 1 Protection of General Health of the Population 1,657 28% 0 2 Transfers to General Education Schools 1,600 54% 0 3 Targeted Social Assistance 1,568 81% 0 4. Primary Health Care (Health Care in Priority Areas) 396 87% 0 5. Strengthening High Schools and University Transition 260 92% 26 6. Digital Infrastructure in Schools (Digital Action Plan for 250 96% 129 schools) Page 54 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment 7. Digital Infrastructure for Health including Telemedicine 150 98% -55 8. Digital Infrastructure for Social Protection and other 108 100% ~55 items Sum Total 5,988 100% ~155 The Government is committed to funding all the activities that are required for the success of the Program, with sub-Programs and activities that require deepening of expenditures mostly related to deepening of IT investments required to modernize human capital delivery and monitoring systems. In addition to the 81% of expenses captured by the three flagship sub-programs, the next biggest sub- program is the one for Primary Health Care (27 03 03). This sub-program is projected with the same zero additional fiscal outlays (Table C2). The fifth ranked sub-program (in terms of size of 2022-2027 outlay) is the program to strengthen High Schools for implementing the new curriculum. The High School strengthening sub-program envisages an increase in outlay of from the three-year historical outlay by about USD 26 million over the linear projection from the previous three years. The Program is completed by a set of digital investments in human capital service delivery and administrative set-up costs of some relatively new Social Protection sub-programs. Table C2 indicates a possible additional fiscal impact of the Program to the tune of USD 155m (without accounting for future inflation). Given the Georgian Government yearly budget in the recent 3 years of approximately US$ 6 billion, the additional outlay would account for a fiscal footprint of approximately 0.43% annually. Budget execution as compared to the budget allocation indicates a mature budgeting process in Georgia, supported by the Program Budget in place since 2013. Georgia has made a substantive effort over more than the past decade to strengthen and make transparent the budget processes. This effort has been supported by a series of DPLs as well as through Public Sector Financial Management Reform support. The Public Expenditure and Financial Accountability Assessment report published in 2018 stated “Overall, the results of the PEFA assessment show that public financial management systems in Georgia are strong and improved as the PFM reform action plan has been implemented. The aggregate expenditure side of the budget performs according to plan. There is an impressive array of information regarding the finances of the budgetary central government. Information is included in the budget on a timely basis. As a result, the budget documents include most of the basic, and much of the supplementary information, required to support a transparent budget process.� The budget has high degree of predictability and there is a close match between the budget and actual spending, as shown with data for the Human Capital sub-Programs in Table C. Indeed, as discussed before, for the health Page 55 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment sector, there is a problem of over-spending on the budget, that the reforms supported by the Program aim to tackle. Table C3: Budget absorption for Human Capital (Historical record of past 5 years) Education Health Social Protection Budgeted Budgeted Budgeted (current (current (current Executed % Executed % Executed % thousand thousand thousand GEL) GEL) GEL) 2016 661,853 99.8% 964,722 102.5% 688,081 100.0% 2017 750,397 99.6% 996,257 102.9% 718,051 99.9% 2018 770,046 100.0% 1,073,929 102.2% 719,990 99.9% 2019 941,194 100.0% 1,157,506 103.2% 828,999 100.0% 2020 973,900 99.9% 1,683,445 99.2% 889,567 99.9% Incentive for successful implementation of the Program expenditure framework comes from a Government wide conviction about the reform, which will be further supported by the institutional arrangements. Georgia has put a priority on the development of human capital. The expenditures allocated for education, health and social protection are the biggest items of government expenditures. However, there has also been a realization that systemic or structural inefficiencies are holding back the productivity of these expenditures. The Program based budgeting system does include sets of targets and indicators, but these indicators for the most part only record that expenditures were made for the intended purposes. For instance, the monitoring and evaluation system records that transfers were made as intended to schools for provision of general education. However, the treasury system of incentives does not extend to long term policy goals of outcomes such as better student learning. The reforms that will be supported by the Program are expected to overcome these problems to bring about a more advanced level of results-orientation. The Ministry of Finance as well as the line Ministries are both interested in this government wide goal of better value for money from public expenditures and to meet the long-term strategic goal of policies and outcomes in line with Georgia’s aspirations to be a modern European state. The institutional arrangements, with an operational level Program Coordination Page 56 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment Unit and a Ministerial level steering committee is expected to support the provision of the required financing. The assessment estimated that the Program expenditure for social protection will amount to 1.7 billion USD over the six years of the Program. The estimate, reported in Table C1, is based on (1) a projection of costs related to currently executed activities, such as the TSA benefits, based on average past expenditure; (2) an assessment of planned expenditures in new items, such as the geographical expansion of SESA, scale up of ALMP, and the estimate of the costs of IT investments. The costing of additional intervention was conducted in consultation with MOILHSA and desk review of internal forecasting by MOILHSA. For SSA, this estimate comprises expenditure on social assistance transfers (budget code 27.02.02) and administrative management of the social protection delivery system, including human resources and software use (budget code 27.01.04). Aggregating all newly introduced items of expenditure under the Program, SSA’s increased expenditure is estimated to be negligible and below 1mln GEL over the duration of the Program. The estimate rests on the assumption that IT adaptation required to improve the TSA delivery system will be mostly undertaken by the MOILHSA IT agency... For SESA, total expenditure under the Program is estimated at 50.5 million GEL throughout the program’s duration, covering (1) increases in administrative costs (budget line 27.01.08) in support of the staffing of regional offices, recruitment of job and vocational counselors, purchase and maintenance of mobile outreach and user interface units, (2) scaling up of intermediation services and ALMPs (budget line 27.05.03) such as internships, wage subsidies, vocational training/retraining; (3) analytic activities related to the updated Worknet (budget line 27.05.01). Of these costs, it is expected that 33 GEL will exceed the regular budget, as expected based on the trends from the most recent years: it is expected that, by the end of the program, yearly expenditure by SESA will triple to achieve over 9 million GEL per year. Digitization activities under the Program will be led by the newly established IT Agency of MOILHSA, with the mandate to run and coordinate all IT related activities used by the Ministry. The IT agency-led activities are expected to cost 67.3 million GEL over the course of the Program. Of the total expenditure, 42.2 million GEL will be constituted of one-off, capital expenditure components (designing software for SP delivery systems, renewing the Worknet portal, establishing the IRSSP, overseeing software specification, performing initial adaptations, purchasing necessary hardware) and the remaining 25.1 million GEL will be made up of higher current expenditure needs throughout the six years (maintenance of software, monitoring and evaluations, regular software upgrades, training). Towards the end of the Program, regular operational maintenance associated to the newly introduced activities is estimated at 4.6 million GEL per year. Funding for the IT agency (under budget line 27.01.10 of the GoG’s budget) is yet to be determined due to the recent establishment of the agency. Page 57 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment Capital expenditure on Social Protection technological infrastructure under the Program will be substantive in 2023-2025, when most of the digitization activities will take place. Figure C.1 below breaks down predicted expenditures under the Program into capital and operational components, and plots them against historical trends in budget (“planned� expenditures in the absence of the Program). This is based on the conservative assumption that SSA’s and SESA’s administrative budget is rigid and any new expenses enter the agencies’ needs additively; in reality, to some extent, expenses will probably be re-directed from low-need items towards the Program. The extent to which this is possible depends on the budget rigidity, on the amount of earmarked expenditure items in the agencies’ activities, and on the fungibility of internal human resources across projects and tasks. Absent detailed information on budget rigidity, this expenditure framework assessment rests on the most conservative assumption that new activities will need to rely on new financing sources. Figure C1. Forecast Yearly Social Protection Program Expenditure, broken down into capital and operational expenditure, 2022-27. Page 58 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment Source: MOHILSA and authors’ calculations. Agencies responsible of implementing the social protection part of the Program have moderate to substantial financial sustainability and predictability of resource availability (substantial for SSA and moderate for SESA); it is therefore of primary importance that the Ministry of Finance (MOF) readily anticipate increased expenditures of SSA and SESA, especially in the first three years of the Program’s cycle. SSA’s administrative budget (27.01.04) has been consistently high over the last years, averaging 5.8 million USD per year between 2019 and 2021. The recent downward trend in expenditure on administrative costs is mostly attributed to the change in SSA’s mandate in 2019, when tasks were transferred away from SSA to newly established agencies were. MOF has been constantly willing to support SSA, as evidenced by the response to the Covid-19 pandemics which had SSA as one of the main delivery channels of emergency support. As far as SESA is concerned, the recency of the agency creates an inherent risk in its capability to obtain stable flows of resources in the face of significantly higher needs. SESA’s expansion of geographic coverage is expected to cost more than three times as much as what is currently allocated; delivery of ALMPs to the target number of 30,000 vulnerable people will cost 14.6 mln USD, implying a threefold increase in expenditure. On the funding side, planned budget accruing to ALMPs has been stable over the last three years, but has significantly grown in terms of administrative expenditures, reflecting the GoG’s commitment to expand SESA. In 2020, allocated budget to SESA’s administrative costs was 703,000 GEL. In 2021, the same figure rose to 1,213,000 GEL. This suggests strong sponsorship of the Program by MOILHSA and commitment on the MOF’s side to release resources to both SSA and SESA, which will have to continue especially in the first three years to support IT updates and upgrades. Rapid delivery of funds to these agencies will highly benefit from the scalable nature of the DLRs, which will guarantee smooth liquidity flows to MOF early on in the Program. Page 59 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment Overall budget resources absorption for the social protection part of the Program is high, with room for improvement for the newly established SESA which saw its initial steps during the Covid-19 pandemics. Budget absorption was measured for this assessment as executed spending over budgeted resources, before any amendments to the latter. SSA’s performance on this respect has been consistently high: budget absorption of the TSA program has been consistently above 94% in the last four years, achieving 100.2% in 2020 and a projected yearly 101.0% in 2021, based on expenditure in the first six months; management of social programs (code 27.01.04), supporting DLRs 1.5 and 1.6, absorbed 67.9% of budgeted resources in 2020 (when the Covid-19 pandemics caused an unexpected reduction in some variable costs, such as in-person verification visits by social agents to TSA applicants) and 92.2% on a yearly basis in the first six months of 2021. Concerning SESA’s activities, the picture is less reassuring: although budget absorption was 74% for code 27.05.01 and 93% for code 27.05.03 on average between 2017 and 2019, in those years the two codes were managed by SSA. In 2020, when SESA started managing the codes, budget absorption fell to 63% and 0.2% respectively. The latter figure is explained by the suspension of ALMPs following Covid-19 and the corresponding diversion of government expenditure towards emergency social assistance measures: in 2021 the delivery of ALMPs has resumed and is expected to improve throughout the year. As far as administrative expenses (27.01.08) are concerned, SESA absorbed 51.7% of budgeted resources in 2020 and a projected 72.6 percent in 2021 (based on data from the first semester). The diversion of funding away from SESA in the wake of the emergency is a concern but it is partly explained by the infancy of the agency at the time: once the agency grows and expands its user base, it will become more resistant to sudden changes in financing. Furthermore, despite the temporary diversion of funding, SESA’s human resources were fully utilized for the administration of the self-employment emergency benefits and the development of the portal for the self-employed. This increase in funding will be rationalized by efficiency gains across Government agencies and by the public good nature of the reporting capacity generated under the SSA’s guidance. Table C4: Budget absorption in SSA and SESA, historical values Planned Executed Absorption Implementing budget budget (GEL) rate Budget item Year agency (GEL) 2017 SSA 676,000 496,000 73.4% State Program for 2018 SSA 700,000 512,200 73.2% Development of 2019 SSA 700,000 526,700 75.2% Employment Promotion Page 60 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment Services (currently 2020 SESA 700,000 443,600 63.4% 27.05.01) 2021 SESA 770,000 - 2017 SSA 2,014,000 1,814,000 90.1% 2018 SSA 2,090,000 2,090,000 100.0% State Program for 2019 SSA 2,090,000 1,850,270 88.5% Jobseekers’ Vocational 2020 SESA 2,090,000 5,000 0.2% Development (currently code 27.05.03) 2021 SESA 2,090,000 - TSA transfers (27.02.02) 2018 SSA 680,000,000 640,661,800 94.2% 2019 SSA 770,002,000 741,286,100 96.3% 2020 SSA 793,000,000 794,687,200 100.2% 2021 SSA 816,000,000 412,466,800 50.5% Management of 2020 SESA 703,000 363,500 51.7% employment facilitation 1,213,000 440,600 36.3% services (27.01.08) 2021 SESA Management of social 2020 SSA 21,577,000 14,652,800 67.9% programs (27.01.04) 2021 SSA 13,480,000 6,214,500 46.1% The social protection component of the Program will require MOF’s championship to ensure alignment of policy objectives, priorities and results with the budget allocation and execution. The Program requires significant coordination of MOILHSA with other ministries. For instance, beneficiary-level data exchanges needed to integrate the social registry can only be put in place under authorization of a governmental decree. Strong political will is therefore necessary for the achievement of the DLRs: absent that, many of the intended outcomes will not be achieved. However, the GoG strongly embraces the reform and is found unlikely to change course over the next years: the package of reforms sustained under the Program is quite accepted across the different stakeholders and in civil society, and the financial incentives set up by the PforR financial arrangements will most likely encourage the MOF to be a sponsor for the Program through its six years. Page 61 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment Transformational reforms in Social Protection will have repercussions onto other public entities, that are however omitted from this expenditure framework assessment for ease of analysis. The MOF might have to meet higher demand from other public entities, in particular MOILHSA’s IT department, Geostat, Ministry of Regional Development, Ministry of the Economy or Ministry of Agriculture. MOILHSA’s IT Agency will face heightened demand internally as it supports (1) the digitization and streamlining of registration to the social registry, (2) interoperability of the Social Registry, Worknet and other IT systems with Civil Registry, municipal databases, and others, (3) Redevelopment of Worknet and interoperability with MOES and MoAgr databases and (4) Incremental development of Integrated Reporting System for Social Protection (IRSSP). Geostat will need to support the development of adequate instruments for skill analysis and forecasting. These agencies will then need to direct staff time towards the Program in the face of competing demand. An accurate assessment of the risks this poses would need to rely on the degree of rigidity of expenditure and workforce allocation in these agencies; absent these data, and in the assumption that flexibility is limited, the assessment highlights that the two agencies might need additional funding (and the MOF’s commitment) to help the achievement of the DLRs. To a lesser extent, interoperability will require that the partner agencies have the capacity to provide the required information regularly and in the required format. For instance, it will be requested that municipalities are able to fill – to the very least – data spreadsheets to report to the central government on the needed variables, and potentially even that they use the central government’s system and directly enter information into it. D. Program Results Framework, including DLIs Results Areas and Theory of Change Transformative change in the provision of human capital requires a concerted effort on multiple fronts, centered on the provision of resources, and the institutional structures and set of incentives driven by the MOF. The main problem that has held back Georgia in the quality and quantity of human capital has been ensuring continuity of reforms, and more importantly, pushing through on technical or politically Page 62 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment difficult reforms until they have achieved complete impact. Examples of reform that require prolonged attention include policies helping the unemployed to get jobs or for medicines to be affordable. Big ticket items such as fixing the financing model for medical care or reforming the financing of general education are particularly demanding. The general education financing model introduced nearly 15 years ago still dictates the provision of public resources to schools even as demographic changes have fundamentally altered the location where education services are needed, and larger economic and social changes have resulted in a shift in the basic objective of education from mere knowledge provision at the beginning of life toward lifelong competency development. Moreover, when a policy is not working and needs to be changed, in the absence of influential interest-groups, things remain as they are, like on preventive health care or improving social benefit targeting. These changes are some of the many fundamental or basic policy changes that comprise the activities of the PforR Program. Table D1 below provides an overview of the Theory of Change of the various activities that make up the Program. Result Area 1: Efficiency for better value of money of human capital expenditures – Strengthening the Human Capital Program by improving the efficiency of education, health, and social protection services. Efficiency for Quality Improvement as a result area in the GoG’s human capital program is tied together across the human development sectors because of common qualitative aspects and a common economic or financial model across subsectors. Efficiency regarding human capital has common qualitative aspects related to the human agent at the center – a student, a patient, or a TSA beneficiary. Ultimately, efficiency and efficacy are combined aspects of the quality-of-service delivery to all beneficiaries. The PforR selects critical subprograms whose quality delivery is essential for Georgia’s Human Capital. The distribution of resources to schools under the general education financing model accounted for nearly one-fourth of all human capital expenditures in Georgia, making this the single largest expenditure item in the Program. Similarly, the PforR focuses on improving public health care spending, close to one-tenth of the GoG budget, as well as improving the quality of care through better regulation, the introduction of clinical guidelines, and revision of the benefits package and performance- based payment system. Result Area 2: Inclusion for quality of human capital expenditures for all beneficiaries. Inclusion of all groups with a special focus on women to access good quality of public services is central to the “European State� vision driving the development strategy in Georgia. According to the EU, “The 20 principles of the European Pillar of Social Rights are the beacon guiding us towards a strong social Europe that is fair, inclusive and full of opportunity.� It can be argued that inclusiveness is central to the European identity, especially for Georgia, which seeks to translate that vision into policies, programs, Page 63 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment and actions. Inclusion is a central pillar of EU policy frameworks for human development. The Program seeks to ensure equality of access for children and adults from rural or geographically remote areas to quality human capital services whether it is ALMP, preschool/kindergartens, benefits, quality high school education, or health services. Inclusion also relates to socially vulnerable groups of the population, people with disabilities, migrants, long-term unemployed, ethnic minorities and social assistance beneficiaries that face difficulty “graduating� from the social safety net. Poor families who have lower education attainment, nutrition, and health outcomes require special attention to build their human capital and lift them out of poverty. Result Area 3: Connectivity of digital systems across all areas of human capital provision – Leveraging digital technology for development. Georgia needs to strengthen the digital capabilities of the population at the same time as making human capital service delivery more efficient and inclusive due to the deployment of digital technologies. The GoG has set up a commission to develop a Long-Term National Strategy for the Development of Digital Economy and Information Society. The goal of this strategy is to “to promote the development of digital services in the public and private sectors, enhance digital literacy, ensure cyber-security and personal data protection, step up the process of establishment of an information society, growth of hi-tech export, better the development of research and innovation, and upgrade Georgia’s competitiveness in the global digital economy.� For the GoG human capital program, the Connectivity results area seeks successful development, deployment, and maintenance of secure information systems and use of EdTech, Telehealth and a Digital Social Registry, and supports the development of the Digital Economy Strategy. The theory of change includes formulation of specific digital strategies for each of the human development sectors. These would be synchronized to the national digital development strategy. The national digital strategy would translate to investments and budget allocation decisions – for connectivity as well as acquisition of software and personnel for developing and maintaining services. These three result areas will also generate significant fiscal savings as described in the Economic Analysis. Potential higher earnings due to gains in productivity associated with education have a net present value (NPV) of approximately US$628 million under the Intermediate scenario as shown below in Table D2. Health savings associated with the program – mostly captured by the lower 40 percent of the population – add an additional US$276 million in benefits. Benefits from social protection add another US$443 million in benefits. Restructuring of the school network, the introduction of DRG payment systems, better primary health care utilization and pharmaceutical reforms are estimated to result in fiscal savings of USD 433m. The NPV of all benefits in the intermediate scenario is estimated at US$ 1,781 million. Page 64 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment Table D2: Projected Gains from the Program (Economic Analysis) Gains due to increased productivity and savings attributed to the Program (in US$) Social Scenario Education Health Protection Fiscal Savings Total 192,648,76 Conservative 313,923,491 1 339,611,706 427,533,574 1,273,717,533 276,279,76 Intermediate 627,846,983 3 443,230,543 433,685,189 1,781,042,478 364,884,60 Optimistic 1,098,732,220 6 546,849,380 439,836,804 2,450,303,009 DLIs, DLRs and rationale for selection: The PforR Program and Bank financing is composed of a set of four Disbursement-Linked Indicators (DLIs) relevant to human capital, and further comprises 22 Disbursement-Linked Results (DLRs) arranged by each of the three sectors of education, health, and social protection. The guiding principle for all the DLRs is the objective of bringing about transformational change. Table 5 below details the DLIs and DLRs for the Program. The text below describes the context and the policy rationale for the selection of each specific DLR, organized by each DLI. DLIs, DLRs and rationale for selection: The PforR Program and Bank financing is composed of a set of four Disbursement-Linked Indicators (DLIs) relevant to human capital, and further comprises 27 Disbursement-Linked Results (DLRs) arranged by each of the three sectors of education, health, and social protection. The guiding principle for all the DLRs is the objective of bringing about transformational change. Table D3 below details the DLIs and DLRs for the Program. Further explanation regarding the policy rationale for the selection of each DLR and the sector context is presented in the stand-alone technical assessment. Table D3: Disbursement Linked Indicators and Disbursement Linked Results Education Health Social Protection Page 65 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment Disbursement Disbursement Linked Result Allocation to DLIs and Linked DLRs Indicator (US$ million) DLI 1: DLR 1.1.1: By June 30, 2024, or such later date as may be established 10 Improvement by the Bank, MOES has: (i) carried out a study in at least three of Government Mountainous Regions, three Rural Regions, and three Urban spending Regions, to calculate the per student operational costs and financing efficiency requirements for Selected Educational Networks to provide the through Competency-based National Curriculum for Upper Secondary Levels; adoption and and (ii) prepared a report based on the findings of the study, implementatio detailing a General Education Financing Model, satisfactory to the n of Bank, including criteria for adequate quality of educational inputs appropriate and efficiency of government spending administrative DLR 1.1.2: By June 30, 2025, or such later date as may be 30 and financing established by the Bank, all General Education Schools in Georgia mechanisms receive financial transfers based on the General Education Financing [US$ 170m] Model specified in DLR 1.1.1 DLR 1.2.1: By December 31, 2024, the DRG Payment System has 10 been functional for at least 12 consecutive months in a manner satisfactory to the Bank, for at least two Service Groups DLR 1.2.2: By December 31, 2026, the DRG Payment System has 30 been functional for at least 12 consecutive months in a manner satisfactory to the Bank, for at least five Service Groups DLR 1.3: By December 31, 2026, or such later date as may be US$7.5 million equivalent established by the Bank, Managed-entry Agreements, satisfactory to for every year during Bank, have been entered into between the Borrower, through a which the DLR has been legally authorized body, and pharmaceutical providers or suppliers achieved [30 m total] for at least 50 percent of all new single-source medicines introduced in Georgia during each year starting with calendar year 2023 DLR 1.4: By December 31, 2023, or such later date as may be 20 established by the Bank, Borrower, through a legally authorized body, has adopted regulations or equivalent legal binding documents, satisfactory to the Bank, to introduce reference pricing Page 66 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment Disbursement Disbursement Linked Result Allocation to DLIs and Linked DLRs Indicator (US$ million) for the purposes of setting or negotiating prices of publicly financed medicines covered under the UHC Program DLR 1.5.1: MOILHSA has adopted a decree, satisfactory to the Bank, 10 requiring a pilot that revises the targeting formula used to identify poor families with children, determines their eligibility for poverty- targeted human capital benefits, and provides benefits under the revised formula DLR 1.5.2: MOILHSA has adopted a decree, satisfactory to the Bank, 10 to implement nationally the revised targeting formula, after completion of the pilot referred to in DLR 1.5.1 DLR 1.6: MOILHSA, through SSA, has completed the digitization of 20 Select Social Benefit Processes resulting in a reduction of the benefit processing time from application to submission of payment orders from five to less than two months and increasing SSA’s ability to calculate the Vulnerability Score on a quarterly basis DLI 2: Adoption DLR 2.1.1: By December 31, 2023, or such later date as may be Subject to a minimum of and established by the Bank, National Authorization Council for General 100 high schools and implementatio Education has provided the authorization to operate High Schools maximum of 300 High n of based on the new Competency-based National Curriculum for Upper Schools authorized; appropriate Secondary Levels US$10,000 equivalent per quality school for the first 100 standards for high schools; US$45,000 human capital equivalent for the next services 200 High Schools [10m enabled across total] all service DLR 2.1.2 By December 31, 2026, or such later date as may be US$100,000 equivalent delivery established by the Bank, National Authorization Council for General per school for each institutions Education has provided the authorization to operate to at least 600 additional High School [US$ 140m] High Schools based on the new Competency-based National authorized above the 300- Curriculum for Upper Secondary Levels High School result Page 67 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment Disbursement Disbursement Linked Result Allocation to DLIs and Linked DLRs Indicator (US$ million) referred to in achievement of DLR 2.1.1, with a maximum of 300 additional High Schools authorized [30m total] DLR 2.1.3: By December 31, 2025, or such later date as may be 20 established by the Bank, at least a 15 percent reduction has been achieved in the UNE Test Score rural-urban performance gap as compared to the 2021 baseline for the UNE Test Score rural-urban performance gap DLR 2.1.4: By June 30, 2024, or such later date as may be 10 established by the Bank, MOES has adopted a new regulation for a new admissions system to transition from High School to university (which factors in school performance and university requirements). DLR 2.1.5: By October 31, 2025, or such later date as may be 20 established by the Bank, MOES has implemented a new admission system adopted in accordance with DLR 2.1.4 DLR 2.1.6: By June 30, 2026, or such later date as may be established 10 by the Bank, the Borrower, through a legally authorized body, has provided the authorization to operate at least 200 kindergartens in rural or mountainous areas on the basis of having met the quality standards set out in the Law on Early and Preschool Education or other official document, including for nutrition, education, and physical environment; all in a manner satisfactory to the Bank DLR 2.2: MOILHSA has adopted an order or an equivalent legally 10 binding document, satisfactory to the Bank, to define clinical pathways and clarify the scope of primary and hospital care for at least the following selected ambulatory-care sensitive conditions: Page 68 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment Disbursement Disbursement Linked Result Allocation to DLIs and Linked DLRs Indicator (US$ million) heart disease, chronic obstructive pulmonary disease, Type 2 diabetes, and mental health conditions DLR 2.3: Government of Georgia has adopted a decree, satisfactory 20 to the Bank, to revise: (i) the type and number of services covered by the government for primary care as part of the benefits package and (ii) the performance-based payments received by primary care providers all for Selected Non-communicable Diseases in order to incentivize the appropriate level of care DLR 2.4: SESA has adopted a methodology, through SESA Director’s 10 order, satisfactory to the Bank and consistent with ESCO, to identify skills in shortage (including skills required for green jobs) and for skills forecasting as a basis used for referrals to vocational training and other SESA ALMPs DLI 3: Inclusion DLR 3.1.1: Regional SESA Offices have been established with US$454,545 for every of the poor and Appropriate Staff and Proper Equipment in a manner satisfactory to regional SESA office vulnerable the Bank subject to a maximum people in social amount of US$5 million protection [Total 5m] services [Baseline: 0 offices in calendar year 2022; Target: 11 offices in calendar year 2026] [US$ 30m] DLR 3.1.2: Increase in the number of Rural Jobseekers receiving US$87,719 for every Employment Support Services through Regional SESA Offices 1,000 Rural Jobseekers served by SESA starting with calendar year 2022 [Total 5m] [Baseline: 0 at the end of calendar year 2021] DLR 3.2: Increase in the number of Vulnerable Unemployed Persons US$476,190 for every who participate in SESA ALMPs 5,000 Vulnerable Unemployed Persons Page 69 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment Disbursement Disbursement Linked Result Allocation to DLIs and Linked DLRs Indicator (US$ million) participating in SESA ALMPs [Total 10m] [Baseline: 3,500 Vulnerable Unemployed Persons in calendar year 2021] DLR 3.3.1: The revised Worknet Job-matching Portal is operational in 5 a manner satisfactory to the Bank DLR 3.3.2: Increase in the number of vacancies posted on the revised US$357,143 for every Worknet Job-matching Portal 10,000 vacancies posted on revised Worknet Job- matching Portal starting with calendar year 2025 [Total 5m] [Baseline: 0 vacancies as of end of calendar year 2021] DLI 4: DLR 4.1: By December 31, 2024, or such later date as may be 15 Improvements established by the Bank, at least 90 percent of Schools with more in adoption of than 170 students across Georgia have adopted School Digital Action new digital Plans satisfactory to the Bank and crossed a threshold score of 75% services and under the infrastructure module of the Self-Reflection Tool upgradation of DLR 4.2: At least 80 percent of primary care providers have access to 15 existing electronic health care records available with MOILHSA for at least 12 administrative consecutive months and management DLR 4.3: At least 100 primary health care and ambulatory providers 15 systems have provided telemedicine consultations that are covered by the towards UHC Program for at least 12 consecutive months meeting appropriate DLR 4.4: An Integrated Reporting System for Social Protection, 15 [EU] standards satisfactory to the Bank, is established by the IT Agency and is functioning, and can generate reports on the coverage of social and Page 70 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment Disbursement Disbursement Linked Result Allocation to DLIs and Linked DLRs Indicator (US$ million) for digital employment services administered by government agencies at the services central and local levels in order to detect gaps and overlaps, reduce exclusion and inclusion errors, and improve the integration among [US$ 60m] social protection services/programs, including other governmental programs supporting households vulnerable to climate change. Total: Education 155 Total: Health 150 Total: Social Protection 95 TOTAL AMOUNT 400 Note: Euro equivalent DLI/DLR amounts, adjusted for front-end fees are in the corresponding Loan Agreement DLI 1: Improvement of Government spending efficiency through adoption and implementation of appropriate administrative and financing mechanisms This DLI will measure the fundamental change to the overall financing design, mechanisms or financing framework for human capital services backed by the MOF. This will include upgrading the financing models and methods for delivering human capital services across the education, health and social protection spheres which will lead to greater cost-efficiency in service delivery and quality as described under each result (DLR). By addressing the fundamental mechanisms and models for financing human capital services and the high cost of these services (medicines, health payments, low student teacher ratio and social benefits not related towards the needy) can be redirected. The fiscal gains can be redirected towards greater quality, benefitting children, patients, the unemployed and the extreme poor. [Education] DLRs 1.1.1 and 1.1.2 will ensure that the new competency-based curriculum reform to revamp the formation of Georgia’s human capital will benefit from an efficient and equitable financial system that would replace the current financial model in place for the past fifteen years. Analysis of the implementation of the financing model from 2006-2021 shows problems regarding efficiency, Page 71 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment transparency, and equity that lead to poor quality of learning that is fundamental to Georgia’s Human Capital. The revised curriculum is expected to lead to greater needs for financial resources at schools, while increasing the need for scale economies. As the curriculum is implemented, starting from 2022, learning will be derived about the financing gaps and the financing formula would be revised in a phased manner. The revised financing formula would include the following considerations: (i) adhere to criteria of transparency and equity, including higher investment for disadvantaged groups (special needs students, ethnic minorities, students in rural and mountain areas); (ii) adequate financing for the successful delivery of the new competency-based curriculum, including staffing of teachers, technical and administrative and support staff; (iv) continued teacher professional development to support teacher individual development plan and blended pedagogy, formative assessment of students, professional development of school principals for instructional leadership, and modern and digital teaching and learning resources; (school level funds for these purposes will supplement centrally provided teacher related expenditures through TPDC); (iv) the financing of maintenance and of energy expenditures that mandate or incentivize emission reducing and energy efficient modalities (like wall and piping insulation, double paned windows); and (v) enhanced role for regional Educational Resource Centers (ERCs). The school financing model will support climate resilience by incentivizing energy efficiency as outlined in the Program Scope section in the PAD – this has mitigation and adaptation ramifications. Mitigation from the reduction in energy consumption and associated benefits, and adaptation from the didactic value of practical action in schools. The latter has added value as students will transmit the values to their homes and communities. [Health] DLRs 1.2.1 and 1.2.2 will ensure that MOILHSA implements the DRG payment system nationwide. To make more effective use of available public funds and to receive better value for money from hospital spending, the NHA needs to purchase hospital services strategically and manage costs effectively. The current complex payment system with different tariff setting and copayment rules enables hospitals to game the system, which leads to increased administrative costs and constrains the NHA’s ability to control costs. [Health] DLR 1.3 will ensure that the GoG adopts regulations to introduce managed-entry agreements and starts using managed entry agreements for single-source patented medicines covered by the UHCP. Outpatient medicines constitute 69 percent of OOP payments, and current reimbursement and purchasing mechanisms further contribute to inefficient pharmaceutical spending. There is considerable scope for improving the efficiency of spending on pharmaceuticals and expanding the coverage of outpatient medicine benefit to lower OOP. Page 72 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment [Health] DLR 1.4 will ensure that MOILHSA adopts regulations to introduce reference pricing for the purposes of setting or negotiating prices of medicines of publicly financed medicines under the UHCP. OOP health spending in Georgia is very large, and limited coverage of outpatient medicines by the public sector means that patients have little choice but to pay OOP for drugs at pharmacies. This increases the risk of impoverishment from OOP spending, creates barriers for accessing timely preventive care services, and is associated with an overreliance on costly hospital and emergency care services, since inpatient medicines are fully covered under UHCP. [Social Protection] DLR 1.5.1 and DLR 1.5.2 will ensure that MOILHSA adopts a ministerial decree to (i) pilot the revised targeting formula used to identify poor families with children and determine eligibility to poverty targeted human capital benefits in selected municipalities by March 2023; and (ii) roll it out nationally by 2024. Despite the high quality of the Proxy Means Testing (PMT) scoring formula used to rank and select beneficiaries of poverty-targeted benefits, its performance to identify poor families with children has deteriorated in the past years. Key reasons for this are that the formula is calibrated based on outdated surveys (HIES 2015); fails to automatically index monetary values, with a distortionary effect and particularly negative consequences on the working poor; and also fails to make use of observable components of household welfare, such as gas and electricity expenditure. The updated TSA selection rules will be reflected in the Social Code for Georgia, expected to be approved by 2023. [Social Protection] DLR 1.6 will ensure MOILHSA, through SSA and the IT Agency, completes the digitization of TSA key implementation processes to reduce benefit processing time from application to SSA’s submission of payment orders from five to less than two months and to adapt faster to sudden vulnerability changes. Major bottlenecks to timely delivery lie in the application, registration, eligibility verification, and enrolment procedures. Application can only be done in-person, causing potential exclusion errors; the vulnerability score assigned to households is not responsive to sudden changes in income and assets, and is assessed only at the time of application and at recertification four years later; most of the eligibility verification is conducted “manually� by social agents, resulting in delays in benefit payments to beneficiaries and high administrative costs. DLI 2: Adoption and implementation of appropriate quality standards for human capital services enabled across all service delivery institutions Page 73 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment This DLI will measure through the proposed DLRs, how the revised and updated financing framework, new standards, strategies, models, and regulations across the human capital services have been implemented in all service delivery institutions ensuring that the desired outcomes are achieved. Expenditures required to achieve this across the implementation agencies, will include hardware and software, social services, medical supplies and consumables, lab equipment, minor civil works and rehabilitation, technical assistance, research, studies and analytical works, training, designs, and consultations. [Education] DLRs 2.1.1 – 2.1.5: Georgia faces a learning quality deficit with OECD-PISA scores indicating a performance much below that of the OECD benchmarks or quality standards. Low learning performance is exacerbated by a quality divide between rural and urban schools as well as inequity in learning outcomes along socio-economic groups. A very peculiar school structure in Georgia is based on the premise that all schools in the country, regardless of location, offer all grades from Grades 1 to 12. In practice, this is not feasible to achieve, which is why most countries in the world have High schools for the Upper Secondary Education level. In Georgia, students faced with poor quality of secondary education just do not attend school, with many learning days lost, especially in Grade 12. Urban students who can afford it take coaching classes to pass their university entrance examination, and the shadow education system is a serious problem to be tackled. The education DLRs under the DLI 2 are targeted to deal with this serious human capital problem. The quality standards would address issues of school safety and climate resilience as well as disaster response and preparedness. This is an important measure to make sure that the built environment for education in Georgia helps to address the problems generated from vulnerability to climate change. [Education] DLR 2.1.1 and DLR 2.1.2 will ensure that National Authorization Council for General Education provides authorization to 300 High Schools (by 2023) and 600 High Schools (by 2026) throughout the country to deliver the new competency-based curriculum. With strong support from the Government of Estonia (ranked top PISA performer in Europe), Georgia is undertaking a thorough revamp of Upper Secondary Education, with a demanding new curriculum that stresses development of competencies required for the 21st Century. Countrywide improvement in teaching and learning is expected to take place through the implementation of a series of school processes, including the critical process of continuous professional development of teachers and school authorization. Authorization of High Schools for the new curriculum implies the introduction of hub schools to address the existing quality and equity shortfalls. These hub schools will be equipped with adequate staff and resources to host students from smaller schools in the district, which otherwise may not have qualified teachers for Page 74 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment key subject domains, specialized laboratories and extra-curricular course offerings introduced by the new curriculum. The new curriculum will include elements of climate resilience and climate mitigation. The support package to these hub schools will include: (i) school-led professional development support to teachers in key subject domains; (ii) increased time-on-task for special needs and disadvantaged students through additional instruction time and tailored teaching; (iii) modern, discrimination-free teaching and learning resources for blended education, laboratories, and physical education facilities required to deliver the competency-based curriculum; (iv) capacity building of ERCs; and (v) improved educational infrastructure to support learning. The quality standards include attention to the environment, including school safety, disaster response and preparedness. [Education] DLR 2.1.3 will ensure a 15 percent reduction of the rural-urban performance gap in the Unified National Examination (UNE). The reduction in the performance gap is an expected result from the strengthening of High schools and the implementation of the secondary school curricular reform. Performance will be measured by the gap in UNE performance in Science, Technology and Mathematics and Foreign Language disciplines, where the baseline performance gap between urban and rural schools is highest. The sought-after performance improvement is of the order of magnitude of 0.2 standard deviations. The improvement on inclusion is related to equity in student access to universities from urban and rural areas. For example, students in rural areas will be able to implement student projects as part of the competency-based learning. These projects, in addition to improving critical thinking and problem solving skills, will directly involve climate resilience in many cases as the curriculum encourages students to develop scientific literacy on climate change issues. [Education] DLR 2.1.4 and DLR 2.1.5 will ensure the adoption of a new regulation for a new admission system to transition from High School to university (which factors in school performance and university requirements) by June 2024 and nationwide implementation of a new admission system by October, 2025. Georgia has enough number of places in universities for the entire graduating cohort of 12th graders, indeed not all entering slots gets filled up, but there is still an intense competition among the 90 percent of the graduating 12th graders who take the UNE. Most students from families who can afford it go for coaching classes to get a better score in the UNE to enable them to enter a university program considered more prestigious or economically more attractive. Even though the current UNE based admission has high public support, it is suboptimal where human capital productivity is concerned. A new system of university entrance would encourage students to develop deeper knowledge and skills based on a national curriculum. Such a system would be less biased in favor of Page 75 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment students from urban areas, have a role for high school grades and for extra-curricular activities. This DLR also will further the climate resilience agenda as students would include climate change related actions in their university applications. For instance, a student may have helped set up a weather lab in her village – the skills and competencies required in such a project should help the student get admission to university. Universities themselves would be able to better customize the admission process for their own admissions. A pilot can be done either in three selected universities or for three selected programs of national importance, such as tourism, business administration, and economics. The admission system reform will complement and will need to go in tandem with the revision of the Higher Education Financing System. Flexibility for universities in tailoring the admission requirements to specific requirements has climate resilience implications. For instance, a course on Climate Change and Sustainable Development was instituted at the prestigious Ilia State University in 2016. Such courses, when offered as part of bachelor degree programs can seek evidence of student’s work on climate change adaptation or mitigation, can be provided advantages in admission. The university can then ensure a highly dedicated and committed set of students for their student body, which is not possible in the current admission system which only considers the UNE test score as the single measure of student readiness. The students can go on to as more climate informed citizens in general walks of life, and some would go for the emerging green jobs sector in the labor market. [Education] DLR 2.1.6 will ensure that by June 30, 2026, or such later date as may be established by the Bank, the Borrower, through a legally authorized body, has provided the authorization to operate at least 200 kindergartens in rural or mountainous areas on the basis of having met the quality standards set out in the Law on Early and Preschool Education or other official document, including for nutrition, education, and physical environment; all in a manner satisfactory to the Bank. Georgia's decentralized preschool system currently faces a number of challenges, including lack of data on children enrolled in or left out of the system, absence of a measurement system to assess the quality-of-service delivery, and lack of feedback mechanisms from service users – children and their parents – on the effectiveness and efficiency of preschool services. The Law on Early and Preschool Education, effective since September 2016, requires municipalities to provide oversight and monitoring of preschools in the implementation of national standards, including education, nutrition and food safety, sanitation and hygiene, and physical environment. Page 76 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment [Health] DLR 2.2 will ensure that MOILHSA adopts national regulation to define clinical pathways for at least three selected ambulatory-care sensitive conditions including heart disease, chronic obstructive pulmonary disease, Type 2 diabetes, and mental health conditions, clarifying the scope of primary and hospital care. Defining appropriate levels of care through clinical pathways will improve health care quality and efficiency, and help contain the cost of care, a prerequisite for extending the revision of the benefits package under DLR 2.3. The patient pathways to be revised will include conditions with a solid body of evidence on optimal care and with the heaviest disease burden in Georgia, including but not limited heart disease, chronic obstructive pulmonary disease, and Type 2 diabetes. [Health] DLR 2.3 will ensure that MOILHSA adopts national regulations to scale up the revised benefits package and performance-based payment system for selected non-communicable diseases at the primary care level. Georgia’s current payment model for primary health care is mainly based on inputs and does not prioritize quality or outcomes. Extensive international evidence shows that carefully designed and regularly updated performance-based payments can create incentives for health care providers to focus more on patient experience, health care quality, and population health outcomes. [Health] DLR 2.2 and DLR 2.3 are related to climate resilience. Improvements in Primary Health Care system will lead to a decrease in hospital admissions, especially for individuals with chronic diseases. It will significantly reduce the travel to hospitals that are mostly in city centers. Improvements in Primary Health Care system will lead to improved surveillance of infectious diseases and appropriate level of care in PHC centers will reduce burden on hospitals to deal with climate disaster related outbreaks and other emergencies [Social Protection] Under DLR 2.4 SESA will develop and adopt a methodology to identify skills in shortage (including skills required in green jobs) and for skills forecasting to be used as a basis for referrals to vocational training. There is a lack of capacity to forecast skills and occupations in demand, limited coordination across key stakeholders (Ministry of Economy, MOES, MOILHSA, SESA, Skills Agency, Sectoral Skills Organizations), and no established mechanisms to adapt the selection of vocational training courses to the skills needed by employers, leading to the poor job relevance of the vocational training system. Achievement of DLR 2.4 will strengthen SESA’s capacity to identify critical skills and occupations consistent with the European Skills, Competences, Qualifications, and Occupations (ESCO) system to improve SESA and MOES’s programming and budget planning towards skills required for jobs that are in demand, and that are “green.� Analysis of skills in demand through the vacancy monitoring system in place in SESA will feed into the Labor Market Information System in the Page 77 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment Ministry of Economy. This DLR builds on results under the EU budget support program 2018-2023 on skills development and matching labor market needs. The support related to green jobs is part of Georgia’s climate mitigation effort. The achievement of this DLR will require close cooperation between SESA and the Skills Agency. DLI 3: Inclusion of the poor and vulnerable people in social protection services [Social Protection]: DLI 3 aims to increase the inclusion of poor and vulnerable people in social protection and employment services to ultimately improve the employability of people from the more remote areas, people at risk of social assistance dependency (TSA beneficiaries), youth, women, IDPs and people with disabilities (PWD). Improved access to jobs among the most vulnerable will be achieved through a geographical expansion of SESA employment services to all regions and rural areas (DLR 3.1.1 and DLR 1.1.2), an expansion of the coverage of ALMPs provided by SESA specially for the more vulnerable groups (DLR 3.2), and the upgrade of the Worknet online job-matching portal (DLR 3.3.1 and DLR 3.3.2). Expenditures will relate to MOILHSA, SESA and SESA offices administrative expenditures, ALMPs development and recurrent costs, Worknet development, hardware and software, consultations, and training. [Social Protection] DLRs 3.1.1 and 3.1.2 will ensure that each region in Georgia has a sufficiently staffed and equipped SESA office to expand the provision of employment services and programs to underserved rural areas and population groups. The geographical coverage of ALMPs and SESA is very unequally distributed, for instance three regions lack a SESA regional office. The scope and scale of ALMPs is too small to help promote human capital utilization among vulnerable job seekers—TSA, IDPs, youth, women, PWD. In 2021, only about 4,300 people received any type of employment service and program in a country with a working age population of 2.3 million people out of which about 550,000 are able to work but jobless (non-student, without disabilities and illness). The achievement of this DLR will ensure that access to employment services is expanded to all regions in the country and to underserved areas by opening three additional regional offices. Additionally, it will activate mobile teams in areas where SESA offices are not present to increase the number of rural jobseekers served up to 20,000 per year. [Social Protection] DLR 3.2 will ensure increased participation in ALMP among vulnerable categories— women, youth, IDP, TSA beneficiaries, PWD. Under this DLR, the number of ALMP beneficiaries – short term vocational training, core competency training, internships, job intermediation, subsidized Page 78 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment employment, individualized consultations – will be increased annually to achieve a total of at least 30,000 ALMP beneficiaries from vulnerable groups by year 2027, which represents 60 percent of the total number of ALMP beneficiaries (50,000) targeted by MOILHSA in 2030. This is to address the limited participation of vulnerable groups in skills training courses, internships, job search assistance. [Social Protection] DLRs 3.3.1 and 3.3.2 will ensure that the Worknet Job-matching Portal is launched and rolled out to increase the number of vacancies posted and jobs matched. The existing Worknet job portal (worknet.gov.ge) collects few vacancies and basic information on the skills and occupations demanded to allow adequate job matching; complex registration procedures discourage employers to register. In 2020, about 10,000 vacancies were posted by about 700 employers and less than 10 percent (809) were successfully matched. This compares with about 60,000 vacancies posted in 2019 according to the nationally representative Skill Demand Survey conducted by the Ministry of Economy. The Worknet job portal will be upgraded to improve job matching outcomes. The data system will use ESCO (see DLR 2.4) as the reference for machine-classification of skills identified. ALMPs will promote access to green jobs among vulnerable job seekers. The program will identify and expand ALMPs, such as short-term skill training, subsidized employment and internships, to facilitate access to employment opportunities in green sectors; SESA will expand the participation of climate vulnerable unemployed in ALMPs, based on the targeting criteria developed through the IRSSP; Inform end user/labor market participants of green skills in high-demand and the low-carbon jobs vacancies; Increase the match of jobseekers with green skills to green industry through Worknet. DLI 4: Improvements in adoption of new digital services and upgradation of existing administrative and management systems towards meeting appropriate [EU] standards for digital services This DLI will measure the implementation of the use of technology and digital provision in improving administrative and management systems across the human capital sectors in line with EU/European standards and best practices. This will include implementation in schools, health care facilities and providers, hospitals, ambulatory care providers, SESA offices and human capital implementation agencies, as means to improve services and the quality of these. Expenditures to achieve this will relate to MOES and MOILHSA administrative costs, hardware, including computers, internet access and connectivity, software and software development, designs and new IT architecture, development of digital plans, training, and consultations. Page 79 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment [Education] DLR 4.1 will ensure that By December 31, 2024, or such later date as may be established by the Bank, at least 90 percent of Schools with more than 170 students across Georgia have adopted School Digital Action Plans satisfactory to the Bank and crossed a threshold score of 75% under the infrastructure module of the Self-Reflection Tool. International literature on successes and failures of digital technologies for learning points to the importance of self-organization on the part of empowered school communities to adapt technologies suited to their specific conditions. School Digital Action Plans will ensure that schools are able to access the digital resources they need (equipment, connectivity, training, subscription services), tailored to their own needs. Stakeholder engagement will enhance the chances of the IT infrastructure being used productively. A large and growing knowledge base on this issue, developed by the European Union, will be tapped for the purpose of implementing this DLR. Digitization of education supports the Government’s climate agenda. Digitization leads to lower pollution and GHG emission on account of reduced travel (mitigation). Investment in School Digital Education Action Plans will be supported by training. The training will include learning continuity through remote learning in the event of disruptions to in-person learning in the event of climate disaster. Disaster and emergency risk management and preventions training will be also be provided to school management and principals. These actions would ensure education service delivery to vulnerable students in the event of climate disaster (adaptation). [Health] DLR 4.2 will ensure that at least 80 percent of Linked Primary Care Providers have access to electronic health care records. By complementing reforms to improve provider payment mechanisms and service delivery, digitalizing and better networking primary care and secondary care providers will help to improve access to health care and health outcomes. [Health] DLR 4.3 will ensure that at least 100 PHC and ambulatory care providers offer telemedicine consultations that will be covered under the UHCP. Telemedicine enables health care facilities to provide safer and cost-effective health care services and improves access to health care services especially in rural areas. In addition, telemedicine helps to mitigate the impact of the COVID-19 outbreak and allow healthcare workers to safely engage with patients by allowing real-time assessments of the patients’ needs and facilitating immediate triage. Page 80 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment Digitization of health is a critical part of the Government’s efforts to combat the impacts of climate change. The health care sector is estimated to contribute 3–8% of the total emissions in developed countries and consumes countless liters of fossil fuel when patients and medical professionals travel to and from appointments, pick up prescriptions, and obtain tests and results. Telemedicine is considered one of the most potent technologies for greenhouse gas reductions and has the potential of reducing air pollution, known to be adversely associated with disease and deaths, by reducing travel and transportation. The benefits are obvious for home care programs and outpatient consultations. The telemedicine programme at University of California, Davis, with its 13,000 outpatient consultations over a period of 5 years, has resulted in a savings of 4.7 million miles of travel and a reduction of 1,700 tons of CO2 emissions.34 In Canada, it has been estimated that more than 11 million home visits by nurses could be replaced by telemedicine, which would result in a reduction of about 120 million km of travel and 33.220 tons of associated greenhouse gas emissions annually.35 In addition, implementation of energy-efficient health management information system has climate benefits. It reduces paper use and travel, for example when patients’ medical history and prescriptions are available to the healthcare providers patients directly. This in turn lessens the negative impact of the paper industry and prevents unnecessary deforestation. Tropical deforestation is considered the single largest contribution of land- use change to global carbon emissions because it releases 1.5 billion tons of carbon each year into the atmosphere.36 Increased use of electronic primary health care records also prevent data loss in the event of climate-induced disasters. Hence, the true potential of e-Health as mitigation strategy is evident when looking at cumulative effects and the larger scale. [Social Protection] DLR 4.4 will ensure that an Integrated Reporting System for Social Protection established by the IT Agency is operational and generates reports on coverage of social and employment services administered at both central and local levels in order to detect program gaps and overlaps, reduce exclusion and inclusion errors, and improve the integration of services/programs. Currently, a large number of operational IT systems (Social Registry, Worknet, local IT systems of municipalities) 34 Yellowlees PM, Chorba K, Parish MB, Wynn-Jones H, Nafiz N. Telemedicine can make healthcare greener. Telemed J e-Health. 2010;16:230–3 35 Scott RE, Perverseff T, Lefebre N. Reducing environmental impact: an example of how e-health can reduce environmental impact and concomitantly improve health; Proceedings of The 1st Annual Conference on e-Health: The Virtual Dimensions of Health and Environment—Empower, Enhance, Enforce; 2009. Apr, pp. 101–10 36 Gullison RE, Frumhoff PC, Canadell JG, Field CB, Nepstad DC, Hayhoe K, et al. Environment. Tropical forests and climate policy. Science. 2007;316:985–6 Page 81 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment housed in different departments, other ministries and municipalities are not fully interoperable and do not automatically update each other. This creates the risk of overlaps of benefits and inclusion errors and limits the different agencies’ ability to identify unserved potential beneficiaries and reach out to them. An Integrated Reporting System would be a powerful tool to allow for sophisticated policy analysis including: (i) profiling of the specific needs and conditions of various groups of the population (“demand side analysis�); (ii) coordination of the provision across local and central programs, including detecting of intended or unintended overlaps (“supply side analysis�); and (iii) analysis of potential “gaps� in coverage of key bundles of benefits and services that could be tailored to the typical needs of these profiled groups, combining the demand- and supply-side analytics. This gap analysis could allow for simulations of fiscal resources needed to extend key benefits, services, ALMPs and emergency programs including to respond to climate shocks to underserved groups, with clear identification of the potential additional beneficiaries. Digitization in the Social Protection sector also helps to combat climate change. The IRSSP will operate as a data warehouse including data on climate related shocks which will strengthen the capacity of the social protection system to adapt to the impacts of climate change. The IRSSP will represent a tool to (i) improve the identification of households who are vulnerable to climate change risks to better target social assistance cash transfers; (ii) inform the scale up of social assistance benefits in response to climate change-induced disasters; (iii) improve institutional coordination to respond to climate change- induced crisis. Further context for Education DLRs Georgia adopted a per capita financing model in 2006. It was revised in 2011 and has functioned for over a decade with successive or incremental tweaks that accreted complexity without any systematic study or revision. In the meanwhile, there have been large scale demographic shifts and changes in the context for policy. Populations in rural areas have declined while schools in urban areas have become more crowded. The introduction of the formula led to an initial optimization of schools, but over time the flow of resources and the needs of communities have once again diverged. An example of this divergence concerns teachers. The Government recently offered incentives for older teachers to retire and over 6,000 availed the offer over the period 2019-2020, at the same time as 2,300 new teachers joined the ranks – the spatial distribution of teachers has changed drastically from what it was in 2011. The financial model needs urgent study and reformulation to deal with problems of efficacy and equity, Page 82 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment while ensuring minimal infrastructural standards. Readiness for digital learning and distance education or blended learning, inclusion of special needs children and extension of compulsory schooling or basic education from nine to ten grades are also related to efficiency of expenditures.37 Typically, school vouchers are based on a single per student amount, which follows the student from the national or local budget to the school This is not the case in today’s Georgia. In a conventional voucher system of financing, the revenue of schools is proportional to their enrollment. Uniformly funded vouchers provide transparency and in theory motivate the schools to attract students in order to maximize revenue. In Georgia today, students enrolled in one school may carry different vouchers, if they attend primary or upper secondary grades. Students enrolled in two schools in a town may carry different vouchers, if their schools have different sizes, or teach in different languages. Schools located in rural areas and in the mountains have their own, separate systems of vouchers. In addition, every school receives a lump sum, which also depends on the number of students, but unlike the standard voucher allocation becomes smaller for larger schools. Schools also receive additional allocation for some groups of students (e.g. special needs students) and for some groups of teachers (e.g. teachers with specific add-ons to their salaries). Moreover, a separate system of vouchers, with significantly lower values, is applied to private schools. Thus, the system of school financing in operation today in Georgia is no longer a voucher model. It is a specific, rather complex model of capitation financing. Clearly, the successive changes to the voucher system in Georgia, introduced through 25 government resolutions, were designed to adapt the system of financing to a very fragmented school system, comprising of schools with very different budget needs. The Georgian school system comprises both small mountain schools with less than 20 students and large urban schools with more than two thousand students. The budget needs of these schools are so different that no simple system of vouchers can accommodate them. In particular, the initial versions of the voucher system in Georgia created a large number of undesirable so-called deficit schools, which could not even pay their teacher salaries and needed additional budget support. The requests for the additional allocation, submitted every month by many schools, placed an enormous administrative burden on the Ministry. The government resolutions that followed created changes to the voucher system were thus rational policy 37 Review of Education Financing Model I – Evolution of the Voucher Formula 2006-2021; and Review of Education Financing Model II – Policy Options for Reform – Working Papers as part of South Caucasus Education ASA – P174980 Page 83 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment response to real issues, and over time they reduced the number of deficit schools. So the complexity of the financing system reflects the diversity of schools in Georgia. However, even this evolved complex financing system is not able to adequately support the needs of different schools across the country. Indeed, more than half of all Georgian schools are not financed through the adjusted voucher system. Instead, their budget needs are assessed independently as so called “small schools�, or schools with fewer than 170 students. Small schools comprise 60% of all schools in the country, and the Ministry employs a specific methodology to assess their costs. The management of such a system of financing, in which majority of schools receive individually calculated allocation, and the remaining schools are subject to exceedingly complex capitation formula, is not easy. It becomes very difficult to ensure in practice two main theoretical benefits of a standard voucher system, namely transparency and equity. Transparency is lacking for both the small schools and the large ones. Available research indicates that initially and also now, small schools do not understand fully how their allocations are calculated, as the methodology is an internal procedure of the Ministry and some degree of negotiations is allowed. Certainly, parents understand even less. For larger schools, transparency is obscured by the complexity of the formula, as described above, and by its frequent changes (many of the resolutions changing the voucher system include only clauses which replace older regulation, so to understand fully the present system, numerous resolutions need to be simultaneously reviewed and considered). Ensuring equity of such a complex and fragmented system of financing is also challenging. The comparison of financing of small and large schools is a difficult task, due to individual treatment of every small school and to possible significant year-to-year fluctuations. Thus, equity of allocation to small schools needs to be carefully monitored (it is not analyzed in the present report). For large schools, as preliminary analysis conducted below indicates, when assessed on the per class basis, rural schools receive higher allocation than urban schools, and mountain schools much higher allocation. This raises important concerns regarding horizontal equity. A separate issue is equity of financing of schools of the same location (urban, rural, mountain). Preliminary research indicates that on average equitable distribution within the type of school location is ensured by two specific instruments of the voucher system for schools with over 170, but fewer than 200 or 300 students, namely that their per student vouchers are larger, and that they receive larger lump sum. Further, at threshold values of the number of students, when the voucher values change (such as 205, 299, 530), an increase of enrollment by one student may lead to a decrease of overall allocation for the school of up to 5% (considering both the per student voucher and the lump sum). Again, this is an undesirable effect of the complex system of Page 84 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment vouchers, reducing equity of the allocation system. It is relevant to note that there is no mechanism protecting individual schools from this adverse effect of the voucher system. The reformed school financing model would be implemented under the assumption that Georgia will maintain its system of autonomous schools, with directors responsible only to the school board, and will continue to finance them using the voucher system. The main goal of the policy option is to make the financing simpler and more transparent. In particular, the simplification will lead to a reduction of types of vouchers and of the different lump sums now used in the system, and will replace them with some coefficients, which are easy to use and which have clear strategic meaning as expression of national education policies. Under this policy option, there will remain over 1200 independent budget users in education (units with responsibilities in the budget process). The streamlining of the current voucher system may include the following steps: Removal of separate vouchers and lump sum values for secondary schools. Instead, a coefficient greater than one may apply to these schools. The value of this coefficient should depend upon curriculum differences (more lessons per week in secondary schools than in primary schools), equipment requirements (more school aids and the corresponding higher maintenance costs in secondary schools). The initial value of this coefficient, based on the current practice, would be 1,2. Removal of separate vouchers and lump sum values for schools providing teaching in languages other than Georgian. Instead, an appropriate coefficient may be used. The value of the coefficient should depend on the additional educational burden of these schools (extra lessons of Georgian language). Removal of separate vouchers and lump sum values for mountain schools. Instead, a separate coefficient may apply to these schools (in analogy with the present treatment of rural schools). The value of the coefficient may be based upon average differences in class size between urban and mountain schools. Removal of separate vouchers and lump sum values for schools based on their student enrollment. Instead, a single voucher and lump sum value may be defined for the category of largest schools (for example, about 1200 students). For smaller schools, separate coefficients greater than one may apply. The values of these coefficients may be based upon average class sizes in smaller schools, in comparison to class sizes in large schools. Page 85 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment Taken together, these simplifications will result with one value of the voucher and of the lump sum, and a system of coefficients. As inflation intervenes, especially as regulated teacher salaries are increased, it will be enough to adjust the value of the voucher and of the lump sum, rather than rewriting the whole formula. At the same time, is there are changes in education policy, for example if curriculum of secondary schools is increased or decreased compared to the curriculum of primary education, the relevant coefficient can be adjusted without the need to recalculate all elements of the voucher system. Separate streamlining process should be adopted towards “small� schools (below 169 students). These schools, forming 60% of all public schools in Georgia, have now their allocation determined on the basis of employment levels, through a simple methodology (see World Bank 2021). The streamlining process may include: Revision of the allocation methodology for “small� schools, in line with their actual needs. This is needed to ensure that no “small� schools receive insufficient allocation and become deficits schools, and at the same time that no “small� schools receive excessive allocation, which would be inefficient. The revised methodology should have the form of a specific budget procedure and should include the budget calendar (dates for all actions specified in the procedure). All the steps of this procedure should be public (information available to school directors, teachers, parents). The procedure should also identify which units of the Ministry are involved in which steps of the procedure. Discussion of the revised methodology with school directors and with budgeting experts, to remove doubtful or unclear steps. Publication of the revised methodology as an obligatory budget procedures to be followed by the school directors and by MES officials. Supporting the piloting and implementation of the National Curriculum revised to match European Union Competency Based curriculum is the context for DLRs 2.1 and 2.2: Georgia recently revised the Primary and Basic Education curriculum which is now aligned with the EU’s Key Competency-based curriculum, and reform of the curriculum for Secondary Education (currently Grades 10 to 12) is the next step. Having a strong, modern curriculum is essential to meet the ambitious, transformative Human Capital goals that the PforR will support. A recent analysis of the Secondary Education points to three problems- (i) absence of clear targets and functions of the upper secondary level, most evident until 2018 in the presence of the so called abituri hours in preparation for university entrance examinations; (ii) problems for schools in devising their own goal oriented programs and deficiencies in student Page 86 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment assessment and (iii) absence at the Secondary level of teacher professional development programs that stimulate intrinsic motivation of teachers, and develop the school staff as a unified team. Digitally Enhanced Ecosystem of Support for Teaching and Learning supported through DLR 2.1: The EU inspired “key competencies� based curriculum places huge demands on the capability of teachers and parents. The responsibilities for teachers and school principals are even more difficult because of the expectation to develop adapted local aspects of the curriculum. Realizing the inherent challenges, the Ministry of Education, Science, Culture and Sports has introduced a pilot scheme where there is a package of support services provided to schools. Of the overall pilot with 300 schools, 120 have received a special digital package under the “Digital Turn� project. Under the DLR 4.1 for School Digital Action Plans regarding teachers with digital competence will be based on results from the application of a validated SELFIE tool (SELFIE stands for Self-reflection on Effective Learning by Fostering the use of Innovative Educational technologies) developed by the EU and adapted for Georgia (Figure D1) – Annex 4 provided a sample of outcomes derived from the pilot. Figure D1: School Digital Education Action Plans to be administered using SELFIE tool Page 87 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment Further context for Health DLRs The ultimate objectives of universal health coverage encompass utilization in line with need, quality in health care, and financing, while the intermediate objectives relate to efficiency, equitable distribution of resources, and transparency. Each health provider payment method sets incentives for providers and patients, and, in turn, has a positive or negative impact on these objectives. Under DLRs 1.2.1 and 1.2.2, the MOILHSA will implement a diagnosis-related group (DRG) payment system for hospital care. In DRG payment system, health care providers are paid based on case groups with a fixed payment per case discharged regardless of inputs used. Therefore, DRG payment will help the NHA improve efficiency by reducing overtreatment and unnecessary services while maintaining quality of care since the health providers have an incentive to reduce the amount of inputs per case. In addition, the DRG payment system can result in a restructuring of the hospital services as providers with insufficient activity may Page 88 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment have to close, merge with other providers, or adjust their service delivery structures.38 Furthermore, the DRG payment system collects standardized information on hospital activity which can improve transparency on the type of volume of health services such as diagnosis, procedures, severity indicators, and patient characteristics delivered by each hospital. The information collected through a DRG system can serve to measure and compare performance across providers (e.g. in relation to cost of treatment or average length of stay). This information can be used strategically to encourage improvements in quality of care or to review patient care pathways systematically.39 Finally, DRG payment lowers the administrative burden by changing the unit payment from every service to a group of services. While DLR 1.2 will help improve efficiency through the introduction of DRG payments for hospital care, DLRs 1.3 and 1.4 will aim to improve the efficiency of pharmaceutical spending. To improve efficiency, and also inclusiveness, of the Georgian health system, medicines need to be more affordable as expensive medicines are significant barriers to accessing medical care. Inefficiencies in pharmaceutical spending adversely affects access to affordable high-quality medicines. Georgia’s 35.7 percent of total health spending is on pharmaceuticals, and its spending on pharmaceuticals s a percent of GDP, 2.54 percent, is twice higher than in the OECD average. Outpatient medicines constitute 69% of OOP health spending in Georgia where medicine prices are high compared to neighboring countries and the cost- plus margin for pharmacies significantly exceeds margins established in EU countries (Richardson & Berdzuli, 2017). High OOP health spending driven by pharmaceutical spending has an adverse impact on households with lower income or those who are otherwise disadvantaged. In 2017, the Georgian population spent 983 million Georgian Lari (approximately US$ 400 million) to purchase pharmaceuticals though OOP payments. High OOP payments for pharmaceuticals place a heavy financial burden on households’ disposable income and create significant financial access barriers, especially for the poor and for patients with chronic conditions. Despite improvements since 2014, differentials in the proportion of people who did not seek care or did not purchase prescribed medicines because too expensive remain high. In 2017, among those who did not undertake a consultation for an acute sickness, 12.3 percent of those in the bottom quintile mentioned cost as the main reason (HUES 2017). 38 O’Dougherty S, Cashin C, Samyshkin E, Ibraimova A, Katsaga A, Kutanov E et al. Case based hospital payment systems. In: Langenbrunner JC, Cashin C, O’Dougherty S, editors. Designing and implementing health care provider payment systems. How-to manuals. Washington (DC): The World Bank; 2009 39 Street A, O’Reilly J, Ward P, Mason A. DRG-based hospital payment and efficiency: theory, evidence, and challenges. In: Busse R, Geissler A, Quentin n W, Wiley M, editors. Diagnosis-related groups in Europe: moving towards transparency, efficiency and quality in hospitals. Maidenhead and New York (NY): Open University Press – McGraw-Hill Education on; 2011 Page 89 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment This was the case for only 2.7 percent of those in the top quintile. Similarly, 15.2 percent of individuals in the bottom quintile cited cost as the main reason they did not purchase prescribed medicine, as opposed to only 5.5 percent in the top quintile. Lack of regulations is one of the leading causes of high prices on the pharmaceutical market. The wholesale and retail mark-ups are as high as can be borne by the market. Low-cost generics are less available in pharmacies compared to the expensive brand name drugs which further contribute to the unnecessarily high expenditures on pharmaceuticals. DLR 1.3 will introduce managed entry agreements in Georgia to improve efficiency of pharmaceutical spending. MEAs are well-established purchasing tools used in many European countries for reimbursement of novel, patent protected, high-cost medicines as their manufacturers have few incentives to reduce prices of new products that face no competition in conventional public procurement. Over the last two decades, they have become an increasingly popular alternative to traditional procurement., and MEAs can be designed to provide simple confidential discounts or to distribute financial risks and risks related to the performance of a treatment. More complex financial agreements aim to decrease the financial exposure of the payer in making a positive reimbursement decision to cover a new medicine. MEAs have potential advantages as a longer term, more sustainable framework that can contribute to affordable patient access to new medicines. Under DLR 1.4, reference pricing will be introduced. In most countries, the reference pricing used for setting the price of specific categories of medicines such as publicly reimbursed medicines and prescription-only medicines.40 Countries vary according to the frequency with which they review and compare reference prices. Hungary reviews prices every three or six months. Countries also vary as to whether or not they permit price rises to reflect national inflation. Enforcement of reference pricing policies and mark ups through the supply chain requires access to information on the prices paid for medicines. In this respect Georgia is relatively well endowed. The NHA receives information on the prices paid by hospitals to procure medicines and a customs database holds information on import prices for all pharmaceuticals. The NHA and customs databases could provide good information on ex- manufacturer and import prices for reference and regulatory purposes. Defining clinical pathways can improve health care quality and efficiency. Georgia’s medical community has published a wide range of clinical guidelines that specify appropriate diagnostics and treatment, including over thirty guidelines specific to primary care. These, however, do not always translate into 40 Cécile Rémuzat, Duccio Urbinati, Olfa Mzoughi, Emna El Hammi, Wael Belgaied & Mondher Toumi (2015) Overview of external reference pricing systems in Europe, Journal of Market Access & Health Policy, 3:1 Page 90 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment clear pathways of care across primary and secondary care services: the fragmentation of the health care system and the dominance of the hospital sector means that navigating the system is difficult for patients, and costly. International experience shows that defining clinical pathways (DLR 2.2) can address this, by distinguishing what care should be provided at what level, and by specifying the basis for referral to other specialists. Defining appropriate levels of care through clinical pathways will also help determine the cost of care, a pre-requisite for extending the revision of the benefits package (DLR 2.3). The patient pathways to be revised will include conditions with a solid body of evidence on optimal care, and with the heaviest disease burden in Georgia, including but not limited heart disease, chronic obstructive pulmonary disease, Type 2 diabetes, and mental health conditions. Further revision of the benefits package and scaling-up of performance-based payments can also support quality and efficiency gains. Georgia’s current payment model for primary health care is mainly based on inputs, and for secondary care is mainly based on activity. Neither of these models prioritizes quality or outcomes. Extensive international evidence shows that carefully designed and regularly updated performance-based payments can create incentives for health care providers to focus more on patient experience, health care quality and population health outcomes. This has been shown to work in Georgia: the health system has successfully piloted performance-based payments for selected NCDs (such as hypertension) and early childhood development in around 75 urban and around 125 rural health care providers. There is good justification, therefore, to support the government to scale-up this program to a larger number of providers, as envisaged by DLR 2.3. Further context for Social Protection DLRs The SP reforms comprised within the boundary of the Program are top priority for the GoG, as they stem from MOILHSA’s long standing commitment to reduce poverty and end child poverty. The World Bank technical assistance in the past four years. has revealed key design issues with the existing formulas, proposed an updated and simulated it. The WB’s work found that a revision of the PMT scoring formula would result in a dramatic decrease in exclusion error from TSA benefits (the largest of the many programs that uses eligibility for that purpose). A budget neutral revision of the PMT score according to the proposed formula would decrease the number of excluded households in the bottom welfare quintile by over 12 percent. Page 91 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment The reforms under the Program will address key shortcomings that hamper the ability of the current formulas (PMT scoring and needs index) to identify those in most need, in particular children. Despite the high quality of the PMT scoring formula (exemplified by its ability to correctly rank over 62 percent of households from the bottom quintile of the welfare distribution), this figure has deteriorated over time. Key reasons for this are that (1) the formula is calibrated based on outdated surveys (HIES 2015), (2) PMT scoring formula fails to automatically index monetary values, with a distortionary effect and particularly negative consequences on the working poor, (3) the formula fails to make use of observable components of household welfare (measured as aggregate consumption), such as gas and electricity expenditure which are only imputed based on observables rather than directly measured based on the available data. The reform of the social protection delivery system to improve efficiency of its targeting and its adaptiveness to aggregate or household income shocks is highly transformative due to its spillover effects on the whole human capital delivery; it has strong demand from MOILHSA; it is achievable within the Program’s life as it builds on substantive existing analytical work cumulated as part of the WB technical assistance to the country; it is a cornerstone of the GoG’s strategic commitment to ending child poverty through the poverty targeted childhood allowance. Under DLR 1.5.1 and DLR 1.5.2, the Program involves a substantial revision of the scoring formula used to identify poor households with children in need of social assistance and human capital benefits. The revised PMT and needs index have being validated with social registry administrative data is ongoing with World Bank technical assistance. The revised formula will be piloted in 4 regions representing the 4 different model specifications in Tbilisi area, large urban, other urban and rural regions. The formula will be then finetuned based on the pilot results and then gradually rolled out nationally. The selection of the four pilot regions needs to be confirmed by MOILHSA. Achievement of DLR 1.5.1 and DLR 1.5.2 will trigger improvements across the spectrum of human capital services, and – by decreasing exclusion errors – will facilitate aggregate improvements in the country’s human capital. The revised PMT formula follows three guiding principles: (1) maximizing precision by directly measuring verifiable components of household welfare through interoperability – data exchange (such as utility expenditure) rather than estimating them; (2) automatically index monetary variables to the prices of the year used for their estimates; (3) remove items that are easy to conceal and manipulate and difficult to verify, such as small livestock. The new needs index will be simplified and made more predictable and computable for households to understand, with little variation across ages. The share of household in the bottom quintile receiving TSA assistance is expected to rise by 11.7% based on economic analysis (Honorati, Sormani, Carraro, 2020). Page 92 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment Under DLR 1.6, key processes of the social assistance delivery chain will be digitized and automated, reducing manual activities, so that the evolving needs of households are rapidly recognized by the system and that monetary support rapidly goes into the hands of those who need them. The Program will support activities to digitize the application, on-line application (G2C), registration, eligibility verification and enrolment processes. First, household application will be made also electronically through an online portal, in addition to in person applications at the SSA offices. Second, most to the variables needed to generate the PMT score will be retrieved and cross-checked automatically by establishing regular data exchanges with other IT systems, eliminating the need for social agents to verify manually, minimizing the number of documents needed by applicants, reducing the number of home visits by social agents. The overall time between application and enrollment will be reduced from five months to less than two months. The re-calculation of the TSA score, which is currently obtained with a household’s request to re-register in the TSA, will be initiated automatically and regularly every month. The regularly updated PMT scores will increase the flexibility to quickly respond to changes in income and other vulnerabilities among families in the social registry that have been scored. Grievances will be taken online and addressed, and the exact computation of the PMT score will be made available for registrants to consult online. The geographical expansion of in-person SESA employment services, the scale-up of ALMPs and the upgrade of the Worknet job portal are of paramount importance to the GoG as evidenced by, the early planning for SESA’s geographical expansion as early as 2022, and the undertaking of commitments with donors (the EU) to support the roll-out of skills trainings. The plan to upgrade Worknet is already developed, including with support from GIZ, and the preparation of bidding documents for a consultant is almost finalized to contract a software development company in 2022. An essential means to counter skill depletion, increase people’s ability to find quality employment and reduce dependence on the safety net, ALMPs are currently underfunded at the central level and market solutions are scarce in remote and mountainous areas. ALMPs directly improve the human capital of their beneficiaries and that of their dependents, children in particular, and help release resources that the social safety net could redirect to those what are less able to work and to provide for themselves. Under DLR 3.1.1 and DLR 3.1.2, SESA employment services and programs will be expanded to all regions in the country. The Program will overcome the current under-provision of ALMPs in rural and mountainous areas through the creation of new regional SESA offices in three unserved regions... New regional SESA offices will be opened in the three regions of Racha-Lechkhumi and Kvemo Svaneti, Samtskhe-Javakheti, and Mtskheta-Mtianeti. Expansion of SESA has been in the plans since 2020. Offices will need adequate staffing and a model to increase geographical penetration to the rural and remote Page 93 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment areas. Building on the experience of the Ministry of Justice of Georgia, MOILHSA will adopt mobile units to outreach and as the first interface between potential beneficiaries and SESA staff as a cost-effective implementation arrangement rather than opening local branch offices in different municipalities. Based on this implementation approach, regional offices will be equipped with appropriate vehicles to serve job-seekers residing in municipalities where SESA offices are not present, especially in rural areas. Under DLR 3.2, the Program will expand access to ALMPs among the more vulnerable and underserved segments of the population by scaling up the supply of ALMPs. Through more extensive geographical coverage, more staff and more intense outreach to employers, SESA will expand the scale of the existing ALMPs (short term vocational training, core competences training, internships, subsidized employment and counseling) from 7,279 beneficiaries to about 45,000 in 2028 and almost 50,000 in 2030. The participation of vulnerable jobseekers in ALMPs is expected to increase to 30,000 per year as of 2028. DLR 2.4 will enable SESA to identify occupations in shortage and conduct medium term skills forecasting as a basis to improve the labor market relevance of the vocation training and retraining courses and the selection of internships and subsidized employment they manage. To better to align ALMPs with the needs of the market, the PforR would support the development of skills shortage tool based on employer data collected through Worknet, the reporting LMIS in Ministry of Economy and SESA outreach services. A methodology will be developed by SESA and the new Skills Agency to identify skills in shortage, including skills demanded in green jobs, and for 5-years skills forecasting based on best practices and Georgia local context. Twinning and budget support from the EU will complement these results by supporting an assessment of future needs in two priority economic sectors. Under DLR 3.3.1 and DLR 3.3.2 SESA’s job matching outcomes will be improved through the upgrade of the Worknet software. MOILHSA has already initiated the development of a new upgraded version of Worknet to significantly shift its job-matching function. It will do so by focusing on the following key changes: (i) develop a user- friendly interfaces for employers (G2B portal, G2B integration services) and jobseeker (G2C portal) to manage their profile, post vacancies/job offers (including those for circular labor migration) and CVs; (ii) massively increase the number of vacancies posted by strengthening the employer outreach strategy; (iii) develop highly performant matching algorithms based on a more disaggregated skills taxonomy and IT modern technologies like artificial intelligence – machine learning; (iv) increase the reliability of the platform through stronger quality controls (a combination of machine algorithm and manual processes); (v) improve infrastructure (server, storage, etc.) capacity; (vi) improve support provided in local employment service offices; (vii) strengthen the interoperability with other Page 94 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment government IT systems including Social Registry, PWD Registry, IDP Registry, EMIS in Ministry of Education and the Statistic-reporting LMIS in the Ministry of Economy; (viii) signing agreements with private providers to have links to postings and data exchange from/to private job-intermediation portals. With a more detailed taxonomy for occupations and skills and a revamped job matching platform (including G2C and G2B web interfaces), more persons from vulnerable groups, students, other jobseekers and employers will register, and job-matches will occur faster and more accurately. Thanks to technical improvements, as well as simplified registration procedures, employers will be more likely to find jobseekers with the right skills and have more productive workers. An efficient job matching function will also contribute to facilitate the reinsertion of vulnerable groups and other workers in jobs in the COVID-19 recovery phase. The revised Worknet will introduce feedback collection tools for its users, to be automatically triggered after online use of the portal and with easily accessible links posted on the portal’s homepage. Data analysis of the feedback collected will be performed by SESA’s monitoring and evaluation unit and will inform further development of the website and of the ALMPs offered by SESA. The feedback collection will also constitute an access point for the submission of grievances by Worknet users, ALMP beneficiaries, and other affected people. Monitoring and evaluation of the human capital sector are limited by the lack of consolidated result indicators on coverage and cost effectiveness of both central and local SP, education and health services, and on potential cases of exclusion and inclusion errors. Currently, a large number of operational IT systems (Social Registry, Worknet, local IT systems of municipalities) housed in different departments and Ministries and municipalities are not fully interoperable and do not automatically update each other. These datasets are usually limited to the self-selected list of beneficiaries or, in the best case, applicants to different human capital services. They do not include information from local government: except for the largest municipalities, there is no uniformly agreed data sharing agreement between local and central government on the provision of social care and other educational or healthcare services and bonuses. This creates the risk of overlaps of benefits and inclusion errors, while it limits the different agencies’ ability to identify unserved potential beneficiaries and reach out to them. It also limits the government’s ability to draw general statistics and maximize the impact of the available data. An Integrated Reporting System for Social Protection (IRSSP) that builds on the TSA social registry and expands capabilities for tracking benefits and beneficiaries across other programs (benefits, services, ALPMs, emergency programs for eco-migration, COVID, etc.) and levels of government will represent a powerful policy tool for MOILHSA. MOILHSA’s existing operational information systems do have some capabilities for tracking beneficiaries across the programs that it manages, which are significant since Page 95 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment the ministry covers social assistance benefits as well as health benefits, such as UHC subsidies. However, at present, there are no links to programs managed by other agencies or municipal governments. Given Georgia’s existing IT systems capabilities, developing these additional reporting capabilities is well within reach. This type of integrated reporting system would indeed be a powerful tool to allow for sophisticated policy analysis including: (a) profiling of the specific needs and conditions of various groups of the population based on the input data (“demand side analysis�); (b) coordination of the provision across programs, including detecting of intended or unintended overlaps (“supply side analysis�); and (c) analysis of potential “gaps� in coverage of key bundles of benefits and services that could be tailored to the typical needs of these profiled groups (combining the demand- and supply-side analytics). This gap analysis could allow for simulations of fiscal resources needed to extend key benefits and services to underserved groups, with clear identification of the potential additional beneficiaries that could be added. The Integrated Reporting System would include a Data Warehouse that collects information from different social programs such as the number and characteristics of beneficiaries, value, expenditure on social programs, performance of programs, such as the frequency of payments/transfers, speed or cycle- time of key processes, number of complaints received and resolved (Figure D2). The IRRSP would include also reporting business intelligence, statistics and other tools. The IRSSP would allow cross- checks using interoperability mechanisms between separate stand-alone benefits administration systems and other administrative information systems such as those managed by municipalities, the social registry, Revenue Service, Civil registry, Disability and IDP registries, and the new Worknet, the new Employer-Employees database developed by MOF, the Electricity company database and GEOSTAT poverty maps. The IRSSP would allow the monitoring and reporting on that information, and disaggregation by geographic location, vulnerable groups, etc. Such analytics on the various programs would not only be useful to the government, but also to citizens for better transparency – open data on the performance and management of social assistance programs. The analytics produced by the IRSSPP will be used for both administration/programming and policy decisions. For instance, the availability of statistics on overlaps and unintended benefits would inform programming decisions; pattern of total (central and local) budget by municipality against poverty, economic activities would inform budget policy decisions at both central and local level. Better information about ALMPs created and executed at the local level (e.g. in the city of Tbilisi or in the Republic of Adjara) will allow SESA to avoid duplication of effort and to learn on successes and limitations of locally delivered programs. Knowledge of the extent of social care subsidies across the country will allow the SSA to make strategic decisions better support those areas that are underserved. Page 96 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment This information will feed back into the GoG’s decision-making process when defining strategic priorities with granular up-to-date knowledge of the context. Under DLR 4.4 the Program supports the development of the Integrated Reporting System for Social Protection (IRSSP) in order to improve the human capital system’s ability to coordinate, integrate information, identify unserved households in need, and reduce errors. The Program will support the development of the IRSSP to be managed by the new IT Agency under MOILHSA, with the purpose of gathering details about all benefits, services, AMLPs and other emergency programs received by a person and/or a family. The development of the IRSSP to start with and for all human capital services gradually, has a strategic impact on the GoG’s ability to monitor and evaluate SP programs in an integrated way and maximize opportunities for coordination between central government and municipalities. The roadmap for the development of the IRSSP by year is presented in Table D3. The goal is relatively ambitious and will require substantive effort on the to ensure coordinated effort from the different public entities involved; however, it is achievable within the six years covered by the program and can be achieved through the recruitment of sufficiently skilled consultants. In subsidiary MOILHSA will have to build a strong team of statisticians for complex reports, indicators and other analytics. This team will teamwork with the IT specialists in charge with integrated reporting system. If GoG decides for a single data warehouse, from the proposed IRSSP will be extracted also the reports proposed by Feasibility Study Report on establishing a Labor Market Information Data Warehouse (financed through EuropeAid/39718/DH/SER/GE) - the list of the reports for labor market is included after the roadmap (Table D3). Thanks to IRSPP, agencies involved with the provision of human capital services will overcome informational barriers and improve their capacity of outreach, administration, and learning. By being connected to the Civil Registry and the IDP Registry, the IRSSP will provide a clear understanding of access to human capital services across the whole population living in Georgia and will not be limited to the self-selected part of the population which has taken the initial step of registering or applying for some of them. IRSSP will thus allow to reduce exclusion errors, optimize budget expenditure and avoid double payments (for instance, overlapping of national and local benefits) or undue payments. It will create administrative efficiencies in processes of verification of eligibility to benefits. It will pose the building block of case management systems, which however falls outside the scope of the Program and will require additional training of social agents and other actors of the delivery systems. IRSSP will improve policies especially cross-institutional addressing vulnerable groups in a more integrated approach; it will improve the capacity to monitor performance, provide comprehensive open data and to generate knowledge public goods. Page 97 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment Table D3. Roadmap of for the Integrated Reporting System for Social Protection (IRSSP) ROADMAP Year 1 Year 2 Year 3 Year 4 Year 5 Annual 2022 2023 2024 2025 2026 Objectives Institutional arrangements for Integrated Reporting System for Social Protection IRSSP (Datawarehouse) and monitoring and oversight Institutional Institutional At least 50% of All the statistics Strategy for arrangements arrangements for the specialists of team is further for IRSSP statistic team statistics team nominated/hired. expansion of the (team, (including the are Complex statistics Integrated memorandums impact in the nominated/hired. for comparing HC Reporting of budget for 2024). in different System for Social understanding localities/regions Protection with for data and for improving other data exchanges, Note: Based on national policies sources and etc.) the actual(2021) but also local other set of lists, data WB policies are statistics and recommends a defined. indicators team of max 5 specialists (including . Complex statisticians and statistics (as team leader). define previous year) are regularly extracted. Integrated Reporting System for Social Protection – yearly objectives Page 98 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment ROADMAP Year 1 Year 2 Year 3 Year 4 Year 5 Annual 2022 2023 2024 2025 2026 Objectives Data Strategy Data sources: Additional data Additional data sources for (including source: source: Social Register individual technical (for national Worknet Education data specifications, benefits) Management gathering detailed Other IT systems information planning, IT systems of managing System (EMIS) governance municipalities emergency SP especially regular and (for local programs attendance (not institutional benefits) simple enrolment) arrangements) for implementing I is defined and approved. Pilot with data gathering from national SR and at least 2 municipalities. Depending on the technical capabilities the following data sources will be integrated (on-line access) or data will be regularly (monthly, bimonthly) transferred to DW Civil register Internally displaced person ('IDP') registry Registry of disabled persons Revenue Registry Page 99 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment ROADMAP Year 1 Year 2 Year 3 Year 4 Year 5 Annual 2022 2023 2024 2025 2026 Objectives Note: All these data sources have to be integrated by year 3 (2024) From IRSSP will be extracted the list of benefits national and local, ALMPs and status of education per family member. The details will be expanded gradually accordingly with the integration of data sources Minimal Minimal set of Additional set of Additional set of set of lists lists: lists for targeted lists for improving to be ALMPs and other Human Capacity List of persons extracted social protection who receive List of children for services: national TSA but who receive reducing don't receive List of persons national TSA errors local benefits; and families and/or local (exclusion without revenue benefits which are and List of persons and who don't not in education inclusion) who receive local receive national (regular and benefits but don’t TSA attendance) improving receive national efficiency TSA; Youths in families List of children in receiving TSA families receiving List of persons without any TSA in remote and families who revenue in the rural (poor) areas receive benefits past 6 months – in the last year of from more local for targeted mandatory municipalities ALMPs; school; List of List of possible beneficiaries exclusion errors (youth, adults, TSA and elder) of ALMP Educations not hired in the (starting from following Civil Register) Page 100 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment ROADMAP Year 1 Year 2 Year 3 Year 4 Year 5 Annual 2022 2023 2024 2025 2026 Objectives 6/12/18/24 months after ALMP. Based on above lists will be calculated indicators per family compositions (single person, adults without children, adults with 1/2/3/more children), regions, municipalities, group of ages (ex. children, youth, adults, elder), disabled and not disable, women and men, vulnerable groups, etc. On each year will be calculated indicators according with available data sources. Other sets of reports which can be extracted from IRSSP are for labor market information: (i) Formal employment by sectors, gender, age groups, vulnerable groups and regions, (ii) Ratio of jobseekers per vacancy by occupational groups, skills, qualifications, gender, age groups, vulnerable groups and regions, (iii) Formal Employment rate of graduates by Education (ISCED), sector, skills, gender, age groups, vulnerable groups and educational institution (tracer study), (iv) Development in employment rate for participants in selected ALMPs by skills, gender, age groups, vulnerable groups and educational institution (tracer study). This list of report was extracted from Feasibility Study Report on establishing a Labor Market Information Data Warehouse (financed through EuropeAid/39718/DH/SER/GE) and it was extended with vulnerable groups and skills because these data will be available in IRSSP (which has more data sources than in the data warehouse proposed by feasibility study). Figure D2. The Integrated Reporting System for Social protection Page 101 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment Data sources operational (dat-to-day IT management information systems) Worknet Other IT systems involved in SP and HC Other operational labor Municipalities Governmental Social Register EMIS (Education Management Information market information IT systems databases/ managing national SA System); Persons with Disabilities - PWD for local benefits/ registers benefits system Registry; Internally Displaced Persons – IDP (ALMPs, vacancies, services/ALMPs Registry; Healthcare systems; jobseekers) Civil Register; Revenue Register; Operational (day to day) data Border Control; exchange for cross-checks Poverty Maps; bulk data bulk data bulk data (ex during enrolment) bulk data Electricity and gas and operational reporting consumption (from utilities); Business Register INTEGRATED REPORTING SYSTEM for SOCIAL PROTECTION and LABOR MARKET INFORMATION (GeoStat); national system managed by MoILHSA IT Agency and shared used by SESA, SSA, MoESD, MoESCS bulk APF register data (Accumulated Pension Fund) detailed data per person id (family id) Note: I ntegrated Reporting gathered from different data sources Syst em for Social Prot ection will need a strong Computed/estimated aggregated data team work raw sets of detailed data per locality, region, benefit, among IT data Surveys specialists and (also per person id/family id) services, ALMP program statistic team (based on lists extracted from IRSSP or random) Different reports showing all the Different indicators, statistics and open data HC details gathered from more aggregated/cumulated as example per: operational systems – data - geography: localities, regions, sources (benefits, services, - ages and gender; vulnerables groups education status, employmeent, - SA benefits/services, ALMPs ... status etc.) Different aggregated indicators, statistics and open for labor Different sets of data (detailed per personal id/family id) as market information (MoE – Statistics LMIS) as example: example: - Formal employment, vacancies/job offers, jobseekers (number - lists for targeted HC activities/services (SA, ALMP, educations and ratio) by economic sectors, occupations, skills, gender, age - possible exclusion and inclusion errors groups, vulnerable groups, regions, levels of education. - education level, skill, occupations, etc. - Development in employment rate for participants in ALMPs Source: By the task team (drawn from various interviews and administrative records) and Feasibility Study Report on establishing a Labor Market Information Data Warehouse - EU financed study). Results monitoring for this PforR operation relies on three core elements – investments, inter- operability and continuity. The Government of Georgia has recent and ongoing investments in digitization in relevant government functions that will support better results monitoring for the entire Government Program, much beyond the PforR Program – the recent advances and the plans to be implemented are outlined in the forthcoming “Long-Term National Strategy for the Development of Digital Economy and Information Society and Its Implementation Plan.� This plan includes creating a new Information Technology Agency within MOILHSA and EMIS Action Plan. Program investments in connectivity including schools, primary health care centers, educational resource centers, learning management systems and electronic medical records, are examples of interventions that will support the generation of data that goes into the monitoring and evaluation of results. Inter-operability of systems, through the adhesion to open-source standards ensures “future-readiness� of the digitized Page 102 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment systems and dashboards being created. Indeed, as data becomes ubiquitous and streams of “Big Data� are generated, the Program includes the foresight of using Artificial Intelligence (AI) to process the data, a measure that is bound to become increasingly common. Continuity in the monitoring of results is provided by the Program including continued participation of Georgia in International Large-scale Student Assessment (ILSA) studies and the availability of Household Survey data from Geostat, the National Statistical Agency of Georgia. Monitoring and Evaluation (M&E) capacity at MOES and MOILHSA is well-developed, and the WB has previous experience relying on the M&E arrangements under the I2Q and Emergency COVID Response projects. Intermediate and final outcomes related to social assistance efficiency can be monitored relying on the TSA database and can be complemented by using the Household Survey (IHS) implemented by GEOSTAT, which captures receipt of social benefits including TSA in a nationally representative survey run every year. Strengthening the cooperation between SSA and GEOSTAT, in particular to ensure that HIES survey measures the variables entering the eligibility TSA score, would be a valuable improvement to the current monitoring arrangement, as pointed out by SSA. Intermediate and final outcomes related to the improved inclusion and labor force participation among vulnerable groups will be monitored through SESA monitoring system and the development of an Integrated Reporting System for Social Protection, supported by the Project. Connectivity between central and local social registry is challenging (local registries being mainly paper based) and almost non-existent. Connecting the local and central information systems is an outcome of the Program and would constitute a key improvement to the monitoring of social assistance delivery, and the analysis of its efficiency. The institutional arrangement of a light footprint PCU housed at the Ministry of Finance with troubleshooting of problems will help set-up a permanent results culture in Government. There has been a previous intent for results-based budgeting in the Ministry of Finance, since Program Based budgeting was introduced in 2012. While the Program Based budgeting does have numerous advantages (for instance, it supports collaboration between hierarchically diverse government agencies working on a particular program), in the case of Georgia, it appears that the results part was not entirely consistent with supporting high performance. For example, the Program 32 02 11 (“My First Computer�) which had a budget of GEL31.35 million (approximately USD 10 million) in the school year 2020-2021 (further increased to GEL43 million the next year) simply states that all first graders will be provided with the computer equipment, but there is no attendant action to make sure that those computers are Page 103 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment used adequately to enhance learning41. In the case of the PforR, with a comprehensive view of Human Capital development, issues like this come in sharp focus. With results tied together in a package of reforms for a long-term learning objective (in the case of Program 32 02 11), it is expected that there will be significant institutional learning for a results-oriented functioning of government agencies. E. Program Economic Evaluation Georgia’s renewed focus on human capital needs to adjust to limited fiscal resources, implying that policies related to human capital need to be operationally efficient, of better quality, and more inclusive. As part of its strategy for improving human capital, a loan package of US$ 400 million is expected to support the implementation of policies in education, health, and social protection, tied to results in these three sectors. The ex-ante economic analysis of the prospective loan implies two important assumptions: (i) the Government of Georgia would enact and implement a package of policies aimed at improving school and student performance in education, at improving the equity and the quality of health services, and improving the targeting and efficiency of social protection programs; (ii) the net effect of the policies associated with the loan would result in net gains in human capital, with corresponding positive impacts on employment, wages and lifetime income of the target student population; on the reduction in health expenditures among users of health services resulting from net reductions in morbidity and mortality, and the increased in health and nutritional outcomes of the population in the lowest 40 percent of the income distribution. The overall approach of the ex-ante analysis is to compare the potential impact of an overall investment in the social sectors through broadly-defined policies and programs, against a counterfactual scenario 41 The idea of providing new computers to 1st graders is a clever one as students get to keep their computers when they move to the next grade, with every new entering cohort of 1st graders getting new computers. The budget document does not mention that the computers are netbooks called “bukis� which by definition are meant to work on the net, typically through wifi. But most Georgian schools do not have adequate Wifi for all students, thus severely limiting even the theoretical usability of the bukis, leave alone the actual practice in classrooms (Source: “Smart, Digitally Enhanced Learning Ecosystems: Bottlenecks to Sustainability in Georgia�, Eka Jeladze and Kai Pata, Sustainability, 2018, 10, 2672; doi:10.3390/su10082672). Page 104 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment where these investments are not made at all. Experience with the application of Cost-Benefit analysis shows that there is a positive and significant association of good cost-benefit analysis with good project performance.42 Rationale for investing in the program The principal reason for investing in the program is to accelerate the accumulation of human capital through improved education and health, with a special focus on the poor and vulnerable, through social protection. If left to private market forces, investment in human capital would be lower and unbalanced, leading to two undesirable outcomes: a lower level of productivity and growth, and an increase in inequality. Basic education, health and social protection have a relatively high content of public good. Although individuals benefit from increased education and health, society also benefits from having more educated and healthy individuals. If left to market forces, investment in health and education would be lower because many people would be left out because of their poverty, their social disadvantages, their location, or their inability to access education and health services.43 Inversely, public investment in human capital is likely to be higher and better distributed by definition, as the role of government is to provide and manage public goods. Increasing public investments in human capital would foster higher living standards, higher social stability, and higher growth. Moreover, by investing in educational and health equity, the distribution of living standards, income, and growth, would be broader based than if left to market forces, yielding a more sustainable and inclusive path to growth. Social protection, in the form of targeted social assistance and employment programs, plays a key role in helping households build and protect human capital. Social assistance provides poor and vulnerable households with the financial stability to invest in human capital services such as health clinics, schools and nutrient food for their children. Public employment services and promotion programs ensure that investments in human capital are fully utilized in working age especially for vulnerable categories of the 42 Squire, Lyn, and Herman van der Tak. 1995. Economic Analysis of Projects. Washington DC: World Bank. Belli, Pedro, Jock Anderson, Howard N. Barnum, John A. Dixon, and Jee-Peng Tan. 2001. Economic analysis of investment operations: analytical tools and practical applications. Washington DC: World Bank. Vawda, Ayesha Y., Peter Moock, J. Price Gittinger, and Harry Anthony Patrinos. 2003. “Economic Analysis of World Bank Education Projects and Project Outcomes.� International Journal of Educational Development 23: 645-660. 43 Kalhoff, Angela. 2014. Why Societies Need Public Goods. Critical Review of International Social and Political Philosophy, Vol. 17, No. 6, pp. 635–651. Page 105 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment population (i.e., those who are socially vulnerable, with lower education, less labor market information and networks, higher mobility constraints, less time due to family care responsibilities, people with disabilities and first labor market entrants) who have more difficulties to access jobs compared to the rest of the population if left to the market. Counterfactual Scenario Counterfactually, service provision of education and health by the private sector could leave a significant sector of the population—the poor and the underserved—behind. Private investors would be rational in not serving those populations because it may not be financially profitable. However, goods such as education and health, especially when built on well targeted social protection policies, yield significant positive externalities, including better and inclusive employment outcomes, where society benefits from having a better educated and healthier population. Hence, it is in the best interest of society to invest in goods that generate positive spill-over that cannot be captured by private investors. Hence, one can say that education and health have a large content of public goods, which provide a clear justification for governments to invest in them. The fact that education and health produce positive spillovers is what makes them quasi-public goods. Social stability, civilized behavior, and a healthy environment can be consumed by an individual at no cost to others (nonrival characteristic), and it would be very expensive to exclude others from consuming them (nonexcludable characteristic).44 It should be noted here that public provision may include private delivery, if the contracting of the private sector to deliver public services is more efficient operationally and/or financially.45 Hence, the counterfactual argument refers only to private investment in public goods. 44 The fact that education and health produce positive spillovers is what makes them quasi-public goods. Social stability, civilized behavior, and a healthy environment can be consumed by an individual at no cost to others (nonrival characteristic), and it would be very expensive to exclude others from consuming them (nonexcludable characteristic). Reiss, Julian. 2021. "Public Goods." The Stanford Encyclopedia of Philosophy (Fall 2021 Edition), Edward N. Zalta (ed.). 45 Roth, G.J. 1987. The Private Provision of Public Services in Developing Countries. New York: Oxford University Press World Bank. 2017; Public Private Partnerships Reference Guide Version 3. Washington DC: World Bank. Baum, Donald, Laura Lewis, Oni Lusk-Stover, and Harry Patrinos. 2014. What Matters Most for Engaging the Private Sector in Education: A Framework Paper. SABER Working Paper Series Number 8. Washington DC: World Bank; Clarke, David, and Aurelie Paviza. 2018. The Private Sector, Universal Health Coverage, and Primary Health Care. Technical Series on Primary Health Care. Geneva: World Health Organization. Page 106 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment Given the slower pace of economic activity due to the pandemic, and given the losses in learning and health, without the program the amount and pace of recovery would be significantly slower, with unwelcome social and human consequences. Although it may be possible for Georgia to come out of the crisis relatively unscathed, the pace of recovery and the access equity of recovery programs would be severely tested. The economic analysis of the program offers the counterfactual argument by not changing labor productivity, not increasing health investments and not increasing social protection. Value Added by the World Bank Given the tight fiscal envelope faced by Georgia after the COVID pandemic, increasing public investment in human capital is severely constrained, at a time when the country needs to recover learning and health losses produced during the year-long quarantine. The reduction in economic activity brought on by the pandemic, the need to invest quickly to recover learning losses, and the need to go beyond recovery to increase human capital and accelerate economic growth, sets the stage for a significant contribution by the World Bank on three fronts: (i) the actual provision of financing at a critical period in Georgian society; (ii) the provision of technical advice in the design, monitoring, and analysis of education, health, and social sector policies that go beyond the existing policy portfolio; the new portfolio will be implemented under a program-for-results framework, which makes public investment more efficient and accountable; and (iii) the lessons learned in the provision of distance education and distance health would be incorporated into the new program, leading to a shift in the nature of government intervention in the social sectors. These characteristics of the program add more value to public policies and programs that would be less possible without World Bank assistance. Economic Impact The program will contribute to Georgia’s development by increasing human capital through polices and interventions whose expected benefits exceed expected costs. In addition, the benefits and costs of the program would bring an added value over the counterfactual situation without the program. Finally, as the PAD demonstrates, the program is the best financial alternative available for accelerating human capital formation, productivity and growth. These assertions are demonstrated by the results of the Page 107 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment Benefit/Cost analysis where benefits and costs are monetized to the maximum extent possible, as described in the next section. Methodology and assumptions The ex-ante economic analysis is based on the estimation of the ratio of program benefits to program costs, the Benefit/Cost (B/C) ratio that determines the financial viability of an investment. This ratio is calculated by dividing the Net Present Value of Benefits (NPVB) over the Net Present Value of Costs (NPVC): NPVB B/C = NPVC (1) Bt ∑T t=0( ) (1+iop )t B/C = C t (2) ∑T t=0((1+iop )t ) where iop is the discount rate, and T is the maximum number of years of program impact on income. The discount rate is the interest rate used to discount a cash flow. It helps determine how much a series of future cash flows is worth as a single lump sum value today. A B/C ratio greater than 1 indicates that a project is profitable, as benefits are greater than costs. Inversely, a B/C ratio lower than 1 indicates that the cost of a project is larger than its benefits, making it unprofitable. Once the benefits and costs are measured and discounted, the results are presented in three ways: (i) the Benefit Cost ratio, (ii) the Net Present Value of the project, which is the value of discounted benefits minus discounted costs; and (iii) the Internal Rate of Return of the investment, which is the rate of interest that equates the discounted present value of expected benefits and the present value of the costs of the project.46 This methodological framework is used to assess the benefits of education, health, and social protection, under the broader framework of Program-for-Results, where the specific packages of policies and interventions are still to be defined in detail. However, the board benefits from each type of 46 Belli, Pedro, Jock Anderson, Howard N. Barnum, John A. Dixon, and Jee-Peng Tan. 2001. Op. cit. Page 108 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment program are delineated below. They are used as the base for estimating the monetary value of program impacts. To illustrate the methodology, one can use the case of education, where the economic analysis is straightforward. For an individual, there are benefits and costs in investing in an additional year of education beyond the primary level, as shown in Figure E1. The direct cost that the individual pays (plus the opportunity cost in terms of foregone earnings) must be lower than the benefits for the investment to make sense. The B/C methodology allows to calculate the Net Present Value (NPV) of costs and benefits to compare the difference between the discounted values of the streams of benefits and costs. This framework relies on long standing analyses showing that the salary structure of any labor market is a function of the educational level of the economically active population (Barro 1991; Castelló-Climent 2013; Hanushek and Kimko 2000; Lucas 1988; Mankiw, Romer, and Weill 1992; Montenegro and Patrinos 2022; Psacharopoulos and Patrinos 2018).47 Thus, the future income of the beneficiary cohorts depends on their education level. In addition to the impact of increased attainment on salaries, there is clear evidence that increased learning has a high impact on wages and on lifetime income. Data from 22 countries on the cognitive skills of adults in the labor force shows that better knowledge about math can make a large impact on salaries. On average, a one-standard deviation increase in numeracy skills is associated with an 18 percent wage increase among workers of prime working age. In six of the 22 countries the returns to better numerical knowledge were associated with an increase in salaries above 21 percent, while in eight countries the wage differentials were between 12 and 15 percent in favor of workers with better 47 Barro, Robert J., 1991. “Economic Growth in a Cross Section of Countries.� The Quarterly Journal of Economics, 106(2): 407-443; Castelló-Climent, Amparo, 2013. “Education and Economic Growth.� Background paper prepared for the Education for All Global Monitoring Report 2013/4 Teaching and Learning: Achieving Quality for All. Paris: UNESCO Hanushek, Eric A. and Dennis D. Kimko. 2000. “Schooling, Labor Force Quality, and the Growth of Nations.� American Economic Review 90(5): 1184- 208; Lucas, Robert E., 1988. “On the Mechanics of Economic Development.� Journal of Monetary Economics, 22(1): 33-42; Mankiw, N. Gregory, David Romer, and David N. Weill, 1992. “A Contribution to the Empirics of Economic Growth.� The Quarterly Journal of Economics, 107(2): 407-437; Montenegro, Claudio and Harry Anthony Patrinos. 2022. “A Data Set of Comparable Estimates of the Privat; Rate of Return to Schooling in the World, 1970-2014.� International Journal of Manpower, forthcoming; Psacharopoulos, George and Harry Anthony Patrinos. 2018. “Returns to investment in education: a decennial review of the global literature.� Education Economics, 26(5): 445-458. Page 109 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment numerical skills. The largest return to better numerical skills was found in the United Sates, with a wage differential of 28 percent.48 Figure E1: Framework for Benefit and Cost Analysis of Education Source: Figure 1 in Jimenez and Patrinos, 2008 In the case of health, the methodology for estimation of the B/C coefficient is the same, but the estimation of benefits is different. Two approaches are used to quantify the benefits of health policies and programs: quality-adjusted life years (QALY), which is based on the effect of interventions on the perceived quality of life benefits by the individual, and disability-adjusted life years (DALYs) which is an indicator of the time lived with a disability and the time lost due to premature mortality. There are several ways to measure QUALYs but DAILYs already have world tables with the duration of time lost due to premature mortality (calculated using standard expected years of life lost with model life-tables) 48 Hanushek, Eric A., Guido Schwerdt, Simon Wiederhold, and Ludger Woessmann. 2013.� Returns to Skills around the World: Evidence from PIAAC.� National Bureau of Economic Research Working Paper 19762. Cambridge MA: NBER. Page 110 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment and time lost due to the weighted reduction in physical capacity caused by morbidity.49 Currently, the top five sources of risk affecting death and disability in Georgia are, in order of priority: high blood pressure, bad dietary habits, tobacco use, diabetes, and obesity. In addition, the top five causes of death are: ischemic heart disease, stroke, hypertensive heart disease, lung cancer, and Alzheimer’s disease.50 Calculating the DALYs for the program requires a detailed listing of the health interventions, which are yet to be defined. As a result, the approach taken in the estimation model is to propose modest reductions in the out-of-pocket expenditures incurred by the lowest 40 percent of the population, under the assumption that they would be the main targets of health interventions associated with the program, and that improved health policies and programs should eventually result in net savings to poor families, which translate into an increased income. For public investment decisions, such as the case of the proposed program for Georgia, costs include public expenditures in education and social protection and lifetime earnings represent the benefits to be used in the estimation of the social rate of return, under the assumption that observed wages are a good approximation of the marginal product of labor. Ideally, the social benefits of education should include positive external effects in health (savings from preventive health practices, family planning and reproductive health savings, and reductions in the number of days lost to disease). To approximate these effects, this analysis includes estimates of net savings from positive impacts of the health component of the program. The point of reference for the economic analysis is the situation without the project, that is, what would have happened if the project had not been realized. This reference point is also called the Basic (No Project) Scenario. This Scenario is compared to the results obtained after the project was undertaken under expected conditions for implementation. In addition, for purposes of this evaluation, the Basic Scenario is compared to Conservative, Intermediate, and Optimistic scenarios to determine the impact of possible variations in future cash flows. The scenarios are based on similar reductions in repetition rates observed in countries that have undergone education reform. Since the income stream of 49 World Health Organization. 2020. “Global Health Estimates 2020: Disease burden by Cause, Age, Sex, by Country and by Region, 2000-2019.� Geneva: WHO. Weinstein, Milton C., George Torrance, and Alistair McGuire. 2009. QALYS: The Basics. Value in Health Vol. 12, Supplement 1. Homedes, Nuria. 1996. “The Disability-Adjusted Life Year (DALY) Definition, Measurement and Potential Use.� Human Capital Development Working Paper 68. Washington DC: World Bank. 50 Source: http://www.healthdata.org/georgia Page 111 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment graduates is projected for their entire working life, these types of comparative scenarios are useful to determine the robustness of the results. The economic analysis contrast the project costs with its expected benefits in terms of increased salaries due to increased productivity, and increased disposable income due to health savings, targeted social assistance benefits. Costs include the total project investment; the financial cost of project funding, and the estimated recurrent costs associated with project implementation. The project benefits derived from investing in education, health, and social protection include: The increase in wages derived from higher labor productivity due to the improved quality of education and health services promoted by the project, The increase in disposable income for an additional 12 percent of households in the poorest 2 quintiles of the income distribution. These households targeted by the project would eventually benefit from increased years of schooling for their children, net reductions in sick days and in health expenditures associated with preventive health practices, and an increase in their lifetime income.51 It is important to note that, for purpose of the economic analysis, the gains apply only for the additional number of poor households that would receive social protection benefits, as the current number of beneficiaries are part of the Basic scenario. The updated PMT scoring formula to be piloted and roll out nationally under the Program is expected to increase the number of beneficiaries from the poorest quintile by about 12%. This is the net gain used in the economic analysis. Assumptions on the Discount Rate Estimated costs and benefits of the project over the working life of the target population are brought to their present value using a discount rate of 5 percent, and presented in the form of three indicators: (a) benefit/cost ratio (B/C), which measures the ratio of discounted benefits to discounted costs; (b) the net present value (NPV) of the total benefits of the project, which is the value of the discounted benefits 51 There are now about 175,000 households already receiving TSA and child benefits, or about 80.6% of the 217,134 households in the poorest quintile of the population. An additional 12% amounts to 26,057 beneficiary households. Page 112 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment once the discounted costs invested to produce them are subtracted; and (c) the internal rate of return (IRR) of the project, which represents the rate of interest that equates the discounted present value of expected benefits and the present value of the costs associated to the project. To provide a sensitivity analysis, these results are computed for three different scenarios: Conservative, Intermediate, and Optimistic. The 5 percent discount rate is a conservative one that falls within the average range outlined in the Cost-Benefit methodology. The discount rate is the interest rate used to determine the net present value of future cash flows produced by the project. As such, it helps determine if the future cash flows from the project will be worth more than the capital outlay needed to fund it. For evaluating a project, the discount rate used in the calculations should reflect the likely returns of funds in their best relevant alternative use, that is, the opportunity cost of capital or investment rate of interest and should also reflect the marginal rate at which savers in a country willingly put their money into savings.52 In other words, the social discount rate should reflect the rate at which a society would be willing to trade present for future consumption. What is that discount rate is open to discussion, as the discount rate for individuals tends to be higher than the discount rate for public investment, where the future consumption of public goods tends to be less urgent for an individual than the discount rate used for future private consumption. The reason is simple: in private consumption the individual is subject to legal and fiscal constraints which—when incorporating them into the cost of future consumption—yields a higher discount rate than in the case of government consumption that cannot tax itself.53 Also, the expected utility derived by an individual from consuming a private good would tend to have a different time preference than in the consumption of a public good hose benefits may continue to be captured by society even after the individual is dead. Given these arguments, a discount rate between 3 and 4 percent has been used for European public investment projects for decades. 52 Belli et al, 2008, p. 222. 53 Kula, Erhun. 2006. The Social Discount Rate in Cost-Benefit Analysis: The British Experience and Lessons to be Learned. Working Paper 2006-19. Milan European Economic Workshops. Feldstein, M. (1964). “The Social Time Preference Discount Rate in Cost Benefit Analysis,� The Economic Journal, 74: 360–379. Page 113 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment Evidence from Latin America, which reflects the time preference of individuals and society more commonly associated with developing and Intermediate-income countries, suggest that a discount rate in the range of 4-5 percent is more appropriate.54 In the case of programs-for-results, where specific policies are to be defined and then focus is on the results, a 5 percent discount rate has been used for Jordan,55 where a similar set of impacts on productivity and lifetime incomes are to be expected as a result of increases in the quality of education. If interest rates charged by the Central Bank are used as proxy for the social discount rate, the recent evidence from Georgia suggests that 7 percent as an appropriate rate for this program.56 However, this rate fluctuates within the year depending on the intentions of the Central Bank to manage inflation and credit demands. From this perspective, a long- term rate of 5 percent discount rate is considered more appropriate. The exchange rate used to convert the project benefits and recurrent costs collected in Georgian Laris (GEL) to United States Dollars (US$) is the annual average nominal exchange rate for the last year (3.3 GEL per US$). It is anticipated that the project benefits channeled through increased labor productivity will begin until the fifth year of the project because it entails education quality improvement, which is a process that requires a few years to build up and consolidate. It is also assumed that these benefits will reach at least 10 cohorts of students enrolled in Georgia’s education system for all scenarios to provide a more conservative analysis. The analysis is based on the best and most up-to-date data available on student enrollment (including internal efficiency indicators, such as repetition and dropout rates); estimates of the returns to education for Georgia; employment prospects; household income distribution; and household 54 Lopez, Humberto. 2008. The Social Discount Rate: Estimates for Nine Latin American Countries. The World Bank Human Development Network Education Team, Policy Research Working Paper 4639. Washington DC: World Bank. 55 World Bank. 2017. “Education Reform Support Program-for-Results.� Program Appraisal Document No. 121282-JO. Education Global Practice, Middle East and North Africa Region. Washington DC: World Bank. World bank 2019. Op. cit. 56 The Central Bank’s interest rate in Georgia ranges between 6.5 and 9.5 percent during the past ten years. https://countryeconomy.com/key-rates/georgia. The annual interest on mortgage loans is about 6.7% https://www.terabank.ge/en/retail/loans/mortgage-loans Page 114 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment expenditures. The data was retrieved from the National Statistics Office of Georgia (GEOSTAT), UNESCO Institute of Statistics (UNESCO-UIS), the National Bank of Georgia (NBG), and additional literature review. Benefits and beneficiaries Education The benefit of investing in another year of education is the gain in earnings for the rest of a person’s working life, estimated at 45 years for a secondary school leaver who retires at 65 years of age. Because completion rates are almost 100 percent, any changes in educational performance would be due to increased learning. Policies affecting education quality could include improvements in pre-service training for teachers, improvements in school-based management, improvements in results measurement, reporting, and accountability, and improvements in the physical and digital infrastructure. Such improvements would lead to a better educated labor force and a corresponding increase in labor productivity. What is known is that, on average, a one-standard-deviation increase in numeracy skills is associated with an 18 percent wage increase among prime-age workers.57 Hence, improvements in education quality will have a net positive effect on salaries and a corresponding increase in lifetime incomes that can be attributed to the program. Analyses using the Harmonized Learning Outcomes (HLO) dataset – part of the Human Capital Index / Project – show that a 1 percent increase in learning—as represented by test scores—is associated with a 7.2 percent increase in annual growth.58 This is clear indication that designing and implementing projects that have a positive Benefit/Cost ratio is a first, but decisive step, in improving human capital and, by extension, inclusive economic growth. According to the information provided by the Ministry of Education, Science, Culture and Sport of Georgia (MESCSG) to GEOSTAT, there were 609,095 children and youth enrolled in primary, lower 57 Hanushek, Eric A., Guido Schwerdt, Simon Wiederhold, and Ludger Woessmann. 2015."Returns to Skills around the World: Evidence from PIAAC." European Economic Review Vol. 73 pp. 103-130. A previous draft version is available at https://www.nber.org/system/files/working_papers/w19762/w19762.pdf 58 Angrist, Noam, Simeon Djankov, Pinelopi K. Goldberg, and Harry A. Patrinos. 2021. “Measuring Human Capital Using Global Learning Data.� Nature Vol. 592, pp. 403–408 (March 2021). https://doi.org/10.1038/s41586-021-03323-7 Page 115 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment secondary and upper secondary education during school year 2020-2021 (Table E1). The proportion of students enrolled in primary education is 55 percent, while the percentage of students in lower secondary and upper secondary education are 23 and 22 percent. Table E1. Enrollment: School year 2020-2021 Educational level Grade Enrollment (Number) (%) 1 59 067 9.7 2 55 655 9.1 3 56 526 9.3 Primary education 4 57 016 9.4 5 60 633 10.0 6 47 675 7.8 7 47 574 7.8 Lower secondary 8 45 475 7.5 education 9 45 031 7.4 10 49 546 8.1 Upper secondary 11 46 082 7.6 education 12 38 815 6.4 Total 609 095 100.0 Georgia’s education system has reached almost universal coverage in basic and lower secondary education, with a 100 percent completion rates for both educational levels (UNICEF 2020). Additionally, the repetition and dropout rates in all grades of primary and lower secondary education are low, with most of them below 1 percent (Table E2). However, upper secondary education remains the bottleneck Page 116 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment of Georgia’s education system because only 66 percent of youth complete this educational level (UNICEF 2020). Table E2. Repetition and dropout rates (%) by grade Educational Grade Repetition Dropout level (%) (%) 1 0.3 0.0 2 0.2 0.3 Primary 3 0.1 0.2 education 4 0.1 0.4 5 0.3 0.5 6 0.3 0.5 7 0.8 0.6 Lower secondary 8 0.9 0.9 education 9 1.1 0.2 10 1.0 12.9 Upper secondary 11 1.0 4.3 education 12 1.0 2.5 Since the repetition and dropout rates are already quite low in primary and lower secondary education, the project will focus their efforts on education quality enhancement to improve human capital. It is assumed that the whole student population of basic, lower secondary and upper secondary education will benefit from better education quality as this will be a nationwide intervention that will positively impact the wages of the students who had access to better quality education. Page 117 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment The reference minimum wage used in this analysis corresponds to the wages reported for the regions of Racha-Lechumi and Lower Svaneti, where the average wage amounts to GEL$ 605 (US$ 183) per month (Wage Centre 2021). The first step to estimate the wage gains due to better quality of education and health promoted by the project is to compute the annual wage that the graduates of each educational grade obtain when inserted in the labor market based on the reference minimum wage (Table E3). This was done using the rate of return to education estimated by Montenegro and Patrinos (2022), which amounts to 7.7 for each additional year of education in Georgia. Table E3 Annual wage (US$) by schooling years Annual wage Educational level Schooling years (US$) 1 2,231 2 2,403 Primary 3 2,588 education 4 2,787 5 3,002 6 3,233 7 3,482 Lower secondary 8 3,750 education 9 4,039 10 4,350 Upper secondary 11 4,685 education 12 5,045 Source: Own estimations based on Montenegro and Patrinos (2022), NBG (2021) and Wage Centre (2021) data Page 118 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment Students who receive better quality education will collect higher wages once inserted in the labor market. Participation and unemployment rates determine the number of impacted students that will be able to find a job in the Georgian labor market. According to GEOSTAT (2021), the participation and unemployment rates in 2020 were 50.5 percent and 18.5 percent. The model also assumes that the enhanced labor productivity brought in by better quality of education will lead to the following wage increases for each scenario: (i) 2 percent for the conservative scenario; (ii) 4 percent for the Intermediate scenario; and (iii) 7 percent for the optimistic scenario. Thus, the anticipated wages for the employed population with access to better education quality are shown in Table E4. Table E4. Annual wage (US$) by schooling years and scenario Educational Schooling years Conservative (2%) Intermediate Optimistic (7%) level (4%) (US$) (US$) (US$) 1 2,276 2,320 2,387 2 2,451 2,499 2,571 Primary 3 2,640 2,691 2,769 education 4 2,843 2,899 2,982 5 3,062 3,122 3,212 6 3,298 3,362 3,459 7 3,552 3,621 3,726 Lower secondary 8 3,825 3,900 4,013 education 9 4,120 4,200 4,321 10 4,437 4,524 4,654 Upper secondary 11 4,778 4,872 5,013 education 12 5,146 5,247 5,399 Health Page 119 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment Investments in health associated with the program could include three broad policies aimed at improving the efficiency of health service provision and a corresponding lower cost of healthcare for consumers, broader institutional coverage leading to improved health access equity, and net reductions in government procurement costs through a better management of cost drivers in the health system.59 To complement social protection, healthcare programs could focus on: (i) improving coverage among the extreme poor and underserved populations; (ii) reorganizing the delivery of primary health care away from hospitals and more towards secondary and tertiary level delivery units; and (iii) reducing the cost of inputs by managing bulk purchases of medicines, expanding preventive care, including vaccinations and preventive health practices, and relying more on community health nurses to deliver care. 60 The net results of these broader policies would be a reduction in the number of work days lost to illness, and an increase in the number of productive years. These results would lead to net increases in productivity and a corresponding increase in income and economic growth, along with a better quality of life.61 The benefits stemming from a new health policy aiming to reduce health expenditures for the poorest households are expected to increment gradually over four years, with project benefits expected to last well beyond project implementation because it is expected to produce permanent changes in health, and changes in health-related behavior. Health benefits for children, for example, will impact them for most of their entire life, positively affecting their future labor productivity and therefore their lifetime earnings. Accordingly, the benefits of the health policy change targeting the poorest 40 percent of households are estimated for: (i) 20 years in the case of the conservative scenario; (ii) 25 years in the case of the Intermediate scenario; and (iii) 30 years in the case of the optimistic scenario. The Households Income and Expenditure Survey 2020 collected by GEOSTAT (2021) estimates that there are 1,085,670 households in Georgia with an average annual health expenditure of GEL$ 1,047 (US$ 59 World Bank. 2019. “Romania Health Program for Results.� Project Appraisal Document P169927. Education Global Practice, Europe and Central Asia. Washington DC: World Bank. 60 World Bank. 2021. “Needs Index in Georgia: Assessment and Proposed Revisions.� Washington DC: World Bank.; Honorati, Maddalena, Roberto Claudio Sormani, and Ludovico Carraro. 2021. “Assessing the Targeting System in Georgia: Proposed Reform Options. Europe and Central Asia.� Education Global Practice, Europe and Central Asia. Washington DC: World Bank. 61 World Bank. 2020. The Human Capital Index 2020 Update: Human Capital in the Time of COVID-19. World Bank, Washington, DC: World Bank. https://openknowledge.worldbank.org/handle/10986/34432 Page 120 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment 321). Households in the first two quintiles of the income distribution spend an average of GEL$ 536 (US$ 164) and GEL$ 769 (US$ 236) annually (Table E5), whereas the average health expenditure of the richest 20 percent of the households is GEL$ 1,959 (US$ 601) per year. Table E5. Average annual household expenditure in health (GEL$ & US$) by quintile, 2020 Households Expenditure in health Quintile (Number) (GEL$) (US$) 1 217,134 535.5 164.2 2 217,134 769.0 235.8 3 217,134 844.4 258.9 4 217,134 1,134.2 347.7 5 217,134 1,958.6 600.5 Source: Own estimations based on Households Income and Expenditure Survey 2020 (GeoStat 2021) Since one of the key objectives of this project is to reduce the health expenditures of the poorest 40 percent of households, the model estimates the household disposable income increase due to savings in health expenditure for 434,268 households included in the first two quintiles of the income distribution. The model uses the following assumptions for each scenario: (i) a 20 percent decrease in household health expenditure in the case of the Conservative scenario; (ii) a 25 percent reduction in household health expenditure for the Intermediate scenario; and (iii) a 50 percent reduction in household health expenditure for the Optimistic scenario. In all cases the savings accrue for 25 years. Social Protection Investment in social protection in the context of this Program refers to poverty targeted social assistance benefits, public employment services and employment promotion programs for youth and vulnerable jobseekers. These policies aim at reducing the poverty gap among extreme poor families, enabling them to improve their nutrition, improve their access to health and education, and bring their productivity and income closer to the mainstream. Page 121 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment Specific benefits from the Social Protection part of the Program would be generated in four different areas: Improvements in the quality of short-term vocational training and VET resulting from the development and/or update of skills, especially those in short supply in the private sector. The increase in quality will improve labor productivity, which will increase the lifetime income of 6 cohorts of TVET graduates. Conservatively, quality improvements in TVET would result in net increases of 1 percent in the salaries of 20,000 graduates of short-term training projected to be benefitted by the program, under the assumption of a 70 percent employment rate. Increased disposable income by $75 per month for an additional 12 percent of households in the poorest quintile. An updated scoring formula will improve targeting of beneficiaries on the bottom quintile and reduced leakage to upper quintiles is a benefit generated by the program. Improved job matching coming from the revamped Worknet online matching tool, resulting in shorter job search and shorter duration of unemployment. Such a benefit is assumed to increase the employment rate by 2 percent points for workers between the age of entry into the labor force and 35 years of age. This age range is a point of departure for the calculations, as the unemployment spells for workers older than 40 are assumed to affect a low percentage of the force. After the age of 40, workers tend to settle into permanent jobs. The digitalization and automation of the registration, eligibility verification and enrolment processes for targeted social assistance will is expected to reduce administrative costs to process applications. These savings, estimated as 10 percent of the total amount of targeted benefits for 20,000 people, amount to US$ 150,000 per year. These benefits may be initially offset by the additional investments during the first two years of the project, to pay for software development, training of social agents etc. to ensure that verification and recertification of vulnerability scores is done electronically. Hence, net savings will start on year 3 of the program and continue indefinitely. Fiscal Impact Analysis Government expenditures for 2021 are projected to be GEL 14,250 million (US$ 4,318 million at a 3.3 Gel/US$ exchange rate).62 Estimated government expenditures in education (including higher education) add up to US$ 554 million in 2021, along with US$ 470 million in health, and US$ 1,037 million in social protection (including social security and pensions), for a total of US$ 2,062 million for 62 Ministry of Finance. 2021. Medium-Term Forecast Reconciliation. Tbilisi. https://www.mof.ge/en/5262 Page 122 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment these social sectors, or 48 percent of the government’s budget. On a yearly basis, the program would add GEL 264 million (US$ 80 million) every year for five years, to the budget for social protection. On a yearly basis this means that the program would increase social sector expenditures by 3.87 percent for five years, and the government budget by 1.85 percent, both of which are modest and temporary increases in government expenditures. Education represents 27 percent of social sector expenditures; health 23 percent, and social protection 50 percent. Keeping these percentages constant, and assuming an annual rate of growth of 2.5 percent for the next 8 years, it is possible to visually represent the modest fiscal impact of the program, as shown in Fig. E2. Fig. E2 Projected impact of the program on social sector expenditures (US$) Under current GDP figures, the financial burden of the project is manageable (Table E6). The estimated annual disbursement by the project represents 3.88 percent of total spending on education, health and social protection, 1.63 percent of the government’s budget, and 0.46 percent of projected GDP for 2021. If the benefits from the program imply substantial gains in the income stream of the target population, net tax income would offset the increase in government outlays. The estimated gains in household Page 123 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment income generated by the program have a net present value of more than US$ 500 million (as it is shown in the next section). Based on Georgia’s 20 percent tax rate on income63, the added wage gain would generate about US$100 million in additional tax revenues, helping offset program costs indirectly. Table E6. Indicators of Fiscal Impact Projected Social Sector spending 2021 (US$ Million) 2,062 Program disbursement per year (US$ Million) 80 Total Govt. spending 2021 (US$ Million) 4,907 GDP 2021 (US$ Million) 17,396 Govt. budget as % of GDP 28.2 Program annual disbursement as % of Social Sector spending 3.88 Program annual disbursement as % of Govt. spending 1.63 Program annual disbursement as % of GDP 0.46 Sources: Social sector spending: World Bank PASD Draft Govt budget and GDP: Min. of Finance https://www.mof.ge/en/5262 Because the Government’s long-term goals is to increase the human capital potential for sustaining the operation after implementation, the probability of an increase on GDP above current trends is high, in which case the program costs will represent a smaller percentage of Government expenditures. What is important, however, is that the distribution of the program benefits would mostly accrue to the lower two quintiles of the population, making GDP gains more equitable, and growth more sustainable. Once program funds are completely disbursed, the additional costs to the Government will be low, as then the program focuses on efficiency and equity, producing net savings to the government. In addition 63 https://investingeorgia.org/en/georgia/taxation Page 124 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment to the direct benefits captured by specific sectors and beneficiaries, the program also produces additional savings to the government: About 50 percent of the 2,300 schools in Georgia have relatively low student-to-teacher ratios. To improve operational efficiency these schools will have to rationalize the configuration of students per classroom and consider measures such as multigrade classrooms and clustering. It is projected that savings from rationalization will be accrued in the process. Net savings starting in year 5 are estimated at an average of US$ 1,500/school in saved heating and maintenance costs, amounting to total net savings of US$72 million. Introduction of diagnosis-related group (DRG) payments will reduce outlay of the Universal Health Service (UHS) on hospital payments. Also, better utilization of Primary Healthcare facilities will reduce burden on hospitals, especially for outpatient care, better public health surveillance with Program investments will also lead to lower future morbidity and mortality. Net savings of these two measures are initially estimated to be at 15% of the government’s expenditures in health, which would be equal to US$ 70 million in 2021. Pharmaceutical reform will lead to lower UHS expenditure on drugs, resulting in higher disposable income amongst lower income households, with multiplier effects with positive fiscal implications on the revenue side. If these savings are passed along to consumers, they will amount to US$ 5 million in the first year of the program, and US$ 21 million per year after the fifth year. Another benefit is the digitalization and modernization of social assistance will lead to fewer errors of inclusion and exclusion. The sum of these savings is equal to the amount wrongly assigned to wrongly included households. Funds allocated by errors of inclusion will be transferred to wrongly excluded households. Hence, although net savings are zero, properly targeting households will reflect the net gain in program efficiency. As will be seen in the next section, the net savings to the Government far exceed the US$ 400 million disbursed by the project. Model Estimation of Economic Benefits and Costs Total benefits The results of the model show that the NPV of project benefits would be US$ 1,274 million under the conservative scenario (Table E7), US$ 1,781 million under the Intermediate scenario (Table E8); and that the benefits would increase up to US$ 2,450 million in the case of the optimistic scenario (Table E9). Page 125 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment Table E7. Cash flow in conservative scenario (US$) Type of Benefit NPV Higher earnings due to education quality improvement 313,923,491 Household income increase due to savings in health expenditure 192,648,761 Benefits of SP/TSA improvements 339,611,706 Government savings 427,533,574 Total benefits 1,273,717,533 Table E8. Cash flow in intermediate scenario (US$) Type of Benefit NPV Higher earnings due to education quality improvement 627,846,983 Household income increase due to savings in health expenditure 276,279,763 Benefits of SP/TSA improvements 443,230,543 Government savings 433,685,189 Total benefits 1,781,042,478 Table E9. Cash flow in optimistic scenario (USD$) Type of Benefit NPV Higher earnings due to education quality improvement 1,098,732,220 Household income increase due to savings in health expenditure 364,884,606 Benefits of SP/TSA improvements 546,849,380 Government savings 439,836,804 Total benefits 2,450,303,009 Page 126 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment Program costs The main cost of project implementation in the World Bank loan, which amounts to US$400 million, disbursed over five years. Additionally, the economic costs include: (i) the financial cost of the grant, and (ii) the estimated recurrent costs of implementing the policy changes promoted by the project. The recurrent costs comprise the costs of launching, maintaining and operating the policy changes to promote improvements in education, health, and social protection. These costs would not happen in the absence of the project and are assumed to be funded with domestic funds. Since this project entails a broad-based policy and implementation program, the model assumes that the Georgian Government will need to invest US$ 400,000 per year in the form of recurrent cost for project implementation for a period of ten years. The model assumes that the economic cost for all three scenarios will be the same because there is no way to estimate different recurrent cost (i.e., the main source of economic cost variation) for each scenario due to the broad base of the project, with no specific components defined within the project framework, which will be useful to differentiate the recurrent cost for each scenario. Table E10 shows the NPV of project costs for all scenarios. When discounting to the present the five disbursements of World Bank’s loan, the NPV of World Bank’s investment amounts to US$ 162 million for all three scenarios, while the financial cost directly associated with the project investment adds up to US$ 2.1 million and the estimated recurrent costs are US$ 210 million. Table E10. NPV of project costs (US$) Cost NPV (US$) Recurrent costs 209,923,056 Financial cost 2,069,835 Principal payment 161,735,789 Page 127 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment Total 373,728,680 Benefit/Cost ratio and sensitivity analysis The model yields positive NPV for all three scenarios at a discount rate of 5%, with and B/C ratios ranging from 1.8 to 3.4, depending on the scenario, when comparing the estimated benefits in the form of higher wages due to education quality improvement, household disposable income increase as a result of health savings for the poorest 40 percent of the households, income increases from social benefits channeled through social protection policies, and net government savings arising from gains in operational efficiency. Under the conservative scenario, which assumes the lowest possible benefits using a 5 percent discount rate, the estimated NPV of net benefits is US$ 553 million, with an Internal Rate of Return (IRR) of 20.8 percent and a B/C ratio of 1.8 (Table E11).64 Table E11. Economic indicators for the conservative scenario Indicator Value Net present value of benefits (US$) 1,273,717,533 Net present value of costs (US$) 373,728,680 Net present value of investment 346,358,134 (US$) Net present value of net benefits 553,630,720 (US$) Internal rate of return (%) 20.8 64 The Internal Rate of Return (IRR) is a discount rate that makes the net present value (NPV) of all cash flows equal to zero in a discounted cash flow analysis. The higher the IRR, the more desirable the investment. The IRR is a uniform metric for investments of different types, allowing for comparison of different investment alternatives. Page 128 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment Benefit/cost ratio 1.8 Payback period (years) 7 In the case of the Intermediate scenario, the model yields a NPV of net benefits of US$ 1,061million, with an IRR of 30.0 percent and a B/C ratio of 2.5 (Table E12). Table E12 Economic indicators for the intermediate scenario Indicator Value Net present value of benefits (US$) 1,781,042,478 Net present value of costs (US$) 373,728,680 Net present value of investment (US$) 346,358,134 Net present value of net benefits (US$) 1,060,955,664 Internal rate of return (%) 30.0 Benefit/cost ratio 2.5 Payback period (years) 5 The optimistic scenario provides the most promising results, with a project NPV of US$ 1,730 million, an IRR of 41.3 percent and a B/C ratio of 3.4 (Table E13). Table E13. Economic indicators for the optimistic scenario Indicator Value Net present value of benefits (US$) 2,450,303,009 Net present value of costs (US$) 373,728,680 Net present value of investment 346,358,134 (US$) Page 129 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment Net present value of net benefits 1,730,216,196 (US$) Internal rate of return (%) 41.2 Benefit/cost ratio 3.4 Payback period (years) 5 The results of the B/C analysis show that under the most likely intermediate scenario, the program is economically feasible, leading to positive gains in human capital, increasing incomes, and fostering growth. The economic evaluation of a Program for Results operation shows the following results on two key questions: Is public provision and/or financing appropriate for the Program? Yes. Without public financing of the program, investment in human capital would be lower for two reasons: (i) underinvestment by private providers of education and health services due to low or negative returns that would result from serving extremely poor, disadvantaged, and less accessible populations, and (ii) underinvestment in the design and implementation of policy reforms and programs aimed at improving human capital, due to the government’s tight fiscal envelope for the next several years. What is the economic impact of the Program, as being currently implemented or planned? Without the program, benefits accrued to education, health, and social protection would be lower than with the program. For example, using the education component alone, lifetime income without the program would have been US$ 15,625 million. With the program, total lifetime income under the intermediate scenario would be US$ 16,250 million, yielding an additional net benefit of US$ 625 million attributable to the program. Such an amount can be accrued to the value added of the World Bank, as it would not have been possible without the project. Page 130 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment Annex 1: All Human Development Expenditures of the Government [Yellow highlighting indicates the items which either wholly or partially constitute the supported Program: Amounts in thousands of nominal Georgian Lari] Mean executed Planned Executed Executed /budgete Budget Budget in Budget in d in past Code Budget Item for 2021 2020 2019 3 years Develop and manage policies in the field of IDPs from the occupied territories in the field of labor, 27 01 01 health and social protection 11,016 12,409 12,278 11,901 27 01 02 Medical practice regulation program 5,672 4,236 3,870 4,593 Manage the Disease Control and Epidemiological 27 01 03 Safety Program 11,300 21,012 19,126 17,146 27 01 04 Managing social protection programs 13,480 14,653 24,032 17,388 Management of state care, protection and 27 01 05 assistance to victims of human trafficking 6,927 4,881 1,409 4,406 Emergency coordination and emergency 27 01 06 management 4,255 3,965 2,494 3,571 27 01 07 Providing IDPs, eco-migrants and livelihoods 5,510 4,231 332 3,358 27 01 08 Management of employment promotion services 1,213 364 - 660 27 01 09 Managing health care programs 4,480 938 - 1,806 2,600,00 2,247,13 1,938,21 2,261,78 27 02 01 Provision of pension to the population 0 8 2 3 Page 131 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment Mean executed Planned Executed Executed /budgete Budget Budget in Budget in d in past Code Budget Item for 2021 2020 2019 3 years 27 02 02 Social assistance to target groups of the population 816,000 794,687 741,286 783,991 27 02 03 Social rehabilitation and child care 40,000 32,487 31,793 34,760 27 02 04 Social benefits in a highland settlement 66,300 59,285 52,795 59,460 Provide state care, protection and assistance to 27 02 05 victims of human trafficking 7,600 7,095 6,337 7,011 Social assistance to the population due to the deterioration of the socio-economic situation caused 27 02 06 by the new coronavirus 250,000 708,073 - 319,358 27 03 01 Protection of the general health of the population 800,000 964,278 828,674 864,317 27 03 02 Public health protection 96,847 100,681 109,783 102,437 Provision of medical services to the population in 27 03 03 priority areas 651,235 503,352 197,777 450,788 27 03 04 Postgraduate medical education 500 68 615 394 27 03 05 Management of state clinics - 2,937 - 979 27 04 Rehabilitation and equipping of medical institutions 30,000 50,725 6,248 28,991 27 05 Labor and Employment System Reform Program 8,860 2,087 3,023 4,657 Page 132 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment Mean executed Planned Executed Executed /budgete Budget Budget in Budget in d in past Code Budget Item for 2021 2020 2019 3 years Reintegration assistance for migrants returning to 27 06 01 Georgia 650 432 650 577 27 06 02 Managing eco-migrant migration 5,000 5,361 4,107 4,823 Creating social and housing conditions for the 27 06 03 resettlement of IDPS 55,000 68,823 63,842 62,555 Promoting the integration of persons with 27 06 04 international protection 85 119 - 68 27 06 05 Livelihood program 1,100 772 80 651 Economic Participation, Housing and Social Infrastructure for Internally Displaced Persons and 27 06 06 Host Communities (KfW) - 16,050 6,268 7,439 Develop state policies and programs in the fields of 32 01 education, science, culture and sports 42,415 33,146 34,532 36,698 32 02 01 Funding for secondary schools 894,000 794,997 710,365 799,787 32 02 02 Promoting teacher professional development 10,282 9,710 12,400 10,797 32 02 03 Providing a safe educational environment 18,750 17,174 17,101 17,675 32 02 04 Encourage successful students 985 354 689 676 Providing educational and living conditions for 32 02 05 especially talented students 240 225 232 232 Page 133 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment Mean executed Planned Executed Executed /budgete Budget Budget in Budget in d in past Code Budget Item for 2021 2020 2019 3 years 32 02 06 Provide students with textbooks 26,300 23,845 23,460 24,535 Financial assistance to teachers and administrative- 32 02 07 technical staff in the occupied regions 4,185 15 1,466 1,889 Access to general education for accused and 32 02 08 convicted persons 250 4,153 4,188 2,864 Develop and facilitate the implementation of the 32 02 09 National Curriculum 380 229 212 274 32 02 10 Providing transportation for public school students 13,550 308 324 4,727 32 02 11 Program " My First Computer " 43,000 12,232 27,613 27,615 32 02 12 Promoting general education 1,600 31,350 32,133 21,694 32 02 13 Promoting general education reform 16,000 1,130 1,329 6,153 32 02 14 Relax and learn together - 12,617 23,821 12,146 32 03 01 Promoting the development of vocational education 60,000 42,422 41,053 47,825 Access to vocational education for convicts and ex- 32 03 02 prisoners 100 8 188 99 32 03 03 Vocational training of national minorities 2,400 2,128 2,330 2,286 Page 134 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment Mean executed Planned Executed Executed /budgete Budget Budget in Budget in d in past Code Budget Item for 2021 2020 2019 3 years 32 04 01 Organizing exams 13,800 13,098 13,418 13,439 State education, graduate grants and youth 32 04 02 promotion 117,200 106,641 125,542 116,461 32 04 03 Promoting higher education 200 168 210 193 32 04 04 Promoting education abroad 7,000 4,716 7,312 6,343 32 04 05 Promoting higher education institutions 15,225 22,404 133 12,587 32 05 01 Promoting scientific grants and scientific research 28,065 30,100 32,662 30,276 32 05 02 Programs of scientific institutions 5,900 5,582 4,624 5,369 Promotion of the Georgian Academy of Agricultural 32 05 03 Sciences 1,110 1,110 1,105 1,108 32 05 04 Promoting scientific research 25,000 19,445 25,340 23,262 32 05 05 Popularization of science 400 600 370 457 32 06 Inclusive education 27,115 21,222 19,088 22,475 Development of infrastructure of general education 32 07 01 institutions 34,010 56,236 76,670 55,639 Page 135 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment Mean executed Planned Executed Executed /budgete Budget Budget in Budget in d in past Code Budget Item for 2021 2020 2019 3 years Development of infrastructure of vocational 32 07 02 education institutions 25,000 21,816 9,538 18,785 Development of the infrastructure of the Ministry and its legal entities and territorial bodies under 32 07 03 public law 1,500 2,240 1,063 1,601 Development of infrastructure of higher education 32 07 04 and scientific institutions 20,000 8,491 8,210 12,234 Development of a public school operation and 32 07 05 maintenance system 2,336 7,402 1,205 3,648 Support for investment in infrastructure and 32 07 06 infrastructure projects 8,000 6,120 10,139 8,086 Support for investments in infrastructure and 32 07 07 infrastructure projects 46,910 26,297 4,642 25,950 32 08 Promoting arts and sports facilities 6,782 5,969 3,824 5,525 32 09 Promoting the development of culture 69,499 68,658 6,368 48,175 Preservation of cultural heritage and improvement 32 10 of the museum system 30,194 24,942 74,509 43,215 Development and promotion of mass and high 32 11 achievement sports 122,400 87,626 26,431 78,819 Measures for social protection and promotion of 32 12 cultural and sports figures 17,122 16,906 126,810 53,613 Innovation, inclusiveness of and the quality of the 32 13 project - Georgia I2Q (IBRD) 26,500 1,075 33,847 20,474 Page 136 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment Mean executed Planned Executed Executed /budgete Budget Budget in Budget in d in past Code Budget Item for 2021 2020 2019 3 years 32 14 Vocational Education I (KfW) 8,500 260 77,385 28,715 32 15 Applied Research Grant Program (IBRD) 3,400 6,021 - 3,140 32 16 State support measures in the field of youth - - 2,006 669 32 17 Millennium challenge - the second project - - - - 7,290,63 7,182,32 5,681,32 6,718,09 TOTAL 5 7 0 8 Page 137 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment Annex 2: HCP Program Expenditure Framework Mean 3 Budget Code 2019 2020 2021 years 32 02 01 Expenses 710,365 794,997 894,000 799,787 Subsidies 400 4,909 Other expenses 709,965 790,088 27 03 01 Expenses 828,674 961,712 800,000 863,462 Page 138 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment Mean 3 Budget Code 2019 2020 2021 years Goods and services 3,788 3,386 Social security 824,822 957,227 Other expenses 64 1,098 Increase in non-financial assets 2,566 27 02 02 Expenses 741,286 794,687 816,000 783,991 Goods and services 2,440 2,055 Social security 738,795 792,603 Other expenses 51 30 27 01 04 Expenses 23,715 14,457 13,480 17,217 Salaries 17,798 10,591 8,520 Goods and services 5,524 3,514 Grants 44 51 Page 139 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment Mean 3 Budget Code 2019 2020 2021 years Social security 295 250 Other expenses 55 51 Increase in non-financial assets 317 196 250 32 01 02 (breakup estimated from 32 01) Expenses 5,396 5,812 6,320 5,842 Salaries 2,352 2,533 2,755 Goods and services 2,883 3,105 3,377 Grants 83 89 97 Social security 69 74 81 32 02 13 01 Expenses 1,329 1,130 16,000 6,153 Goods and services 529 695 Subsidies 463 165 Grants 142 Page 140 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment Mean 3 Budget Code 2019 2020 2021 years Other expenses 196 270 32 02 02 Expenses 12,336 9,708 10,257 10,767 Salaries 429 508 Goods and services 11,771 9,119 Grants 54 1 Social security 21 16 Other expenses 61 64 Increase in non-financial assets 65 2 32 04 01 Expenses 13,040 12,859 13,547 13,149 Salaries 3,169 3,182 3,200 Goods and services 8,959 9,288 Grants 857 310 Page 141 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment Mean 3 Budget Code 2019 2020 2021 years Social security 41 65 Other expenses 14 14 Increase in non-financial assets 378 239 253 32 07 01 76,670 56,236 34,010 55,639 Expenses 7,136 6,009 9,010 Goods and services 3,778 3,600 Subsidies 120 Grants 550 1,505 Social security 1 12 Other expenses 2,687 892 Increase in non-financial assets 69,533 50,227 25,000 27 03 03 Expenses 197,650 496,706 650,840 448,399 Page 142 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment Mean 3 Budget Code 2019 2020 2021 years Goods and services 40,399 213,215 Subsidies 2,121 Grants 17,029 Social security 156,022 242,936 Other expenses 1,229 21,405 Increase in non-financial assets 127 6,645 27 04 6,248 50,725 30,000 28,991 Expenses 714 2,547 500 Goods and services 374 1,716 Other expenses 340 830 Increase in non-financial assets 5,534 48,178 29,500 27 01 08 Expenses 401 301 1,213 638 Page 143 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment Mean 3 Budget Code 2019 2020 2021 years Salaries 290 259 Goods and services 104 39 Social security 2 3 Other expenses 6 0 Increase in non-financial assets 1 63 27 05 03 [27 05 only available from master] Expenses 3,008 1,360 8,860 4,409 Goods and services 1,177 1,338 Social security 26 20 Other expenses 1,804 3 Increase in non-financial assets 16 726 32 01 04 Expenses (break-up not available) 9,758 10,327 18,130 12,739 32 02 11 Page 144 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment Mean 3 Budget Code 2019 2020 2021 years Expenses 27,613 12,232 13,550 17,798 Goods and services 6,982 1,271 Subsidies 2 Grants 20,628 10,961 Other expenses 0 32 13 I2Q Project (no break-up available) 26,500 1,075 33,847 20,474 32 02 15 E-learning (no allocation) 32 02 16 ICT in schools no allocation 27 03 02 Expenses 105,186 99,454 96,847 100,496 Salaries 185 159 Goods and services 42,817 54,184 Page 145 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment Mean 3 Budget Code 2019 2020 2021 years Subsidies 9,181 2,530 Social security 40,660 38,088 Other expenses 12,344 4,495 Increase in non-financial assets 4,597 1,226 Annex 3: ERCs classified by Schools supported by ERC Page 146 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment ERC City Village Mountain Cumulative Schools Schools Schools Sum 1 Akhalkalaki 0 0 65 65 2 Kazbegi 0 0 7 72 3 Keda 0 0 29 101 4 Khulo 0 0 49 150 5 Lentekhi 0 0 11 161 6 Mestia 0 0 24 185 7 Ninotsminda 0 0 38 223 8 Oni 0 0 9 232 9 Shuakhevi 0 0 42 274 10 Tianeti 0 0 13 287 11 Tsageri 0 0 25 312 12 Tsalka 0 0 31 343 13 Ambrolauri 2 0 20 365 14 Aspindza 0 2 16 383 15 Adigeni 0 3 22 408 16 Dmanisi 2 4 21 435 17 Akhaltsikhe 7 8 23 473 18 Dusheti 2 15 21 511 19 Borjomi 5 5 10 531 20 Sachkhere 3 16 13 563 21 Tetri Tskaro 0 15 9 587 22 Akhmeta 3 13 7 610 Page 147 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment ERC City Village Mountain Cumulative Schools Schools Schools Sum 23 Chiatura 7 25 11 653 24 Sagarejo 4 17 4 678 25 Signagi 3 14 3 698 26 Gori 12 44 9 763 27 Kaspi 4 21 4 792 28 Tkibuli 6 15 3 816 29 Khelvachauri 0 31 4 851 30 Mtskheta 2 23 3 879 31 Chokhatauri 0 28 3 910 32 Tskaltubo 4 31 4 949 33 Kareli 2 30 3 984 34 Khashuri 9 20 3 1016 35 Gurjaani 4 19 2 1041 36 Kharagauli 0 23 2 1066 37 Marneuli 8 61 5 1140 38 Bolnisi 4 26 2 1172 40 Khoni 4 20 1 1218 41 Telavi 9 17 1 1245 42 Gardabani 4 30 1 1280 43 Abasha 4 19 0 1303 44 Abkhazia 9 6 0 1318 45 Batumi 27 0 0 1345 Page 148 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment ERC City Village Mountain Cumulative Schools Schools Schools Sum 46 Chkhorotsku 2 17 0 1364 47 Dedoplistskaro 2 13 0 1379 48 Didube-Chugureti 14 0 0 1393 49 Djveli Tbilisi 36 0 0 1429 50 Gldani-Nadzaladevi 47 0 0 1476 51 Isani-Samgori 47 0 0 1523 52 Khobi 2 23 0 1548 53 Kobuleti 5 41 0 1594 54 Kutaisi 39 0 0 1633 55 Kvareli 2 19 0 1654 56 Lagodekhi 2 25 0 1681 57 Lanchkhuti 3 21 0 1705 58 Martvili 2 35 0 1742 59 Ozurgeti 5 37 0 1784 60 Poti 11 0 0 1795 61 Rustavi 23 0 0 1818 62 Samtredia 12 23 0 1853 63 Senaki 4 16 0 1873 64 Terjola 2 22 0 1897 65 Tsalenjikha 11 19 0 1927 66 Vake-Saburtalo 29 0 0 1956 67 Vani 2 28 0 1986 Page 149 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment ERC City Village Mountain Cumulative Schools Schools Schools Sum 68 Zestafoni 7 26 0 2019 69 Zugdidi 12 39 0 2070 Page 150 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment Annex 4: School Digital Action Plan: SELFIE Georgian Pilot Data from Sample of Schools Page 151 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment More than 3 School Location (C=City & V=Village) C C C C C C C C C C C V C C C C C C C V C C V V V C C V (out of 28) Digital strategy 2 3 3 3 3 3 3 2 3 3 3 0 3 3 3 3 3 3 3 3 4 3 4 4 4 4 4 4 7 Strategy development with teachers 2 3 3 3 3 3 3 3 3 3 3 0 2 3 4 3 3 3 4 3 4 3 4 4 4 4 5 4 9 Leadership New ways of teaching 3 3 3 3 3 3 3 3 3 3 3 0 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 16 Time to explore digital teaching 3 3 3 3 3 3 3 3 3 3 3 0 3 4 4 4 4 4 4 3 4 4 0 0 0 4 5 4 11 Involving companies in strategy 3 3 3 3 3 3 3 3 3 3 3 0 3 4 4 4 0 3 4 3 4 3 0 0 0 0 5 4 7 Progress review 2 3 3 3 3 3 3 3 3 3 3 0 4 3 3 3 3 3 3 5 3 4 4 4 4 4 5 4 9 Cooperation and Discussion on the use of technology 2 3 3 3 3 3 3 3 3 3 3 4 4 3 3 3 4 3 3 4 3 4 4 4 5 4 4 4 11 Professional Partnerships 2 3 3 3 3 3 3 3 3 3 3 0 4 2 3 3 3 3 3 4 3 4 4 4 4 4 4 5 9 Networking Communication 3 3 3 3 3 3 3 3 3 3 4 0 4 3 3 3 0 4 3 5 4 4 0 0 0 0 4 5 8 Infrastructure 3 3 3 3 3 3 3 3 4 3 3 0 3 3 4 3 4 4 4 4 4 3 5 4 4 4 5 5 13 Infrastructure and Digital devices for teaching 3 3 3 3 3 3 3 3 3 3 3 0 3 3 4 3 3 4 4 4 4 3 5 4 5 4 5 4 11 equipment Internet access 3 3 3 3 3 3 3 4 3 3 3 4 4 3 4 3 3 4 4 4 4 4 5 4 5 4 5 4 15 Continuing CPD needs in institution 3 3 3 4 4 4 4 4 4 4 4 0 4 4 4 4 4 4 4 4 4 4 4 5 4 4 4 4 24 professional CPD needs in cooperation 3 3 3 3 3 4 4 4 4 4 4 0 4 4 4 4 4 3 3 4 3 4 4 5 5 4 4 4 19 development Participation in CPD 3 3 3 3 3 3 3 4 3 3 3 0 3 3 3 3 4 3 3 3 3 4 4 4 5 4 4 4 9 Online education resources 3 3 3 3 3 3 4 3 3 3 3 0 4 3 3 3 4 4 4 4 4 4 4 4 4 4 4 13 Teaching and Creating the digital resources 3 3 3 3 3 3 4 3 3 3 3 0 3 3 3 3 4 3 3 3 3 4 4 4 4 4 3 4 8 learning – support Using virtual learning environment 2 3 3 3 3 3 3 3 3 3 3 4 3 3 3 3 4 3 3 3 3 3 4 4 4 4 4 4 8 Communicating with the school community 2 3 3 3 3 3 3 3 3 3 3 0 2 3 3 3 4 3 3 3 3 3 4 4 4 4 4 4 7 Tailoring to students' needs in institution 3 3 3 3 3 3 3 3 3 4 3 4 3 3 3 3 3 4 4 4 4 4 4 0 4 4 4 4 12 Tailoring to students' needs in company 3 3 3 3 3 3 3 3 4 4 4 0 3 3 3 3 0 4 4 4 4 4 0 4 0 0 4 4 11 Pedagogy Foresting creativity 3 3 3 3 3 3 3 3 4 4 4 4 3 4 3 4 3 4 4 4 4 4 4 0 4 4 4 4 16 classroom Engaging students 3 3 3 3 3 3 3 3 3 4 4 4 3 4 3 4 3 4 4 3 4 4 4 4 4 4 4 4 15 implementation student collaboration 0 3 3 3 3 3 3 4 4 4 4 4 3 4 3 4 3 4 4 4 4 4 4 4 3 4 5 4 17 Cross-curricular projects 0 3 3 3 3 3 3 3 4 4 4 4 3 4 3 4 3 4 4 4 4 4 4 4 4 4 4 4 17 More than 3 (out of 25) 0 0 0 1 1 2 4 5 7 8 8 8 9 9 9 9 11 14 15 16 17 18 20 20 20 22 24 25 Page 152 of 153 The World Bank Georgia Human Capital Program (P175455) Technical Assessment C: City School; V: Village School Page 153 of 153