The World Bank E-Health Project (P131290) REPORT NO.: RES32779 RESTRUCTURING PAPER ON A PROPOSED PROJECT RESTRUCTURING OF E-HEALTH PROJECT APPROVED ON JUNE 6, 2014 TO MONGOLIA HEALTH, NUTRITION & POPULATION EAST ASIA AND PACIFIC Regional Vice President: Victoria Kwakwa Country Director: Bert Hofman Senior Global Practice Director: Timothy Grant Evans Practice Manager/Manager: Enis Baris Task Team Leader: Dinesh M. Nair, Anna L Wielogorska, Pagma Genden The World Bank E-Health Project (P131290) ABBREVIATIONS AND ACRONYMS EA Enterprise Architecture HIEP Health Information Exchange Platform IT Information Technology MOH Ministry of Health NHIC National Health Information Center PACS Picture Archiving and Communication System PDO Project Development Objective PIU Project Implementation Unit TORs Terms of Reference The World Bank E-Health Project (P131290) BASIC DATA Product Information Project ID Financing Instrument P131290 Investment Project Financing Original EA Category Current EA Category Not Required (C) Not Required (C) Approval Date Current Closing Date 06-Jun-2014 30-Sep-2020 Organizations Borrower Responsible Agency Mongolia Ministry of Health Project Development Objective (PDO) Original PDO To improve integration and utilization of health information and e-health solutions for better health service delivery in selected pilot sites. OPS_TABLE_PDO_CURRENTPDO Summary Status of Financing Net Ln/Cr/Tf Approval Signing Effectiveness Closing Commitment Disbursed Undisbursed IDA-54890 06-Jun-2014 18-Jun-2015 17-Aug-2015 30-Sep-2020 19.50 1.16 16.79 Policy Waiver(s) Does this restructuring trigger the need for any policy waiver(s)? No The World Bank E-Health Project (P131290) I. PROJECT STATUS AND RATIONALE FOR RESTRUCTURING 1. The Mongolia E-Health Project was approved by the World Bank Board for US$19.50 million on June 6, 2014 and became effective, 14.6 months after approval, on June 29, 2015, the result of lengthy in-country procedures for signing and ratification of the Financing Agreement. This is not a Project-specific problem, but an issue affecting the country’s overall portfolio at that time. 2. A level 2 restructuring was carried out in June 2015 to factor in the delayed Government approvals and effectiveness, and the closing date for the Project was extended by 12 months (to 30 Sept 2020). The dated covenant regarding the establishment of the Project Implementation Unit (PIU) was also pushed back by a year and the end-target dates in the Results Framework were also extended by a year. No modifications were made to the original Project Development Objective (PDO). A. Status of Achievement of PDOs and Key Performance Indicators 3. The Project indictors are focused on the final Project deliverables since there has been no data available to monitor progress of either the PDO level or the intermediate level results indicators. The initial design of the Project presumed that the first year would be spent on “Foundational Activities” (Component 1), however this design underestimated the impact of changes in leadership at the Ministry. Therefore, the first few years of implementation have been devoted almost entirely to preparatory activities such as establishing the PIU, and drafting terms of reference (TORs) for contracts. The delays and setbacks related to these preparatory activities have meant that none of the foundational activities have been implemented yet. As a result, there were no tangible results by the Mid-term Review. B. Implementation Status 4. Project Ratings. Project Implementation to date has been unsatisfactory. After the delayed effectiveness, the Project saw several changes in leadership; whilst these did not result in any substantive changes of direction for the Project, it was necessary each time to go back to the beginning, which effectively resulted in “wheelspin” for Project implementation. Project Implementation Status Reports reflect the relative lack of progress; with the progress towards achievement of the PDO rated Moderately Unsatisfactory for 17.8 months, and implementation progress has been rated Unsatisfactory/Moderately Unsatisfactory for 4 out of 6 reports. 5. Implementation progress has improved significantly since October 2017. The new leadership at the Ministry of Health (MOH), especially the Minister of Health and Project Director, have shown exceptional levels of commitment and drive, which has contributed significantly to improved implementation progress. The PIU has been fully staffed since August 2017, and the staff have shown a high level of competence and focus on expediting Project progress. The stability in Government and in the PIU is a key factor behind the improved Project implementation. 6. While the pace of implementation has improved, only a limited number of activities have been completed. As a result, there has been no change in the Result Framework indicators. One of the tasks underpinning the foundational activities, the combined Feasibility Study of primary health care information technology (IT) equipment needs and digital equipment for Picture Archiving and Communication System (PACS), has been completed. The results show where the major gaps and limitations are both in IT, for the primary care level, and in digital equipment. The study tour in March 2018 to Estonia greatly enhanced the Government’s knowledge and awareness and provided a vision of the tangible benefits from a successful Project. Aside from these two activities, the only real progress to date has been the hiring of consultants, the drafting of TORs and Bidding Documents, and the advanced stages of procurement for a firm to carry out the Health The World Bank E-Health Project (P131290) Enterprise Architecture (EA) activity. However, due to the failure to attract qualified service providers, the EA contract will now be re-advertised, but with a reduced scope to cover activities that will only be relevant to the Project. 7. The slow pace of implementation since Project effectiveness reflect the lack of stability and continuity in management, capacity constraints, and a relative lack of familiarity with World Bank processes. These have been documented extensively during implementation support reviews in 2016 and 2017. The frequent changes in senior leadership at the MOH associated with elections and parliamentary re-shuffles meant that there was a considerable degree of instability in the management of the Project from 2015-2017. In some cases, these changes in Government were accompanied by changes in PIU staff. This lack of stability and continuity in the overall management of the Project was compounded by capacity constraints at the MOH. The Project is not only technically complex, but the procurement has also become cumbersome due to unnecessary fragmentation. As this is the first World Bank Project implemented by MOH, staff have faced a steep learning curve; for instance, the TORs and Bidding Documents had to be revised multiple times before they were of sufficient quality for a no objection to be issued. 8. Procurement, Financial Management, Project Management, and Audit Status. Procurement performance has been rated moderately unsatisfactory, while financial management has continued to perform at a moderately satisfactorily level. While counterpart funding has been satisfactory, it has not been at the levels originally envisaged and committed by the Government of Mongolia during negotiations. The last audit report was “unqualified” and was submitted timely. Overall Project Management has been rated as moderately unsatisfactory. 9. Monitoring and Evaluation is rated moderately unsatisfactory since there has been no data available to track the Results Framework indicators. 10. Project Disbursements. It took 12.6 months to make the first disbursement; to date only US$1.16 million has been disbursed for startup activities, and the bulk of the funds (US$16.79 million) remains undisbursed. Fund utilization over the last two years has covered the establishment of the PIU, PIU related operational expenses, international long-term consultants, a national feasibility study, and a study tour to Estonia. Whilst these could be effective ‘foundational activities”, and several TORs and Technical Specifications have been finalized, the Project has yet to show any tangible progress in core activities. Government has maintained its commitment to e-Health with both policy and financing support including establishing an E Health Division, with full time staff, and support for recurrent expenditures such as salaries, and operational and maintenance costs. However, gaps exist, especially in meeting the capital costs for the data storage in the National Data Center, which will be partially met through the Project. C. Rationale for Restructuring 11. Given the foregoing challenges, if the Project is implemented as is, there is a very high risk of the Project Development Objective (PDO) not being achieved. Implementation experience over the past three years has shown clearly that the Project design, activities and implementation processes need to be simplified where possible to improve efficiency and the likelihood of achieving key Project indicators. Leaving the current Project design and sequencing of activities unchanged would mean that the PIU and MOH would continue to grapple with technical capacity constraints, and complex processes. Furthermore, as the Government is unlikely to commit most the originally anticipated counterpart funds, the financing of the Project needs to be restructured to ensure the PDOs are achieved. 12. The purpose of the proposed Level 2 restructuring is to revisit implementation strategies to ensure the core objectives are achieved within the remaining timeframe. The government expressed a strong preference to keep the Project closing date unchanged and not make any substantive changes to the core Components of the Project. To fast-track Project implementation, the mission agreed with the Government on the following strategies for the restructuring of the Project: (a) Given the large number of sub-Components, including foundational activities, there is a need to consolidate and focus on core objectives. Over the remainder of the Project, prioritizing Project activities that underpin achievement The World Bank E-Health Project (P131290) of the PDO, especially establishment of the health information exchange platform (HIEP), will be the primary focus. Project implementation post-Mid-Term Review will aim to focus on collapsing several activities around the design, development and piloting of HIEP. This will include a number of activities currently defined as foundational activities, however these would be focused on only what is necessary to ensure the successful implementation of the HIEP. (b) As the Government has clearly indicated that they will not be able to honor their commitment to provide hardware for HIEP, it is necessary to revisit the financing required and reallocate funds to ensure the appropriate equipment is made available at the National Data Center to host the HIEP and related central archives. (c) In line with the feasibility study findings, which identified a major gap in IT equipment at the primary care level, support will be given to the primary health care system with investments in e-health hardware and software capacities. (d) There needs to be a clear delineation of the roles of the various e-Health institutions within government as follows: the e-Health Division within the MOH is responsible for defining standards, the e-Health policy and overall policy governance; the PIU is responsible for the implementation of the e-Health Project; the Health Information Unit of the Center for Health Development would be responsible for the long-term sustainability and implementation of the e-Health system, and the National Data Center would be the repository for all stored health data. (e) Given the changes in the components and to enable proactive monitoring the Results Framework will be revised 13. In summary, going forward the project will see substantial consolidation while at the same time dropping and replacing elements that are no longer central to delivering the objectives with more relevant actions. Many activities planned under old Component 2 on Clinical Data Collection, Access and Sharing, such as Health Information Exchange Platform, portals and viewer, PACS software, IT maintenance and support, and old Component 3 of Establishing Nation1 Health Information Center (NHIC), such as Health Dashboard and NHIC standards, will be consolidated into a single HIEP contract under new Component 1 titled “Building the E-Health Integration System”. Further procurement of digital PACS converters is being dropped and the two international consultancies (user training and IT training) under old Component 4 on Institutional Capacity Strengthening are being streamlined for gaining efficiencies. Evaluation of testing originally planned under old Component 2 will also be dropped. The hardware for primary healthcare facilities and connectivity to high speed internet highway has been added into new Component 1 to successfully implement HIEP. II. DESCRIPTION OF PROPOSED CHANGES D. PDO – defining the scope of the engagement 14. The PDO at design stage was “to improve integration and utilization of health information and e-health solutions for better health service delivery in selected pilot sites”, and is being maintained as such, since the Mid-term Review clearly showed the commitment of MOH to this objective. Furthermore, the discussions with stakeholders during the Mid-term Review showed that although the Project, as designed, requires some simplification, it is still relevant and very much necessary for the development of better health services throughout Mongolia. However, findings from the implementation experience is that the Project reach is much larger than the “pilots” per se, considering that the benefits of e-health integration will flow to the entire health system, similar to the benefits from capacity building and institutional strengthening, which may directly benefit only a limited number of individuals, but will percolate across the entire health system. With this in mind, the scope of the “pilot” in the PDO is being defined as all geographical and operational activities that the Project finances seek to impact in delivering the goal of better integration and use of data. Notwithstanding the above, the core of the system development of the HIEP will be tested at specific pilot sites which would receive the majority, but not all, of the hardware investments. With this recognition the PDO indicators are being expanded to capture the impact and benefits flowing across all parts of the health system and, possibly, not uniquely to the narrow definition of “pilot sites”. The World Bank E-Health Project (P131290) E. Changes to Project Components 15. The Project currently has five Components: (1) e- health foundational activities; (2) clinical data collection, access and sharing; (3) National Health Information Center; (4) institutional strengthening and capacity building; and (5) Project management. To simplify implementation, the current Components 1 and 2 will be merged into a new Component called Build the E Health Integration System and, similarly, the current Components 3 and 4 will be merged into a new Component called Institutional Strengthening and Capacity Building. The current Component 5 on Project Management will become Component 3 and will largely remain as is, and include support for hiring of a specialist in monitoring and evaluation of results achievement. The following sections detail the changes envisaged in each Component. 16. New Component 1 (Restructured Component 1 and 2): Build the E-Health Integration System. The core deliverable of the Project is HIEP, which forms the basis for data integration across different levels of care and management. The design and implementation of HIEP in the original Project design involves a series of parallel activities under Components 1 and 2 (standardization, enterprise architecture (EA) implementation, HIEP and relevant e-services development, and IT infrastructure development). The Project will be restructured with a workplan that focuses on the practical implementation of HIEP and execution of the remaining Component 1 and 2 (foundational) activities according to the evolution and needs of the steps for development of the HIEP. With a primary goal of making existing systems interoperable, the sub-Components of HIEP will be Electronic Health Record (EHR), Electronic Medical Record (EMR) for primary health care providers, e-referral service (referral and report/test result), and possibly other applicable e-services, an identification and authentication tool which might use the national e-services, network services for primary health care providers connection, statistical reporting based on data accessible through HIEP, and security, privacy, quality and IT management protocols. HIEP will be implemented through an iterative design process, including development, testing and implementation, through the cooperation of MOH, the PIU, and the IT developer who will be selected through a procurement process to be carried out between June and September 2018. In addition, this Component will need to include additional hardware initially to be provided by the Government of Mongolia through the National Data Center (see Appendix 4- Figure 1 which provides an illustration of this consolidated format). MOH suggested Uvs aimag and National Center for Traumatology and Orthopedics as possible pilot sites. The pilot sites selected for Project support are health facilities of Uvs, Khuvsgul aimags and Songinokhairkhan District, Shastin Central Hospital (State 3rd General Hospital) and National Center for Traumatology and Orthopedics. 17. Support for the primary health care system with investments in e-health hardware and software capacities. The feasibility study identified gaps in IT equipment at the primary centers; there were only 198 computers being used in 41 primary level health care centers covered by the study, and 59% of the computers in use at primary health care facilities could be considered as low capacity or out of date. From a total of 198 computers in referral level hospitals, 157 had been in use for more than 3 years, and 79% were past their life expectancy. Given that the primary health system has received minimal e-health investments to date, the revised Component 1 will include support for hardware and software equipment. Initial estimates indicate that at least 2,280 computers would be required to cover all primary health facilities and delivery will be phased to ensure that sites have access to connectivity first. 18. New Component 2 (Restructured Component 3 and 4): Institutional Strengthening and Capacity Building. The restructured Component 2 aims to bring together essential elements of institutional strengthening and capacity building under the original Components 3 and 4. Component 3 had key elements of setting up the National Health Information Center (NHIC) to oversee the overall all aspects of E-Health and play a lead role in designing and implementing a change management strategy. Component 4 sought to invest in human capacity needed for institutionalization of E-Health in Mongolia. As part of the restructured Component 2, strengthening health professional’s basic computer skills and HIEP literacy will be a core objective. Further, for IT specialists, network administration and information security management training will be supported. Advanced training in EA and health information interoperability will enable select professionals to manage the changes in business processes and health information flows in the health sector. Establishing a training center, and setting the building blocks for an e-Health system by ensuring facility connectivity, and the legal framework for The World Bank E-Health Project (P131290) smooth functioning will form essential activities of this Component. The Project will seek to ensure Government commitment in allocating internet service fees in the state budget for future sustainability of the operational HIEP system. Furthermore, with the primary goal of establishing an interoperable system across the country, primary health care facilities will progressively be connected to the high-speed network to enable health information exchange. 19. The team considered converting this Component to a Disbursement Linked Indicator, using a results-based financing model. However, it was felt that this would lead to substantial changes that require increased capacities and substantial relearning which, at the mid-term stage, would be disruptive to the current progress. Consequently, this Component will continue to utilize the standard implementation model; to ensure that identified policy and institutional bottlenecks are removed, an agreed action plan (see Appendix 3) will be monitored as part of the review missions and linked to Project performance ratings (recorded in Implementation Status Reports). The agreed action plan will be included in the Project Implementation Manual within 2 months of the restructuring. 20. New Component 3 (original Component 5): Project Management. This Component will retain the same scope of activities as the original Component 5: financing the PIU and a team of experts to support implementation of the Project. Further, to systematically mainstream citizen engagement and ensure beneficiary feedback into the operation, the PIU will undertake a survey (in July 2019) to further strengthen implementation. Project IDA Cost Financing Component (Original) Component (Revised) (Original) (Revised) (in US$M)1 (in US$M) Component1 e-health foundational activities 2.62 Component 1: Build the E-Health Component 2: Clinical data Collection, Access and 15.98 Integration System 12.36 Sharing Component 3: National Health Information Center 2.22 (NHIC) Component 2: Institutional 2.30 Component 4: Institutional Strengthening and Strengthening and Capacity Building 1.20 Capacity Building Component 5: Project management 1.10 Component 3: Project management 1.22 (1.16) Total 19.50 disbursed2 F. Institutional structures 21. To ensure successful achievement of Project objectives, long term sustainability after Project closes, and strengthening the capacity building of the health sector, the key e-Health institutions responsibilities will be streamlined. The Project Implementation Manual will be updated to reflect and formalize the refined division of responsibilities among the various E-Health institutions for the following areas: 1 includes GOM contribution for component 2 2 Figures in brackets are disbursements; data as of June 14, 2018 The World Bank E-Health Project (P131290) (a) E-Health Division of the MOH will be responsible for defining and enforcing e-Health Policy and overall policy governance, coordinating e-Health initiatives and projects being implemented in the health sector, determining and enforcing nationwide health system infrastructure, including hardware and software, enhancing e-Health legal environment by improving health policies, regulations, standards and legal acts. (b) Health Information Unit of Center for Health Development will be responsible for continuous capacity building of existing and new health professionals and specialists, undertaking health information and health technology training, maintenance and improvement of health applications and hardware, operating the HIEP helpdesk, operating health data dashboard to supply health information to policy and decision makers, maintenance and improvement of HIEP throughout the Project implementation as well as after the Project closes. (c) Based on the agreement made between MOH and the Project, for the provision of hardware, the National Data Center shall securely and safely store and maintain national health data throughout the Project implementation period and after it closes. The Project will continue to work to strengthen the three institutions to build up the e-Health system in the country. (d) The PIU will continue to be responsible for the implementation of the E-Health Project. G. Changes to Procurement methods 22. Procurement framework and methods. The Project restructuring would entail a shift to the use of the Procurement Framework rather than the Guidelines. This would offer the possibility of using more modern methods of procurement and specific procedures that would be more suited to the successful development of the HIEP and offer a greater flexibility to MOH which would help them achieve the agreed objectives. The use of the Procurement Framework shall be applicable to all new procurement processes following completion of the restructuring process. However, so as not to lose the recent increased pace of implementation, and given the relatively long lead time required for this package, the HIEP procurement shall be carried out following the Procurement Regulations for Borrowers, July 2016, updated October 2017. In addition, a Project Procurement Strategy for Development will be developed following the change to the applicable procurement framework 23. Using the new procurement frameworks’ flexibility for the procurement of HIEP. The combination of a number of the initially planned activities into larger, more results-oriented, procurement packages takes into account the core objective of establishing a comprehensive HIEP. Doing this efficiently and effectively will take into account developments that have taken place since the Project was conceived. Specifically, the procurement of HIEP will be carried out under a single stage procurement using initial selection, which is faster than using a full two stage process. In addition, it allows the use of evaluation criteria at the initial selection stage and limits the number of bidders invited to prepare full proposals; this is one example of the flexibility provided by the new framework. During implementation other features of the new procurement framework would be used to enhance implementation effectiveness. H. Changes to the Results Framework 24. While the Results Framework itself remains relevant, a key challenge thus far has been that there has been no data available on the indicators. This is partly because the PDO indicators have been targeted as end of Project outcomes, and activities relating to most of the intermediate indicators have just started up. In addition, MOH has indicated their desire to see the scope of the Project impact the entire health sector, and not just the pilot areas, such as building up the capacity of primary health care with hardware and software solutions. To enable better tracking of the Project, new indicators will be introduced, and some of the PDO indicators will be revised, and the intermediate indicators will reflect the change in the Components’ structure and scope of activities. This inclusion of some intermediate outcome and process indicators will enable a more proactive follow through of Project implementation. 25. Progress at the PDO level will be tracked with two additional indicators reflecting the importance of connecting the primary care system to the e-health system, and the capacity building efforts to implement the e-health system. The The World Bank E-Health Project (P131290) indicator “Percentage of patient episodes for which information, including medical image information is available for secure viewing at pilot facilities)” will be dropped from the PDO level, but will be monitored at the intermediate level by the indicator “% of digital images generated at pilot facilities that are transmitted to central Picture Archiving and Communication System (PACS)”. The two additional indicators being included at the PDO level are: “proportion of soums and family health centers transmitting monthly e-health statistical reports” and “improvements in information technology (IT) and health information management capacity among health professionals”. The latter was an intermediate indicator but, considering the emphasis on capacity and institution building, this is being elevated to PDO level. At the intermediate level five new indicators are being added (number of soums and family health centers with working computers; number of soums connected to the high speed internet; health data information storage capacities improved; E-signatures used as a valid endorsement for health insurance claims; beneficiary feedback incorporated in Project implementation) and two are being dropped (% of agencies which adopt and utilize the EA, and HDS and number of personnel trained) as the former has been largely incorporated into the HIEP work and the latter would now be measured as part of PDO 4. 26. In summary, at the PDO level the Project will be monitored by four indicators: (a) percentage of statistical reports out of total reports produced by Project area facilities that are transmitted electronically through HIEP to the Center for Health Development; (b) percentage of electronic referrals out of total referrals at facilities; (c) number of soum and family health center facilities transmitting monthly e-health statistical reports; and (d) improvements in health information management capacity among health professionals. To reflect the new Component structure at the intermediate level, the Project will be monitored, under Component 1, on building the HIEP and will be tracked by 5 indicators: (a) health data standards developed; (b) percentage of pilot facilities which design, develop and implement successful piloting of HIEP; (c) percentage of digital images that are transmitted at central PACS; (d) number of primary health care facilities having working computers; and (e) storage capacities for health data information improved. Component 2 on Capacity and Institutional Building will be tracked at the intermediate level with 4 indicators: (a) Training Center has been designed, implemented and ready to use; (b) Soums connected to high speed internet; (c) E-signatures used as a valid endorsement for health insurance claims; and (d) beneficiary feedback incorporated into implementation. See Appendix 1 for the changes made and the justification for each indicator. 27. The PIU will continue to monitor and report on implementation of the Results Framework and ensure that this is updated bi-annually. In doing this, the PIU will work closely with the Center for Health Development to draw on the routine health information data. The PIU will also be responsible for tracking implementation on the Action Plan, which aims to ensure that identified policy and institutional bottlenecks are removed. The aim is to ensure that both the Results Framework indicators, and action plan data feed into the semi-annual review missions and Project performance ratings. I. Changes to the Financing distribution 28. Funds will be reallocated based on the revised Component scopes. The overall funding envelop will remain the same. However, there will be reallocation between the Components to factor in the change in Components, including their scope. Component 1, which will focus on the Building the E- Health Integration system, will have an overall budget allocation of US$15.98 million; the major new additional cost involves building the primary care IT hardware and software costs which will have a budget outlay of US$1.79 million. Component 2, which supports institutional strengthening and capacity building, will have a budget allocation of US$2.3 million. Costs for Component 3, Project management, will remain the same at US$1.22 million. MOH has reassessed the counterpart contribution which will be mainly related to the hiring of staff as discussed below, and is estimated to amount to US$4.25 million. J. Compliance 29. The Project includes two legal covenants: the TORs developed for key staff at the National Health Information Center (NHIC), and to recruit and maintain staff in NHIC as in the TOR for the duration of the Project. The TORs have been developed, and hence the first legal covenant has been deemed partially complied with. With regards to the latter The World Bank E-Health Project (P131290) covenant “hiring of staff for NHIC”, Government has set up an E Health Division, with dedicated full-time staff within the MOH, with the mandate for defining standards, the e-Health policy, and overall policy governance. Since the mandate of the Center for Health Development already covers the technical role for data management, therefore the formation of the NHIC as a separate institution is redundant. Consequently, this legal covenant will be dropped. III. SUMMARY OF CHANGES Changed Not Changed Results Framework ✔ Components and Cost ✔ Reallocation between Disbursement Categories ✔ Disbursement Estimates ✔ Legal Covenants ✔ Institutional Arrangements ✔ Procurement ✔ Implementing Agency ✔ DDO Status ✔ Project's Development Objectives ✔ Loan Closing Date(s) ✔ Cancellations Proposed ✔ Disbursements Arrangements ✔ Overall Risk Rating ✔ Safeguard Policies Triggered ✔ EA category ✔ Financial Management ✔ APA Reliance ✔ Implementation Schedule ✔ Other Change(s) ✔ Economic and Financial Analysis ✔ Technical Analysis ✔ The World Bank E-Health Project (P131290) Social Analysis ✔ Environmental Analysis ✔ IV. DETAILED CHANGE(S) OPS_DETAILEDCHANGES_COMPONENTS_TABLE COMPONENTS Current Current Proposed Proposed Cost Action Component Name Component Name Cost (US$M) (US$M) Marked for eHealth Foundational Activities 2.85 0.00 Deletion Clinical Data Collection, Access Marked for 16.06 0.00 and Sharing Deletion National Health Information Marked for 2.44 0.00 Center Deletion Institutional Strengthening and Marked for 1.20 0.00 Capacity Building Deletion Project Management 1.20 Revised Project Management 1.22 Build the E-Health Integration 0.00 New 15.98 System Institutional Strengthening and 0.00 New 2.30 Capacity Building: Component TOTAL 23.75 19.50 OPS_DETAILEDCHANGES_REALLOCATION _TABLE REALLOCATION BETWEEN DISBURSEMENT CATEGORIES Financing % Current Allocation Actuals + Committed Proposed Allocation (Type Total) Current Proposed IDA-54890-001 | Currency: XDR iLap Category Sequence No: 1 Current Expenditure Category: GDs/nonCSs/CSs and IOCs for Pt.5 750,000.00 118,007.88 118,007.88 100.00 100.00 iLap Category Sequence No: 2 Current Expenditure Category: GDs/nonCSs and CSs for Pt.1,2,3 and The World Bank E-Health Project (P131290) 11,950,000.00 0.00 0.00 100.00 100.00 Current Expenditure Category: (3)Goods, non-consulting services, and consulting iLap Category Sequence No: 3 services for Parts 1 and 2 of the Project 0.00 0.00 11,909,570.00 100.00 Current Expenditure Category: (4)Goods, non-consulting services, consulting iLap Category Sequence No: 4 services, and Incremental Operating Costs for Part 3 of the Project 0.00 0.00 672,422.12 100.00 Total 12,700,000.00 118,007.88 12,700,000.00 OPS_DETAILEDCHANGES_DISBURSEMENT_TABLE DISBURSEMENT ESTIMATES Change in Disbursement Estimates Yes Year Current Proposed 2015 0.00 0.00 2016 1,500,000.00 100,000.00 2017 4,000,000.00 500,000.00 2018 5,000,000.00 5,500,000.00 2019 5,500,000.00 10,500,000.00 2020 3,500,000.00 2,900,000.00 OPS_DETAILEDCHANGES_LEGCOV_TABLE LEGAL COVENANTS Loan/Credit/TF Description Status Action Finance Agreement :Terms of Reference for the Key Technical Staff Positions for NHIC | Description :The Borrower shall, no later than six months after the Partially complied IDA-54890 No Change Effective Date, prepare terms of reference acceptable with to the Association forthe key technical staff positions for NHIC. | The due date for this was changed to The World Bank E-Health Project (P131290) December 31, 2016 based on exchange of letters between the Government and the Bank. Finance Agreement :Schedule 2 Section I.A.3.(b) | Description :Finance Agreement :Key Technical Staff Positions | Description :By January 1, 2018 establish and thereafter maintain for the duration of the Project the Partially complied Marked for IDA-54890 technical staff positions referred to in the preceding with Deletion sub-paragraph and fill such positions with staff having qualifications and terms of reference acceptable to the Association. | Due Date :01-Jan-2018 The World Bank E-Health Project (P131290) . Results framework COUNTRY: Mongolia E-Health Project Project Development Objectives(s) To improve integration and utilization of health information and e-health solutions for better health service delivery in selected pilot sites. Project Development Objective Indicators by Objectives/ Outcomes RESULT_FRAME_TBL_PDO Indicator Name DLI Baseline Intermediate Targets End Target 2014 1 2 3 4 2021 Project Development Objective (Action: This Objective is New) (1) Percentage of patient episodes for which information, including medical 0.00 0.00 0.00 20.00 40.00 60.00 image information is available for secure viewing at pil ot facilities (Percentage) Action: This indicator has been Marked for Deletion (2) Percentage of statistical reports out of total reports produced by pilot facilities that are transmitted electronically 0.00 0.00 0.00 50.00 60.00 75.00 through HIEP to the National Centre for Health Development (Percentage) The World Bank E-Health Project (P131290) RESULT_FRAME_TBL_PDO Indicator Name DLI Baseline Intermediate Targets End Target 2014 1 2 3 4 2021 (3) Percentage of electronic referrals out of total referrals 0.00 0.00 0.00 10.00 25.00 50.00 at pilot facilities (Percentage) (4) Number of project area soum and family health center 200.00 450.00 facilities transmitting monthly e-health reports (Number) Action: This indicator has been Revised (5) Improvements in IT literacy and health information 0.00 1,000.00 management capacity among health professionals (Number) Action: This indicator has been Revised PDO Table SPACE Intermediate Results Indicators by Components RESULT_FRAME_TBL_IO Indicator Name DLI Baseline Intermediate Targets End Target 2014 1 2 3 4 2021 Component 1: Build the E-Health Integration System (Action: This Component is New) Component 1: Health Data Standards are developed, no Yes Yes Yes Yes validated and promulgated by MOH (Text) The World Bank E-Health Project (P131290) RESULT_FRAME_TBL_IO Indicator Name DLI Baseline Intermediate Targets End Target 2014 1 2 3 4 2021 Component 1: Percentage of agencies which adopt and utilize the EA and HDS in the 0.00 0.00 0.00 10.00 25.00 50.00 construction of software modules and communication layers. (Percentage) Action: This indicator has been Marked for Deletion Component 1: Percentage of pilot facilities which design, develop, and implement 0.00 0.00 0.00 20.00 40.00 80.00 successful piloting of eHealth Portal, Viewer and HIEP. (Percentage) Component 1: Percentage of digital images generated in pilot facilities which are 0.00 0.00 0.00 20.00 40.00 80.00 transmitted to the central PACS. (Percentage) Component 1: Number of primary healthcare facilities (soum, FHC) having at least 1 110.00 300.00 500.00 upgraded and working computer (Number) Action: This indicator has been Revised The World Bank E-Health Project (P131290) RESULT_FRAME_TBL_IO Indicator Name DLI Baseline Intermediate Targets End Target 2014 1 2 3 4 2021 Component 1:Health data Limited Data storage Health Data Storage information storage capacities Limited storage capacity capacity capacities enhanced improved (Text) Action: This indicator has been Revised Component 2: Institutional Strengthening and Capacity Building (Action: This Component is New) Component 2: Training Center has been designed, No No No Yes Yes Yes implemented and is ready for use. (Text) Component 2: Number of soum and family health centers 0.00 100.00 300.00 connected to high speed internet highway (Number) Action: This indicator has been Revised Health personnel receiving 0.00 0.00 20.00 75.00 125.00 200.00 training (number) (Number) Action: This indicator has been Marked for Deletion Component 4: Improvements in IT literacy and health information management 0.00 0.00 20.00 75.00 125.00 200.00 capacity among health professionals (Number) Action: This indicator has been Marked for Deletion The World Bank E-Health Project (P131290) RESULT_FRAME_TBL_IO Indicator Name DLI Baseline Intermediate Targets End Target 2014 1 2 3 4 2021 Component 2: E signatures used as a valid endorsement No No Yes for health insurance claims (Text) Action: This indicator has been Revised Component 2: Beneficiary consultation outcomes and No In progress Yes feedback integrated in project implementation (Text) Action: This indicator is New IO Table SPACE . The World Bank E-Health Project (P131290) Appendix 1 RESULTS FRAMEWORK End Revised Actual Target PDO PDO/Compo Original Revised Baseline (Dec Justification (Sept nent 2017) 2020) PDO 1 Percentage of Drop End of Project patient episodes for indicator and which information, monitored including medical through image information is intermediate 0 0 60 available for secure indicators viewing at pilot facilities (Percentage, Custom) PDO 2 PDO Percentage of Continue End of Project statistical reports indicator out of total reports produced by pilot sites that are transmitted 0 0 75 electronically through HIEP to the Center for Health Development (Percentage, Custom) PDO 3 PDO Percentage of Continue End of Project electronic referrals indicator out of total referrals 0 0 50 at pilot facilities (Percentage, Custom) PDO 3 (New) PDO Number of Last mile linkage Project area with the e-health soum and system family health center 2003 0 450 facilities transmitting monthly e- health reports 3 Source: Center for Health Development health statistics department The World Bank E-Health Project (P131290) End Revised Actual Target PDO PDO/Compo Original Revised Baseline (Dec Justification (Sept nent 2017) 2020) PDO 4 (New) PDO Improvements Previous in IT literacy Intermediate and health Indicator: information Highlights the management importance of capacity 0 0 1,000 capacity building among health for utilization and professionals integration. Target (Number, increased to Custom) reflect increase in scope Intermediate Indicators Component 1 1 Health Data Continue no Integral to HIEP Standards are (3 developed, validated standard No yes and promulgated by s drafted MOH (Text, Custom) by Sept. 2018 Percentage of Drop The impact of this agencies which indicator may fall adopt and utilize the outside the Project EA and HDS in the implementation construction of 0 0 50 period. software modules and communication layers. (Percentage, Custom) Component 2 1 Percentage of pilot Continue Building blocks to facilities which HIEP. Indicator design, develop, and simplified to implement 0 0 80 reflect bringing successful piloting of activities under HIEP. (Percentage, HIEP. Custom) 1 Percentage of digital Continue Building blocks to images generated in HIEP pilot facilities which are transmitted to 0 0 80 the central PACS. (Percentage, Custom) The World Bank E-Health Project (P131290) 1 (New) Number of Last mile linkage of primary health the e-Health care facilities system (soum, FHC) 110 110 500 having at least 1 upgraded and working computer 1 (New) Health data No no yes Institutional information capacity to store, storage manage and capacities secure data improved Component 3 2 Training Center has Continue No no yes Critical to capacity been designed, building implemented and is ready for use. Component 4 2 Improvements in IT Move to PDO 0 0 1,000 Critical to capacity literacy and health level and building information dropped at management Intermediate capacity among level e health professionals (Number, Custom) 2 Number of online Drop 0 0 2 Linked to new PDO training courses indicator 4 (Number, Custom) Health personnel Drop 0 75 200 Included as a receiving training outcome PDO (number) ( Number) indicator 2 (New) Number of 0 0 70 Institutional pilot soum readiness for inter and family connectedness health centers connected to high speed internet highway 2 (New) E signatures No no yes Institutional used as a valid readiness for an e- endorsement health system for health insurance claims 2 (New) Beneficiary No No Yes Beneficiary consultation feedback outcomes and integrated in feedback Project integrated in implementation Project The World Bank E-Health Project (P131290) implementatio n The World Bank E-Health Project (P131290) Appendix 2 : Revised Costing Compone Activity name Old activities Year 2017* Year 2018 Year 2019 Year 2020 Total nt name amount Compone 1.1 HIEP and 1.1.1. HIEP consolidated package 0 1,020,600 4,422,600 1,360,800 6,804,000 nt 1. relevant e- 1.1.2a. HIEP IC (E-Health IC additional work) 0 28,000 0 0 28,000 Building services 1.1.2b. HIEP IC (EA IC additional work) 0 22,000 0 0 22,000 the E- development 1.1.3. HIEP LC 0 6,000 24,000 12,000 42,000 Health 1.1.4. HIEP hardware (HIEP, portal, health data) 0 700,700 700,700 0 1,401,400 Integratio 1.1.5.a. PACS hardware 0 0 844,000 875,784 1,719,784 n System 1.1.5.b. PACS software license 0 0 0 500,000 500,000 1.1.6. Dashboard hardware 0 0 25,500 0 25,500 1.1.7. HL7 subscription, standards 0 18,500 6,500 0 25,000 1.1.8. Helpdesk service 0 0 0 120,000 120,000 1.1.9 Piloting and testing 0 0 0 600,000 600,000 1.1.10 Piloting and testing IC 0 0 0 105,000 105,000 1.1.11 Piloting and testing LC 0 0 0 12,000 12,000 1.1.12 E-Health applications software 0 0 0 106,000 106,000 1.1.13 Feasibility study 23,715.87 0 0 0 23,716 1.1.14 HIEP technical support to MOH 0 4,000 24,000 12,000 40,000 1.2 1.2.1. E-Health IC 80,425.24 109,072 133,156 74,252 396,905 Standardization 1.2.2. HD LC 0 12,000 24,000 12,000 48,000 1.2.3. HD standards LC 1,472.76 24,000 24,000 12,000 61,473 1.2.4. HD exchange standards LC 0 12,000 24,000 12,000 48,000 1.2.5. E-Health legal LC 0 6,000 6,000 8,000 20,000 1.2.6.1. HD&IT standards ICB 0 21,840 196,560 0 218,400 1.2.6.2. Printing of HD&IT standards 0 0 0 30,000 30,000 1.2.7.1. HDD ICB 0 0 220,080 94,320 314,400 1.2.7.2. Printing of HDD 0 0 0 50,000 50,000 1.2.8. Consultative meeting 0 5,000 13,000 18,000 36,000 1.2.9 Standards enforcement IC 0 0 0 105,000 105,000 2.2.6. Standards enforcement hardware and 0 0 60,000 0 60,000 software 1.3 IT 1.3.1. Health statistics LC 0 0 24,000 12,000 36,000 infrastructure 1.3.2 Health statistics IC 0 0 0 52,500 52,500 development 1.3.3 Primary healthcare hardware 0 736,514 1,058,986 0 1,795,500 1.4 Enterprise 1.4.1. EA IC 4,095.62 120,000 120,000 82,274 326,370 architecture 1.4.2. Development of EA and IF 0 75,140 676,260 0 751,400 implementation 1.4.3. EA licenses 0 42,000 10,000 10,000 62,000 109,709.49 2,963,366 8,637,342 4,275,930 15,986,347 Compone 2.1 Building the 2.1.1.a. Basic and advanced user trainings for 0 20,000 125,000 325,000 470,000 nt 2. health health professionals Institutio information 2.1.1.b. Printing of user training materials (basic, 0 7,200 7,200 0 14,400 nal management advanced) Strengthe capacity of 2.1.2. Local consultancy on basic user training 0 8,000 24,000 0 32,000 ning and health 2.1.3. Local consultancy on advanced user training 0 4,000 24,000 12,000 40,000 Capacity professionals 2.1.4.1 Network security associate training for IT 0 0 33,000 0 33,000 Building specialists 2.1.4.2 Per diem, travel costs of 2.1.5.1 participants 0 0 12,007 0 12,007 2.1.5. Network security professional training for IT 0 0 0 30,000 30,000 specialists 2.1.6. Network management and information 0 0 128,000 0 128,000 security management training (external) 2.1.7.1. Application development and DBMS 0 0 5,000 0 5,000 training 2.1.7.2. Per diem, travel costs of 2.1.8.1 0 0 8,367 0 8,367 participants 2.1.8.1. Security training 0 7,000 18,000 0 25,000 The World Bank E-Health Project (P131290) 2.1.8.2. Per diem, travel costs of 2.1.9.1 0 8,367 0 0 8,367 participants 2.1.9. HL/FHIR for Center of Health Development – 0 0 12,000 0 12,000 (5+5) x $1,200= $12,000 2.1.10. TOGAF trainining for 20 professionals 0 0 102,000 0 102,000 2.1.12 Advanced user training for health 0 0 0 30,000 30,000 professionals of aimag and district general hospitals, specialized hospitals 2.2.4. Capacity building of beneficiaries (external) 0 80,000 50,000 50,000 180,000 2.2.5. Capacity building of beneficiaries and 0 90,000 90,000 20,000 200,000 communication for development (local) 0 224,566.6 638,573.33 467,000 1,330,140 7 2.2 Setting the 2.2.1. Connectivity to high speed highway 0 0 100,000 0 100,000 building blocks 2.2.2. Internet connectivity costs of project areas 0 21,900 317,600 298,360 637,860 for a E- Health system by ensuring facility connectivity 0 21,900 417,600 298,360 737,860 2.3 Establishing 2.3.1. IT and user training LC (training manager with 0 6,000 24,000 12,000 42,000 the training IT background) center 2.3.2. Training center equipment and furniture, 0 140,000 0 0 140,000 refurbishment 2.3.3. Development of two online distance courses 0 0 50,000 0 50,000 and e-Learning platform 0 146,000 74,000 12,000 232,000 00 392,467 1,130,173 777,360 2,300,000 Compone 3.1 Project 3.1.1. Project management 280,473.56 298,020 332,641 302,518 1,213,653 nt 3. management Project managem ent 280,473.56 298,020 332,641 302,518 1,213,653 GRAND TOTAL 390,183.05 3,653,853 10,100,156 5,355,808 19,500,000 The World Bank E-Health Project (P131290) Appendix 3 Agreed Action Plan Serial Actions Target 2018 Target 2019 Target 2020 No 1 Improvements in IT 1.1 Course work 1.2 Course work 1.4 Number of staff literacy and health developed for training of developed for user trained in each and show information trainers of user training training on HIEP, EMR, post assessment score management on computer literacy and EHR, clinical portal and improvement of over capacity among health information viewer literacy 50% health professionals management 1.3 Number of staff trained in each course showing post assessment score improvement of 50% 2 Training Center has 2.1 Training Center 2.2 Training Center fully 2.3 Training Center been designed, TORs developed, agreed functional and develops conducts 4 onsite implemented and is and issued by MOH 2 Online programs courses, ready for use. eLearning Platform available 3 Building Block for 3.1. 100 Soum and 3.2 100 additional Soum 3.3 100 additional Soum E Health family health centers and family health centers and family health centers established connected to high speed connected to high speed connected to high speed internet highway internet highway internet highway 4 3.4 MOH legal 3.5 National Data Center notification of use of e Stores all health data signatures for health insurance claims Appendix 4 Figure 1 Training center Hardware & EA Software (1.1.2), HelpDesk Software (3.2.6) Furniture (3.2.11, 3.2.12) Develop Enterprise Architecture (EA) (Business, data, application and technology architecture) (1.1.4)