The World Bank Implementation Status & Results Report Research and Innovation in Science and Technology Project (P121842) Research and Innovation in Science and Technology Project (P121842) EAST ASIA AND PACIFIC | Indonesia | Education Global Practice | IBRD/IDA | Investment Project Financing | FY 2013 | Seq No: 15 | ARCHIVED on 22-Dec-2020 | ISR45095 | Implementing Agencies: Republic of Indonesia, Ministry of Research Technology / BRIN Key Dates Key Project Dates Bank Approval Date: 29-Mar-2013 Effectiveness Date: 12-Jun-2013 Planned Mid Term Review Date: 13-Jun-2016 Actual Mid-Term Review Date: 09-Feb-2016 Original Closing Date: 31-Dec-2020 Revised Closing Date: 31-Dec-2021 pdoTable Project Development Objectives Project Development Objective (from Project Appraisal Document) To create an enabling policy environment for research and development in science and technology, to improve thepublicresearchanddevelopment institutes’ performance, and to improve science, technology and innovation human resource capacity. Has the Project Development Objective been changed since Board Approval of the Project Objective? Yes Board Approved Revised Project Development Objective (If project is formally restructured) To support the government in: (1) improving STI human resource capacity; and (2) strengthening technology transfer, institutional functioning, and data management of public research agencies (LPNKs). Components Table Name Strengthening the Innovation System and Technology Transfer at Public Research Agencies:(Cost $2.50 M) Strengthening Public Research Funding:(Cost $2.50 M) Developing Science and Technology Human Resources Capacity:(Cost $68.00 M) Project Management:(Cost $7.00 M) Overall Ratings Name Previous Rating Current Rating Progress towards achievement of PDO Satisfactory Satisfactory Overall Implementation Progress (IP) Satisfactory Satisfactory Overall Risk Rating Moderate Low Implementation Status and Key Decisions Despite constraints caused by the COVID-19 pandemic, overall implementation of the RISET Project towards achievement of the Project Development Objective (PDO) has progressed during this reporting period and the results have positively impacted the achievement of the PDO. 12/22/2020 Page 1 of 8 The World Bank Implementation Status & Results Report Research and Innovation in Science and Technology Project (P121842) The project has been extended for one year, to a new closing date of December 31, 2021. A partial loan cancellation of US$5.23 million was also processed. The project has achieved PDO indicator 1: Lembaga Pemerintah Non Kementerian/ Public Research Institute (LPNK) research agency reforms are institutionalized.' In 2020, reforms of five Science and Technology Parks (STPs) in three LPNKs has been completed by developing innovation clusters to strengthen STP excellence. A TTO Management Guideline, technology valuation, incentive system guideline, and standard operating procedure have also been finalized. Fifteen TTO staff have undertaken training to be certified as Registered Technology Transfer Professionals (RTTPs). Progress has been made in achieving PDO indicator 2: ‘Improved regulation on the output-based cost standard for the Science Technology Innovation research funding system issued on an annual basis.’ The project team is finalizing an academic paper on the urgency of establishing the Indonesian Research Funding Forum (IRFF), which will be submitted to Kemristek/BRIN for legalization. The Monevrisbang (Research and Development Monitoring and Evaluation system) is now integrated into the SIIN (Sistem Inovasi Ilmu Pengetahun dan Teknologi Nasional) or National Science Technology Innovation (STI) System of the Ministry. A Presidential Regulation has been proposed by the Ministry to legalize the SIIN as a formal National STI system. Progress has been made in achieving PDO 3: ‘Increased percentage of Masters’ and Ph.D. degree holders in LPNKs and MoRTHE supported by the RISET Project.’ Of the 457 scholars supported by the project, the number who have graduated has increased from 307 (67 percent) in June 2020 to 347 (76%) by beginning of December 2020. Thirty scholars are expected complete their studies in December 2020, 62 (14 percent) will complete their studies using their own funds, and 18 have been unable to complete their studies. For the Non-Degree Program (NDP), the number of participants increased from 1,982 in June 2020 to 2,073 in December 2020. The NDP in 2020 could not be fully implemented due to the COVID- 19 pandemic, which means that 100 participants from 2020 will participate in the NDPs in 2021. Risks Systematic Operations Risk-rating Tool Risk Category Rating at Approval Previous Rating Current Rating Political and Governance -- Moderate Low Macroeconomic -- Low Low Sector Strategies and Policies -- Moderate Low Technical Design of Project or Program -- Low Low Institutional Capacity for Implementation and -- Moderate Low Sustainability Fiduciary -- Substantial Moderate Environment and Social -- Low Low Stakeholders -- Low Low Other -- Moderate Moderate Overall -- Moderate Low Results PDO Indicators by Objectives / Outcomes 12/22/2020 Page 2 of 8 The World Bank Implementation Status & Results Report Research and Innovation in Science and Technology Project (P121842) To support the government in strengthening technology transfer IN00940298 ►1. LPNK research agency reforms are institutionalized (Text, Custom) Baseline Actual (Previous) Actual (Current) End Target Reforms are Reforms are Value NA In progress institutionalized institutionalized Date 30-Sep-2013 19-Jun-2020 22-Dec-2020 31-Dec-2020 This PDO has been achieved through institutionalization of reforms in five STPs in three LPNKs and four TTOs in four LPNs. This has been done through issuance of a number of regulations and standard operating procedures – both for management and operationalization of STPs and TTOs. TTO human resource development has been done through sending 15 TTO staff to attend RTTP training and thereby Comments: acquire certification. Three commercialization models for STI inventions have been shared with the broader STP network through three policy papers on: STP-based commercialization model sectoral-based commercialization model; and industrial cluster-based innovation model. To support the government in strengthening institutional functioning and data management of LPNKs IN00940299 ►2. Improved regulation on the output-based cost standard for the STI research funding system issued on an annual basis (Text, Custom) Baseline Actual (Previous) Actual (Current) End Target Refined regulation is Refined regulation is issued based on issued based on Value NA In progress annual review covering annual review covering research priorities research priorities Date 13-May-2011 19-Jun-2020 22-Dec-2020 31-Dec-2019 The team has carried out advocacy to strengthen implementation of an output-based cost standards regulation, with the issuance of MoF Regulation No. PER-7/PB/2019, which responded to implementation challenges arising from previous MoF regulation No. PER-15/PB/2017. Monevrisbang (Research and Comments: Development Monitoring and Evaluation System has been advanced and is ready to be adopted by MoRT as an executive decision-making tool and used by LPDP. To support the government in improving STI human resources capacity IN00940297 ►3. Increased % of Master's and Ph.D. degree holders in LPNK and MoRTHE supported by the RISET Project (Percentage, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 67.01 76.00 100.00 Date 30-Sep-2013 19-Jun-2020 22-Dec-2020 31-Dec-2020 As of December 2020, of 457 scholars, 347 (76 per cent) have graduated, 30 (6 percent) will complete their studies by the end of 2020 and will return to Indonesia in early 2021. A further 62 scholars (14 percent) have not completed their study and are no longer funded by the project as the validity period to receive project scholarship has expired. These unfunded scholars will continue their studies using their Comments: own funds, with 28 studying remotely from Indonesia and 34 remaining based in their countries of study. Eighteen students have been unable to complete their study (4 percent), including 1 student who passed away and one who was dismissed by his institution. IN00940300 12/22/2020 Page 3 of 8 The World Bank Implementation Status & Results Report Research and Innovation in Science and Technology Project (P121842) ►4. Master’s degree holders who participate in applied science research programs (Percentage, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 95.00 92.00 90.00 Date 30-Sep-2013 19-Jun-2020 22-Dec-2020 31-Dec-2020 This PDO has been achieved, with 217 (92 per cent) Master’s degree holders now involved in research and occupying posts aligned with their field of studies within their respective LPNKs. The remaining 10 Comments: Master’s Degree graduates are still waiting to be allocated positions aligned with their expertise, while no responses has yet been received from the other 9 graduates. IN00940306 ►5. Ph.D. degree holders who manage or coordinate applied science research programs (Percentage, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 91.00 92.00 90.00 Date 30-Sep-2013 19-Jun-2020 22-Dec-2020 31-Dec-2020 This PDO has also been achieved. with 103 (92 per cent) of 111 PhD graduates leading research in their fields and occupying posts aligned with their field of studies within their respective LPNKs. The remaining Comments: 6 PhD Degree holders are still waiting to be assigned suitable research tasks and allocated positions aligned with their field of studies. Responses have not yet been received from 2 other graduates Overall Comments PDO The project has completed a level-2 project restructuring, which has included extension of the project to December 31, 2021 and partial loan cancellation of an amount of US$ 5.23 million. During the mission, it was agreed that project activities during the period of extension will focus on implementation of Component 3b (Non-Degree Program) and 4 (Project Management). Activities on Component 1 (Strengthening the Innovation System and Technology Transfer at Public Research Agencies), 2 (Strengthening Public Research Funding), and 3a (Fellowship/Degree Program) will be completed by December 31, 2020. A Project Implementation Completion Report (ICR) for Components 1, 2, and 3a is currently being prepared to document implementation of the components from 2013 to 2020. Intermediate Results Indicators by Components Strengthening the Innovation System and Technology Transfer at Public Research Agencies IN00940308 ►1. National innovation strategy is developed (Component 1) (Text, Custom) Baseline Actual (Previous) Actual (Current) End Target Policy papers have Strategy action plans informed the Law No. are submitted for Strategy action plans is 11/2019 on National consideration to the Value NA included in the RPJMN System of Science, new national research 2020-2024. Technology and and innovation master Innovation. plan Date 30-Sep-2013 19-Jun-2020 22-Dec-2020 31-Dec-2019 IN00940312 12/22/2020 Page 4 of 8 The World Bank Implementation Status & Results Report Research and Innovation in Science and Technology Project (P121842) ►2. National Information System for STI is established and integrated into ministry’s database system (Component 1) (Text, Custom) Baseline Actual (Previous) Actual (Current) End Target Data collection at STPs to inform Kemristek/BRIN database. Monevrisbang STI data is collected tool is also being used to through Monevrisbang monitor research and system, which is being STI data is collected Value Not started development managed used and integrated and reported by the Kemristek/BRIN into Ministry’s data and LPNKs, as well as by base system. MoF for LPDP research grant proposal selection. Date 30-Sep-2013 19-Jun-2020 22-Dec-2020 31-Dec-2020 IN00940316 ►3. Existing regulations in the STI legal framework are reviewed and modifications proposed (Component 1) (Text, Custom) Baseline Actual (Previous) Actual (Current) End Target Policy papers on the STI legal framework and innovation funding Final modifications are system, as well as proposed and TTO regulation is being Value NA commercialization, submitted for drafted have been published consideration to and have informed a relevant ministries. number of regulations and implementation. Date 30-Sep-2013 19-Jun-2020 22-Dec-2020 31-Dec-2020 Strengthening Public Research Funding IN00940309 ►1. Increased number of research programs in LPNKs and MoRTHE focused on higher ranks of the Technology Readiness Level (TRL) (Component 2) (Text, Custom) Baseline Actual (Previous) Actual (Current) End Target It is noted that the TRL It has been confirmed measurement exercise that, for evaluation in this project is purposes, the numerical confined to self- indicator of TRL has to assessment. The be combined with the formal TRL system Increased number of other relevant indicators. Value NA measurement is research programs of Further desk-analysis is currently managed by higher TRL required to contribute to the R&D Directorate upcoming regulation on (Risbang Directorate) TRL System as under MoRTHE mandated by Law Regulation No. No:11/2019 42/2016 Date 30-Sep-2013 19-Jun-2020 22-Dec-2020 31-Dec-2019 IN00940313 ►2. Strengthened R&D monitoring & evaluation capacity (Component 2) (Number, Custom) Baseline Actual (Previous) Actual (Current) End Target 12/22/2020 Page 5 of 8 The World Bank Implementation Status & Results Report Research and Innovation in Science and Technology Project (P121842) Value 0.00 12.00 12.00 12.00 Date 30-Sep-2013 19-Jun-2020 22-Dec-2020 31-Dec-2020 Twelve independent evaluations of multi-year research activities -- selected from the health, medicine, food/Agriculture, energy, and advanced materials research fields -- were conducted and lessons were Comments: documented. The target was achieved in 2019. Developing Science and Technology Human Resources Capacity IN00940307 ►1. Number of LPNK, MoRTHE, Research Agency staff who have studied overseas as supported by RISET-Project (Component 3) (Number, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 457.00 457.00 422.00 Date 30-Sep-2013 19-Jun-2020 22-Dec-2020 31-Dec-2020 IN00940311 ►2. Number of participants in RISET-Project’s Non-Degree professional courses and training (Component 3) (Number, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 1,982.00 2,073.00 2,407.00 Date 30-Sep-2013 19-Jun-2020 22-Dec-2020 31-Dec-2021 IN00940315 ►Direct project beneficiaries (Number, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 2,439.00 2,530.00 2,829.00 Date 30-Sep-2013 19-Jun-2020 22-Dec-2020 31-Dec-2021 IN00940317 Female beneficiaries (Percentage, Custom Supplement) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 33.00 32.00 28.00 Project Management IN00940310 ►1. Project implementation and monitoring reports (include financial reports) are completed (Component 4) (Yes/No, Custom) Baseline Actual (Previous) Actual (Current) End Target Value No Yes Yes Yes Date 30-Sep-2013 19-Jun-2020 22-Dec-2020 31-Dec-2020 Comments: While IFR 2019 has been submitted in a timely manner, the PMO still needs to submit the annual report 12/22/2020 Page 6 of 8 The World Bank Implementation Status & Results Report Research and Innovation in Science and Technology Project (P121842) IN00940314 ►2. Computerized project monitoring database is established, updated, and key information is placed on MoRTHE project website (Component 4) (Yes/No, Custom) Baseline Actual (Previous) Actual (Current) End Target Value No Yes Yes Yes Date 30-Sep-2013 19-Jun-2020 22-Dec-2020 31-Dec-2020 The website link is: http://risetpro.ristekdikti.go.id and content is being continuously updated. Comments: Performance-Based Conditions Data on Financial Performance Disbursements (by loan) Project Loan/Credit/TF Status Currency Original Revised Cancelled Disbursed Undisbursed % Disbursed P121842 IBRD-82450 Effective USD 95.00 74.76 20.24 67.58 7.18 90% Key Dates (by loan) Project Loan/Credit/TF Status Approval Date Signing Date Effectiveness Date Orig. Closing Date Rev. Closing Date P121842 IBRD-82450 Effective 29-Mar-2013 12-Apr-2013 12-Jun-2013 31-Dec-2020 31-Dec-2021 Cumulative Disbursements 12/22/2020 Page 7 of 8 The World Bank Implementation Status & Results Report Research and Innovation in Science and Technology Project (P121842) PBC Disbursement Achievement Disbursed amount in Disbursement % PBC ID PBC Type Description Coc PBC Amount Status Coc for PBC Restructuring History Level 1 Approved on 30-Jan-2017 ,Level 2 Approved on 24-Nov-2020 Related Project(s) There are no related projects. 12/22/2020 Page 8 of 8