Understanding the Importance of Institutional Development INSTITUTIONAL DEVELOPMENT ROADMAP HANDBOOK Effective Design, Implementation and Evaluation of Project-led Institutional Development in the Transport Sector RADIA BENAMGHAR JUNE 2018 © 2018 The World Bank 1818 H Street NW Washington DC 20433 Telephone: 202-473-1000 Internet: www.worldbank.org This work is a product of the staff of The World Bank with external contributions. The findings, interpretations, and conclusions expressed in this work do not necessarily reflect the views of The World Bank, its Board of Executive Directors, or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of The World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. Rights and Permissions The material in this work is subject to copyright. Because the World Bank encourages dissemination of its knowledge, this work may be reproduced, in whole or in part, for noncommercial purposes as long as full attribution to this work is given. Any queries on rights and licenses, including subsidiary rights, should be addressed to World Bank Publications, The World Bank Group, 1818 H Street NW, Washington, DC 20433, USA; fax: 202-522-2625; e-mail: pubrights@worldbank.org. Understanding the Importance of Institutional Development INSTITUTIONAL DEVELOPMENT ROADMAP HANDBOOK Effective Design, Implementation and Evaluation of Project-led Institutional Development in the Transport Sector RADIA BENAMGHAR JUNE 2018 INSTITUTIONAL DEVELOPMENT ROADMAP HANDBOOK Handbook Contents Foreword. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .iii Acknowledgment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . v Abbreviations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii Executive Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix Introducing the Institutional Development Roadmap. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1 Applying the Institutional Development Roadmap. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 Step 1: Contextual Assessment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 Step 2: Institutional Capacity Assessment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 Step 3: Design of Institutional Development Activities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 Step 4: Implementation, Monitoring and Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 The Institutional Development Roadmap Toolkit. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 Step 1 (a): Contextual Assessment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 Step 1 (b): Assessing Context against Standards. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 Step 1 (c): Maturity levels of various areas of Contextual Assessment. . . . . . . . . . . . . . . . . . . . . . . . . . . 27 Step 2 (a): Institutional Capability Assessment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 Step 2 (b): Maturity level of Capability Assessment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 Step 3: Design of Institutional Development Activities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 Step 4: Implementation, Monitoring and Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 Annexes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 Annex A: Preparation for the Road Agency (RA) institutional and functional review . . . . . . . . . . . . . . . . 40 Annex B: Contextual Assessment for the Identification of Pre-Project Activity . . . . . . . . . . . . . . . . . . . . 44 Annex C: Capability Assessment for ID Design. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52 Annex D: Pilot application of the Institutional Development Roadmap . . . . . . . . . . . . . . . . . . . . . . . . . 59   i ii INSTITUTIONAL DEVELOPMENT ROADMAP HANDBOOK Foreword The World Bank (WB) is a vital source of financial and techni- cal assistance across the transport sector in India. However, the estimated scale of the required investment in this sector The transport section (US$ 500 billion or 3.6% of GDP over tens years) means that in India will require an WB staff urgently need support in planning, implementing and investment of nearly monitoring sustainable institutional development. $500 billion (3.6 Following a 2014 SARDE review of the application of good percent of GDP) over practice, the World Bank commissioned a cross-case com- the next 10 years parison within the India transport portfolio to develop an evidence-based understanding of the success of institutional development interventions. The findings from this research have been incorporated into a Handbook (‘The Institutional Development Roadmap (IDR) Handbook’) which serves to assist users support the project lifecycle, drawing upon lessons learned and international best practice for capability development. THE IDR is an evidence-based framework that promotes a structured and systematic approach to the development of Progress has been a ‘plan for change’ and robust development related activi- made with respect ties. The framework supports users with project-led devel- to the application of opment design, implementation, monitoring and evaluation good practice but this that enables the results to be captured and the impact to be appeared to be erratic assessed. - SARDE, 2014 The Institutional Development Roadmap (IDR) represents the first step in a more robust approach to preparing for project- led institutional development and ensuring maximum value from World Bank investment and support in the transport sector in India. Karla Gonzalez Carvajal Practice Manager South Asia Region Transport Global Practice The World Bank FOREWORD iii iv INSTITUTIONAL DEVELOPMENT ROADMAP HANDBOOK Acknowledgment Acknowledgment is due to the many contributors to the Transport Sector Reform through building and strengthening institutional capacity to sustain transport investments over time. This publication has been produced thanks to the contribution of the Australian Government Department of Foreign Affairs and Trade (DFAT) through the Partnership for South Asia. The study was led by Radia Benamghar—Transport Specialist Guidance and support were provided by Karla Gonzalez Carvajal—Transport Practice Manager The core team was composed of Nancy Madter, Nicola Bates, and Graham Dalton, and was led by Professor Denise Bower—Executive Director of the Major Projects Association (MPA), Professor of Engineering Project Management at the University of Leeds, an Independent Member of the Mott MacDonald Shareholder Committee and a Director of Routemap Ltd. The second core team was composed of Sadia Afolabi and was led by Veronique Kabongo, Lead Operations Officer from Governance GP. The extended team included Stephanie Matter, Zahra Hassanali, Paul Kaiser, and Mesfin Wodajo Jijo. Administrative support was provided by Tema Alawari Kio-Michael and Genevieve Maria Dutta. The team thanks the following peer reviewers for their insightful and constructive comments and suggestions: Mohamed Khatouri, Christopher Bennett, Mustapha Benmaamar, Vivek Srivastava, and Ana Bellver Vazquez-Dodero. ACKNOWLEDGMENT v vi INSTITUTIONAL DEVELOPMENT ROADMAP HANDBOOK Abbreviations and Defined Terms CDRF Capacity Development Results Framework GP Governance Practice Government National or State Government HR Human Resources IA Implementing Agency (e.g. Public Works Department) ID Institutional Development IDR Institutional Development Roadmap IDRt Institutional Development Roadmap Toolkit M&E Monitoring and Evaluation PAD Project Appraisal Document PDO Project Development Objective SAR South Asia Region TTL Task Team Leader WB World Bank WBG World Bank Group WBI World Bank Institute ABBREVIATIONS AND DEFINED TERMS vii viii INSTITUTIONAL DEVELOPMENT ROADMAP HANDBOOK Executive Summary Successful ID requires: • Empowerment of the IA • Full life-cycle, project led ID The World Bank (WB) is a vital source of financial and technical assistance across the transport sector in India. While finan- Step 1: cial resources and technical expertise are vital, they are not Sector dynamics sufficient to promote sustainable development (UNDP 2008; and strategic risk World Bank 2011). This requires strong and durable institutions in client countries. In India’s transport sector, the extent to which sustained Institutional Development has been achieved varies signifi- cantly between projects. An understanding of the factors that Step 2: led to successful outcomes and sustained institutional develop- Implementing ment is key to ensuring long-term added value of WB and client Agency capability country investments. A WB cross-case comparison of projects in the Indian transport portfolio found that Institutional strengthening plays a key role in determining success. It was even more effective when ID was included in the Project Development Objective (PDO). Step 3: Based on lessons learned in India, and the best practices used Design of ID by the WB and other development partners, SAR Transport enabling activities developed the Institutional Development Roadmap to support and M&E the successful integration of ID in WB transport infrastructure projects. This approach: Enables WB Task Team Leaders (TTLs) and IAs to take a more systematic, nuanced, and outcome-based approach to project- led ID; Step 4: Implementation, Ensures that ID is appropriately included in the PDO; and Monitoring, Evaluation Builds on recognized and established best practice by enabling ID to be designed to match capability to the context in which it will be utilized. EXECUTIVE SUMMARY ix The Institutional Development Roadmap is a structured and rigorous four-step approach to planning. It is flexible and adaptable to different contexts and needs. It supports strategic, data-driven decision- making for project-led ID throughout the project life cycle. Clear instructions and a set of four interrelated and mutu- ally reinforcing tools are provided in this Handbook for TTLs and IAs to use in the design, implementation, monitoring, and evaluation of project-led ID. Taken together, these tools facilitate the development of a PDO that integrates ID with infrastructure investments by identifying opportunities for sustainable results beyond the life of the project. Since there are time and cost implications for implementing IDR tools, TTL’s and IAs will need to jointly plan accordingly at the outset. Templates are provided so that completion of the documenta- tion is straight forward. An illustrative example is also provided. BOX 1: OVERVIEW OF ROLES AND THE IDR Who Should Use the IDR Who, Why, and When to Use the IDR Task Team Leader (TTL) Who: TTLs are trained by ID experts to familiarize themselves with the IDR approach and how it works in all phases of the project life cycle. Why: To guide TTLs to diagnose capacity gaps and the needs of IAs for the effective design, implementation, monitoring, and evaluation of sustainable ID. When: The Institutional Development Roadmap (IDR) should be initiated during the early stages of project identification and continue throughout the project life cycle. This facilitates the identification of sustainable ID interventions at the outset and supports dialogue with key stakeholders to support ID throughout implementation. ID Expert Who: TTL, along with assistance of ID experts, should take the lead in the design of the ID interventions, including the results framework. Why: To support the needs analysis for the effective design, monitoring, and evaluation of sustainable ID. When: The analysis should be initiated at early stages from project identification and continue throughout the project life cycle. IA Project Director (or Who: The IA should designate a Project Director, who is empowered to work closely equivalent) responsible for with the TTL and ID expert as required. developing, implementing, Why: The Project Director is trained by the ID expert and TTL to oversee the process and owning ID outcomes from the client’s side. The Project Director is responsible for managing and supervising the use of IDR tools; facilitating dialogue around the findings; serving as a liaison between the TTL, ID expert, and key stakeholders; helping to determine the project’s scale and scope; regularly monitoring project activities; and periodically sharing updates with the TTL, ID expert, and key stakeholders as required. When: The Project Director should be identified at early stages of project identification. x INSTITUTIONAL DEVELOPMENT ROADMAP HANDBOOK Introducing the IDR D elivering services is far more complex and less ●● Empowerment of Implementing Agencies (IAs); obvious than building or upgrading road, airport, and or port infrastructure. It requires a foundation of ●● Full life-cycle, project-led ID that targets common institutional capacities that span policy, strategy, weaknesses in capacity. governance, and management. The ability to recognize areas for reform, build the capacity to implement this This cross-case analysis revealed ID-related elements reform, and translate this into successful projects that that contribute to successful projects. These elements deliver sustainable outcomes can be challenging. The are both contextual (tackling institutional challenges, Institutional Development Roadmap presents a set of Government commitment, IA commitment) and technical tools that enables users to develop projects that suc- (Human Resources, Environmental & Social Safeguards, cessfully and sustainably integrate infrastructure and Monitoring & Evaluation, Procurement [includ- institutional development (ID). These tools provide a ing contract management], Maintenance, Financial systematic assessment of the reform environment and Management, and Road Safety [for road projects]). the achievement of clear and measurable development outcomes. Human Resources and Planning & Design were fol- lowed by Procurement (including contract management) ID refers to activities undertaken to strengthen the and Social & Environmental Safeguards. Project-led capacity of public sector organizations to sustainably Maintenance interventions were also highlighted in deliver results through enhanced planning, delivery, and two projects. Based on these findings, SAR Transport asset maintenance capabilities. Institutions are broadly developed the IDRt to help TTLs (in consultation with understood as “systems of established and embedded Governance Practice (GP) colleagues) and IAs take social rules that structure social interactions” (Hodgson a more systematic, nuanced, and outcome-based 2006), while organizations are administrative and func- approach to developing, implementing, monitoring, and tional structures that formally bring together people for evaluating project-led ID. a specific purpose or goal. Institutions influence, either formally or informally, how public-sector delivery organi- The IDR is a robust approach to institutional assess- zations operate and carry out their bounded missions. ment and planning for project-led ID. Built upon learning from WB and international best practices such as the A holistic approach to ID is required if WB investments Project Initiation Routemap (IPA 2016), the IDR is not are sustainable in the long term. In India’s transport intended to replace existing WB practices used through- sector, the extent to which sustained improvements in out the project life cycle but to capitalize on project- institutional capacity have been achieved varies signifi- led ID to achieve sustainable development outcomes. cantly between projects. A WB cross-case comparison For example, the World Bank Institute (WBI) Capacity of projects in the Indian transport portfolio identified two Development and Results Framework (CDRF) captures primary critical success factors for sustained institutional WB’s best practices and provides helpful practical guid- capacity improvements: ance on capacity building. INTRODUCING THE IDR 1 TOOLKIT (IDRT) resources are distributed and contested in specific areas of possible project support (DFID 2009; OECD- The IDR presents a set of four interrelated tools that DAC 2005; World Bank 2007; World Bank 2008). It can help TTLs and IAs develop projects that coherently was designed to identify opportunities and challenges integrate infrastructure and ID, specifically focusing on in these areas by addressing how formal and informal building sustainable capacity to achieve both. The IDR interests, incentives, and institutions support or prevent contributes toward ensuring high impact, sustainable the reforms required for sustainable ID. The tools also development by providing a systematic way to assess recognize the importance of engaging key stakeholders the environment for change and the capacity of IAs to (including citizens impacted by potential WB projects) achieve clear and measurable development outcomes. in designing and implementing sustainable interven- tions that are achievable within the project life cycle and The IDRT is informed by the building blocks of politi- sustainable after the project closes. cal economy analysis, focusing on how power and FIGURE 1: THE IDR APPROACH Figure 1 illustrates the IDRT approach, and it comprises as well as the importance and influence of all relevant the following four (4) steps that support decision making stakeholders. The second focuses on the capability throughout the project life cycle. assessment (capacity of stakeholders; the development of project-led ID such as formal frameworks, monitor- ●● Step 1: Contextual Assessment ing and evaluation frameworks, staffing, policies etc.). ●● Step 2: Institutional Capability Assessment The third step focuses on the implementation design of activities based on the IDRT scoring tool card (Table 6) ●● Step 3: Design of ID Activities which provides an accurate capacity level of the institu- ●● Step 4: Implementation, Monitoring, and tion based on assessments of the previous two steps. Evaluation The fourth and last step focuses on the implementation, monitoring, and evaluation of project activities to enable This approach is informed by best practices in inter- and capture impact. The IDR tools described and pre- national development, lessons learned from past sented in this document correspond to these four steps. SAR transport projects, and the Bank’s operational requirements. The first step listed above focuses on The IDRT approach complements the WB project life an in-depth understanding of the operational context, cycle and provides rapid appraisals to identify areas for 2 INSTITUTIONAL DEVELOPMENT ROADMAP HANDBOOK discussion and action to support the development of the and provide the basis for prioritizing short-, medium-, or Project Appraisal Document (PAD), project implementa- long-term ID intervention options to determine which tion, monitoring, and evaluation (Figure 2). The IDRT ones have the greatest potential for enhancing the can also be used to explore more than one project idea capacity needed for sustainable ID. FIGURE 2: THE IDR APPROACH AND THE WB PROJECT LIFE CYCLE Before using the tools described in this Handbook, the The IDRT should be viewed as a “whole of business” TTL and IA should consider time and cost implications approach to developing, implementing, monitoring, and for WB and Government staff (including travel and train- evaluating WB investments. The WB and governments ing requirements). The TTL and IA should specifically (represented by the TTL and IA) can work together to explore possible joint financing arrangements between identify short- and medium-term, and longer term ID the WB, IA, state and/or national government(s), and requirements for sustaining costly infrastructure invest- interested development partners. Including develop- ments. The tools presented in this handbook and the ment partners in this conversation is especially impor- pro formas provided in the annexes specifically focus on tant for monitoring ID outcomes after projects close, the transport sector, but they can be adapted for other since sustainable ID often takes longer than the typical teral sectors and for use by other bilateral and multila-​ project WB life cycle. development partners. INTRODUCING THE IDR 3 4 INSTITUTIONAL DEVELOPMENT ROADMAP HANDBOOK Applying the IDR T he Institutional Development Roadmap and sup- One of the main objectives of the tool is to support a porting toolkit (IDRT) was designed for the WB structured dialogue between the Bank team and the and partner governments to ensure that proj- client; it supports the project Task Team in adopting a ects effectively integrate infrastructure and ID to more methodical approach to project design by match- achieve sustainable development outcomes. The four ing capability to the context. steps described in Figure 1 provide users with a detailed roadmap to effectively apply IDR tools to the full project During the project identification phase, the IDRt is a life cycle. They are the culmination of two years of desk great resource that allows the project team to collect review, field research, and extensive consultations, but and analyze various information that not only supports SAR Transport recognizes that for the IDRT to be effec- the design of tailored interventions to the client but is tive, it needs to be continuously updated and improved also a means to determine the right financing instrument based on evidence and experience. Taken together for the project based on the types of the needed inter- with the phase one and phase two studies that focused ventions. As such, the first step before conducting the on “Understanding the Importance of Institutional IDRt assessments, the “identification” step, allows the Development to Project Outcomes,” the Handbook Bank team to develop a clear project plan, which is then serves as a starting point for promoting and supporting informed by the assessment outcomes to create project institutional strengthening and reform needed for sus- activities that are adapted to client context. tainable asset management and maintenance. Step 1: Contextual Assessment The Contextual Assessment step in the IDRt approach the factors that commonly contribute to complexity, is focused on undertaking an assessment of the which can reduce the likelihood of achieving outcomes, ‘readiness’ of the delivery environment to support the exposing projects to risk and uncertainty, and reducing development objectives and any required institutional value for money. For example, in countries which have development. To do this, the Contextual Assessment federal arrangements (such as India), there are multiple is used in two ways: (i) to understand the contextual levels of “context” to be considered: local community, factors and why increased pre-project activity may be state, national, regional, and global. This tool is flexible required to manage strategic risks to achieving develop- enough to accommodate any or all levels as required by ment goals; and (ii) to evaluate the contextual situation on-the-ground realities. against typical standards to inform the scoring process. Overall, the tool will allow assessment of the broader Three key factors have been identified to enable an political, social, and economic environment, and sector in-depth understanding of the client context, namely: dynamics, to identify challenges to the achievement (i) the sociopolitical environment, (ii) policy instruments, of development objectives. This tool is framed around and (iii) organizational arrangements. Assessing the APPLYING THE IDR 5 sociopolitical environment permits to better understand HOW DO YOU USE THE CONTEXTUAL the likelihood of change and the confidence in the ASSESSMENT TOOL? implementing agency's capacity to adapt to that change while considering various stakeholders’ influence. The The Contextual Assessment Tool (understanding the policy instrument allows to better understand the opera- context and assessing the context) should be used tional performance of the implementing agency and its by TTLs and the IA during project identification and capacity to align with other projects, as well as financial preparation. The section on understanding the context viability; while the organizational arrangements allow presents six factors divided into 16 sub-factors, namely: to grasp the amount and magnitude of transformational (i) stability of the environment; (ii) stakeholders and change required to support new ways of working in a Influencers; (iii) strategic importance of the project; (iv) given context (e.g., new delivery models). alignment of objectives; (v) financial viability; and (vi) execution complexity, that should be considered in turn, The tool then allows for a two-scoring method: the self- along with guiding questions. The section on assessing assessment by the IA and the assessment conducted by the context provides three levels of contextual situations an expert, i.e., the project TTL or team member based against typical standards that should be used in both on three levels of contextual situations against typical the self-assessment and the expert assessment. (See standards. As such, the two-level assessment permits Phase One Study Annex 2, WBI 2012; and UNDP 2008 more realistic analyses of the context under review, and for examples of questions that are relevant for this tool). provides an increased involvement in the process of assessing strengths and areas in need of improvement, Figure 3 showcases the “understanding the context” especially for the IA. The IA’ s strong involvement in the subsection to illustrate the context review process process will enable its ownership in the IDRt approach. through the application of factors and sub-factors for a better understanding/assessment of the context. This is accomplished by using a visual of the contextual assess- WHAT IS THE PURPOSE OF THE CONTEXTUAL ment showing an example of a factor used to assess the ASSESSMENT TOOL? context and identify challenges. The Contextual Assessment Tool provides an in-depth Figure 4 illustrates the “the assessing the context” understanding and analysis of the context to ensure that subsection which helps the assessor to determine, the environment is apt for sustainable ID. This includes based on the proposed three levels, at what level a an assessment of: (i) the sociopolitical environment, given context is situated based on typical standards which focuses on the stability of the environment and and norms. the role and importance of stakeholders and influenc- ers; (ii) policy instruments, with a particular focus on The related questions included in the “assessing the unpacking the strategic importance of the project, the context” section, through its three levels, will enable alignment of project objectives, and its financial viability; TTLs and IAs to assess the level of specific context and (iii) organizational arrangements such as execution against typical standards and norms. For example, if a complexity. All factors have a significant impact on the TTL and IA are jointly developing a transport project successful delivery of the project. For example, using in Nepal, the following additional questions related to this tool enables the TTL and IA to systematically assess implementation continuity and completion risks could stakeholder commitment needed to support successful be added to the assessing the context section: (1) project-led ID, and determine the level of this commit- What impact has the protracted, post-conflict politi- ment as well. cal transition had on formal and informal institutional arrangements in Nepal’s transport sector?; and (2) How The assessment questions can be used by TTLs and the has this transition impacted the capacity of key govern- IA to assess the need for and plan preparation activi- ment entities to build a road, bridge, or airport and sus- ties to manage existing situations, or retrospectively to tain this asset after the project ends? Answers to these identify the root causes of outcomes that have not been types of questions reveal country-specific political, achieved in the past. economic, and/or social opportunities and challenges 6 INSTITUTIONAL DEVELOPMENT ROADMAP HANDBOOK of potential project interventions and related ID activi- relevant to a possible project intervention to determine ties. The guiding questions in this section can also be if a related ID activity is realistic and sustainable. used to stimulate discussion and/or identify information FIGURE 3: UNDERSTANDING THE CONTEXT Understanding the Context Review the contextual factors and why increased Asses the context by answering the contextual considerations related to each of the factors. pre-project activity may be required to manage Where challenges to the achivement of the development goal raised and identify the potential strategic risk to the development goal. cause of the challenge. Factors Guiding Questions Socio-Political Environments Leadership Commitment • Are relevant leaders commited to the reform/development goal? • Is there evidence of bureaucaratic support for the development goal (required legislations, expressed public support)? Stability of the Environment • Is commitment reflected in investment priorities? The environment stability has an impact on the • Is support protected against political change? Implementing Agency's capacity to deliver and will influence how activities and risks will be managed. Conductive political, • Have there been any recent political, economic, or social issues The liklihood of change and confidence in the economic, and/or social that have impacted the general governance environment? Implementing Agency´s capacity to adapt to that environment. • Are there any political, economic, or social issues that have change will inform pre-project activity needed to impacted or disrupted planning and service delivery in the mitigate the risks and stabilize the environment. transport sector? • What are the key contextual factors that have impacted meaningful reforms in the sector (including community, state [for federal systems], national, regional, and global factors)? FIGURE 4: ASSESSING THE CONTEXT Assessing the Context Assess the context by evaluating the contextual situation against typical standards and inform scoring Level 1 Level 2 Level 3 There are significant organizational challenges. Some elements of good practice are There is an outcome focus and maximum Practices, processes and systems are outdated being utilized. Systems and processes are value is achieved from the asset and external or immature. appropriate for the current requirement but relationships may not be sustainable ❑❑ Ownership of the project is weak and ❑❑ Adequate legal frameworks are in place and ❑❑ Legal frameworks and investment plans legislation has not been enacted or effectively aligned with investments policies. are clearly articulated in a regional/national approved through appropriate channels. development plan and a public monitoring mechnism is in place. ❑❑ There is no investment plan or it is disconnected to the project objectives ❑❑ There is evidence of high turnover in project ❑❑ There is public disclosure and monitoring of ❑❑ There is public disclosure and monitoring of leadership linked to new governemental public investments. public investments. appointments. APPLYING THE IDR 7 Once a deeper understanding of the context, along with Note: A project must be proposed before the assessments can be completed. It is not expected that the project will necessarily have identifying challenges and opportunities in the delivery been clearly defined at this stage. If project preparation has com- environment, are gained, the next step is to assess menced, then the likelihood of mitigating risk to the project is reduced and the considerations should be addressed from a risk management the capability of the IA to effectively operate in this point of view that recognizes the reduced scope for change. environment. Step 2: Institutional Capability Assessment This step in the IDR approach assists users in moving is developing a road transport project, this tool could from a broad institutional challenge, as identified in a assess the capability of the road agency to perform project concept note and broadly contextualized in Step required maintenance on the existing road network, 1, to identifying specific capability gaps in the IA through including new roads built with project funds. If there a simplified and user-friendly gap analysis. is a gap between the required and actual capability of this agency, this would provide the rationale for one or The Institutional Capability Assessment is used to articu- more targeted, project-led ID interventions in the road late the institutional challenges in terms of the current maintenance regime to address this gap. capacities and the desired/required capacity to sup- port the PDO and the delivery of the project. This step considers what is needed for project delivery and/or the WHAT IS THE PURPOSE OF THE IA CAPABILITY development objectives and how this differs from cur- ASSESSMENT TOOL? rent capability. The approach assesses the capacity of the identified agency to perform technical and manage- The IA Capability Assessment Tool assists users to iden- rial roles and responsibilities required. The gap between tify capacity gaps in the IA that are specifically related the required and actual capacity of the agency provides to institutional challenges in the transport sector. It pro- the rationale for one or more targeted interventions. vides evidence of the capacity that may require devel- The purpose of this assessment is to consider, for each opment to support project delivery and the achievement capability area, whether there is evidence of a capability of the development objectives. (good practice), and if not, describe the gap in capability such that development areas to close the gap can then The purpose of this assessment tool is to consider, for be defined. each capability area, whether there is evidence of IA capability (good practice) through a self-assessment of To accomplish this, the IA Capability Assessment Tool the IA agency along with an expert assessment. If evi- has two parts: a gap analysis section on core capability dence of good practices do not exist, the tool enables areas and a second section on a good implementation/ describing the gap in capability such that development evidence practice matrix based on a three-level assess- areas to close the gap can then be defined. ment against typical standards and norms. Overall, the IA Capability Assessment tool articulates the institu- The IDRt scoring tool in this step will allow for a two- tional challenges in terms of the current capacities scoring method: the self-assessment by the IA and the and the desired/required capacity to support the PDO assessment conducted by an expert, i.e., the project TTL and the delivery of a potential transport project. The or team member, based on three levels of in practice capabilities identified in the gap analysis section (see capability levels against typical standards. As such, the Table 5) are indicative of the common strengths and two-level assessment permits more realistic analyses weaknesses in IA capacity. This does not necessarily of the institutional capability and increases the IA’ s mean that the highest level of capability is required. involvement in the process of assessing its strengths However, this step considers what is needed for project and areas in need of improvement, therefore, enabling a delivery and/or the development objectives and how stronger ownership of the entire process. this differs from current capacity. For example, if a TTL 8 INSTITUTIONAL DEVELOPMENT ROADMAP HANDBOOK HOW DO YOU USE THE IA CAPABILITY development areas. If the current capacity of the IA is ASSESSMENT TOOL? uncertain, evidence of good practice and further areas to probe can be found in the IA Capability Assessment. The Capability Assessment Tool can be used by TTLs and IAs during project preparation to articulate the The Institutional Capability Tool focuses on five clusters changes needed to support project success and inform of technical areas that need to be considered when robust planning for project-led ID. determining IA capability gaps and identifying possible development areas related to those gaps via the assess- The IA Capability Assessment is used to understand ing practice section. These technical area clusters are the current and desired IA capability level and identify described in Table 1. TABLE 1: REQUIRED TECHNICAL CAPABILITY AREAS FOR IAS TO EFFECTIVELY DELIVER PROJECTS Project governance Focus: Transparent project governance policies and procedures arrangements Problem: Ineffective, formal delegation of authority, institutional accountability, and decision making; weak alignment of project development objectives to sector development goals ID Solution: Introduce development agreements and robust governance frameworks (systems and processes for delegation, decision making, and change management); facilitate clarity, accountability and action (which engenders private sector confidence) Human resources Focus: Transparent HR hiring policies and procedures to support project development goals Problem: Ineffective and opaque HR hiring policies and procedures ID Solution: Support good HR policies and practices to create a positive work ethic and help management attract, develop, and motivate staff to achieve results and deliver qualitative services to the public Environmental and social Focus: Effective project asset maintenance and management safeguards Problem: Weak provision of maintenance services to customers along with underutilization of systems of asset management to achieve outcomes ID Solution: Support transparent and robust approach to asset management to improve service delivery, including output and outcome performance measurement based on customer expectations Asset management Focus: Focus: Effective project asset maintenance and management approach Problem: Weak provision of maintenance services to customers along with underutilization of systems of asset management to achieve outcomes ID Solution: Support transparent and robust approach to asset management to improve service delivery, including output and outcome performance measurement based on customer expectations Business practices and Focus: Development and implementation of systems and processes for effective project implementation implementation procedures Problem: Immature industry/business practices ID Solution: Develop industry/business best practices, including project delivery capabilities, technical knowledge, and market understanding Users should review each statement in the gap analysis the IA and jointly review the Capability Assessment sec- section in Table 1; this might necessitate searching for tion (see Table 4) for their insights into the IA’s capacity additional information to determine if there is existing gaps. Based on this review, users can then determine evidence of “lagging,” “good,” or “exemplar” practices in whether there is a capacity gap (difference between the five required technical areas. This provides a way for current and good practice) and identify the associated the user to systematically assess the capacity—or level development area to ‘close the gap’ if needed. A pro- of readiness—of the IA across and within technical areas. forma is provided in the annex. The user should record answers to the guiding questions in the comment box to the right of the scoring boxes. Note: In most cases Steps 1 and 2 should be conducted together to provide a comprehensive picture of the required project preparation activity and the agreed gaps to be addressed through project-led ID. It may be helpful to interview and obtain documents from other development partners that have worked with APPLYING THE IDR 9 FIGURE 5: EXAMPLE OF A CONTEXTUAL ASSESSMENT SCORING METHOD Contextual Assessment Self-assessment Expert Sociopolitical Stability of the environment Leadership commitment 3 1 Conducive political, economic, and/or social environment 3 1 Delegated authority 3 1 Stakeholders and influencers Stakeholders analysis 3 1 Effective engagement mechanisms 3 1 Policy instruments Strategic importance of the project Reform rationale 3 1 Link to policy delivery 3 1 Accountability of the project delivery 3 1 Alignment of objectives Interdependencies with other Government supported objectives 3 1 Alignment with regional/national strategy 3 1 Performance of the agency 3 1 Financial viability Availability and sources of funding 3 1 Budget Management 3 1 Organizational arrangements Execution complexity Extent of change 3 1 Project management/expertise 3 1 Need for innovative/untested technology 3 1 FIGURE 6: EXAMPLE OF A SCORING GRAPH 10 INSTITUTIONAL DEVELOPMENT ROADMAP HANDBOOK Step 3: Design of ID Activities The step on preparing for designing project activities self-assessment and the expert assessment, along with will focus on developing a sustainable implementation its graph. action plan that will help determine the scope for sus- tainable project-led ID. It is important that the develop- ment areas should be jointly agreed upon by the WB WHAT IS THE PURPOSE OF THE ACTIVITY and the Government. Reaching agreement may require PLANNING LOG? an iterative policy dialogue with Government leaders and/or targeted stakeholder consultations. The cross- Project-led ID requires the buy-in and ownership of comparison of South Asia transport projects identified the IA. The purpose of the Activity Planning Log in strong Government and IA commitment as contributors both Steps 1 and 2 is to capture agreed activities. The to project success. Across 9 of 12 closed projects, and 8 assessments conducted in Steps 1 and 2 have identi- of 12 ongoing projects in India, Government and IA com- fied project preparation activities that, if carried out, will mitment promoted an enabling environment for project enable the management of strategic risks to achieve the success. Strong Government commitment was demon- desired project outcomes. This list of activities, along strated through a clear regulatory and policy framework, with the agreed development areas, should be recorded and from commitment of financial and human resources. in the Activity Planning Log in each step. A proforma is provided in the annex. The Design of ID Activities section is based on out- puts from the assessments conducted in Steps 1 and 2. These assessments, coupled with their IDRt scor- HOW DO YOU USE THE ACTIVITY PLANNING ing levels, will determine how to tailor interventions LOG DESIGN? to address each factor identified gap. The Contextual Assessment in Step 1, categorized in six key areas with The Activity Planning Log should be used by the IA, their sub-factors, will help identify challenges to the stakeholders, and the TTL following completion of the delivery environment. As such, the “Activity Planning Contextual and Capacity Assessments to agree on Log” will allow to propose adapted interventions. These what activities need to be undertaken in preparation should be discussed and agreed upon by the TTL and for the ID. Since it is essential that the IA agrees with IA along with key stakeholders as development areas the proposed activities, the Activity Planning Log is best to appropriate for needs analyses in the final stage of ID developed through sustained engagement between the planning. TTL and the IA, along with other relevant stakeholders. This could take the form of multiple project develop- Recall that in Step 2, the Capability Assessment identi- ment team meetings that culminate in one or more fied and described in Table 1 were grouped in five broad stakeholder workshops. areas: (i) project governance arrangements; (ii) human resources; (iii) environment and social safeguards; (iv) The team should review the completed Contextual asset management approach; and (v) business practices Assessment section (Step 1) and the suggested activities and implementation procedures. The step has a column in its Activity Planning column. They should also review “Activity Planning Log” which allows for the previously the completed Capability Assessment (Step 2) and the identified capability gaps to be translated into discrete development areas recorded in its Activity Planning col- areas for development or wider areas for change. These umn following the assessing good practice section. should be discussed and agreed upon by the TTL and IA as development areas to appropriate for needs analyses Based on the analysis of the data obtained from the in the final stage of ID planning. Agreement should be tools described above, the TTL and IA should agree on captured in the Activities Planning Log. the project preparation activities required, consider- ing which activities should support the management The examples in Figures 5 and 6 showcase the tools’ of strategic risks and help to create an environment scoring methods. There are two levels of scores, the for sustainable ID. Explicit attention should be paid to the formal and informal institutional arrangements APPLYING THE IDR 11 that characterize the sector, and the stakeholders that activity and when it needs to be done, as well as any need to be engaged in all phases of the project cycle. other necessary information that needs to be recorded The agreed actions should be recorded in the Activity on each activity. Planning column with a clear definition of what the activ- ity is. The next column on “comments and notes” should be used to further detail who is going to carry out which Step 4: Implementation, Monitoring, and Evaluation To enable change and allow for the impact of ID to be forms the basis for developing a PDO that successfully captured, the agreed development areas are translated integrates infrastructure and ID. through a needs assessment into capacity develop- ment responses. This involves the identification of Agreed Development Areas are input in the first column the enablers (or “change agents”) required to support of the template. The next step is for users to consider change (to close the gap) in the areas of people, pro- what is required to enable the change related to the cess, and systems, along with defining specific actions agreed development area. There are three categories related to each enabler. of change enablers on this template that are described below, with questions that users can ask when complet- The indicators of progress are identified as intermediate ing the Enablers of Change section of the template. outputs and as outcomes related to the agreed develop- ment area described in the PDO. • People: Who is required to positively impact the Agreed Development Area? Are there particular Government organizations that require restructur- WHAT IS THE PURPOSE OF THE ID ing or reorganization? Who is impacted? Are there IMPLEMENTATION, MONITORING, AND capacity gaps that require targeted training for EVALUATION? staff? What is the capacity gap and who should receive the training? The ID Implementation, Monitoring, and Evaluation step provides an approach to transforming the agreed • Process: What processes are required to positively development areas (gaps), through a needs assessment, impact the Agreed Development Area? What spe- into packages of ID accompanied by clear indicators cific business practices are required, and do they to monitor and evaluate performance. It recognizes need to be revised or replaced? Are there techni- that the project is a vehicle for ID, representing a step cal area strategies that require revision or replace- change in capacity. The full benefit of the ID may not ment as well? be realized within the duration of the project. However, the actions and intermediate outputs (as indicators of • Systems: What systems are required to positively change) can be developed for the project and for cap- impact the Agreed Development Area? What needs turing impact. to be done to improve these systems? Once this section is completed, the user will have the HOW DO YOU USE THE ID IMPLEMENTATION, granularity required to design useful interventions, tar- MONITORING, AND EVALUATION? get what needs to be regularly monitored and evaluated and how this should be done, and determine what types The ID Implementation, Monitoring, and Evaluation of data will be required as metrics to measure outputs step should be used by TTLs to assist the IA in design- and higher level outcomes needed to monitor and mea- ing the ID, developing indicators for inclusion in the sure progress. PAD. It is based on the scoring process in Step 3 and 12 INSTITUTIONAL DEVELOPMENT ROADMAP HANDBOOK As such, users should challenge themselves to ensure ID components, agreed with the IA and recorded in the they have fully considered all enablers. For example, ID Component column of the template. if the actions identified to enable development in the Agreed Development Area are all under one of the Having agreed on the ID components, users should then template columns (i.e., all actions are related to the reflect on the ID components along with the associated development of systems), then reconsider whether there Development Areas (linking back to the institutional are activities required related to the culture (people) or challenge), and then identify appropriate Intermediate working practices (process) to integrate and operational- Outputs. Consideration should be given to how the activi- ize any investment in systems development. When this ties within the ID component work to enable development is done, the next stage is to determine the ID compo- and what would represent a step change in IA capacity. nents (packages). To develop the PDO, users should review the Users now need to consider whether it is appropriate to Intermediate Outputs identified, develop intermediate package actions related to development area or if effort outcome indicators based on these outputs, and then would be better organized around enablers of change. create a PDO statement that adequately describes the The template in Annex B has been designed to support totality of the project, balancing objectives related to users to look across the range of enabling activities and any civil works constructed and the capacities devel- conceptualize different ways of packaging the interven- oped for their sustainability. tions, rather than simply defining an ID component for each development activity. There are several World Bank and United Nations Development Program reports that provide examples of For example, if there are a significant number of actions indicators that have proved successful in projects across under People Enablers, then it might be a better value to technical areas (see WBI 2009, pp. 74–82; WBI 2012; develop a package of work focusing on organizational UNDP 2008, pp. 41–73). These indicators may need to development. A similar approach should be applied if be adapted for ID specifically in the transport sector. several interrelated process or system interventions are required. These packages can then be described as the APPLYING THE IDR 13 Understanding the Importance of Institutional Development INSTITUTIONAL DEVELOPMENT ROADMAP HANDBOOK Effective Design, Implementation and Evaluation of Project-led Institutional Development in the Transport Sector RADIA BENAMGHAR JUNE 2018 The IDR Toolkit Step 1(a): Contextual Assessment Guiding Questions With reference to Figure 1 it is clear that a thorough assessment of the context in which the invetsment is taking place is important. The prompts in the table below guide the user through such an assessment. Proformas are available for ease of information capture. TABLE 2: FACTORS OF CONTEXTUAL ASSESSMENT AND GUIDING QUESTIONS Factor Description of Factors Guiding Questions SOCIOPOLITICAL ENVIRONMENT Stability of the environment Leadership commitment ❑❑ Q1: Are relevant leaders committed to the reform/ development objectives? The environment stability has an impact on the Implementing Agency's capacity to deliver and will influence how activities and risks will be managed. The likelihood of change and confidence in the Conducive political, economic, and/or ❑❑ Q2: Is the current political, economic, and/or social Implementing Agency's capacity to adapt to that social environment environment conducive for project-led ID? change will inform pre-project activity needed to mitigate the risks and stabilize the environment. Delegated authority ❑❑ Q3: Has the necessary delegated management responsibility/autonomy been granted? Stakeholders and influencers Stakeholders analysis ❑❑ Q1: Who are the relevant stakeholders involved in shaping the development objectives? Are stake- A significant number of stakeholders with high holders likely to change? levels of influence expose the development objectives to risk from differing expectations ❑❑ Q2: Is there a consensus among these stakeholders and influencers regarding the importance of the development objectives? If not what are the differ- ences and why? ❑❑ Q3: Can these differences be addressed or are they irreconcilable? ❑❑ Q4: Is the process for defining the project objec- tives clearly communicated and transparent? ❑❑ Q5: Are there regular, formal engagement and reporting channels? Effective engagement mechanisms ❑❑ Q6: Does the IA/management have experience working with the project stakeholders and/or in the targeted communities? ❑❑ Q7: Does IA have adequate human resource capac- ity to effectively engage key stakeholders? ❑❑ Q8: Are data used to support decision-making and lead stakeholders? ❑❑ Q9: Does the IA have systems in place for citizen engagement? ❑❑ Q10: Is relevant information shared appropriately? POLICY INSTRUMENTS Strategic importance of the project Reform rationale ❑❑ Q1: Does the implementing agency have a good understanding of the strategic reasons for the High expectations of the project outcomes reform? require strong project sponsorship (government commitment and strategic level ownership of the ❑❑ Q2: Has the risk of the introduction of new policy/ business case). Ambitious projects much have strategy been investigated? Are there potentially clear accountability at a senior level conflicting policies? ❑❑ Q3: Is there confidence in the primary IA's ability to lead partners through the proposed reform? 16 INSTITUTIONAL DEVELOPMENT ROADMAP HANDBOOK Factor Description of Factors Guiding Questions Link to policy delivery ❑❑ Q4: Is the project critical to the delivery of policy and key strategic objectives? ❑❑ Q5: Have challenges to delivery been identified at the system level (policy, legal, and regulatory)? Accountability of the project delivery ❑❑ Q6: Have the authorities appointed a senior leader accountable for the project? Are project leaders accountable for the delivery of project outcomes? ❑❑ Q7: Are there regular reporting mechanisms between the project leadership and authorities? ❑❑ Q8: Is there evidence of agreed, realistic project outcomes and representative targets? What relevant official decisions have been made (public record)? ❑❑ Q9: Is there a clear record of the IA and relevant entities following key strategic objectives? Alignment of objectives Interdependencies with other ❑❑ Q1: Are there multiple funding organizations and/or Government supported objectives responsible implementing agencies? Misaligned or poorly followed objectives reduce the likelihood of the project-led ID contributing to ❑❑ Q2: Have the institutional challenges been identi- sustained capacity development fied and agreed on? ❑❑ Q3: Are there clear and jointly owned sector devel- opment targets (national, state, and/or local level)? What activities on the critical path are not under the control of the primary IA? ❑❑ Q4: Does planning sufficiently consider the timing of other initiatives? ❑❑ Q5: How would you describe the working relation- ships between the IA and other relevant entities? Alignment with regional/national strategy ❑❑ Q6: Are the proposed project development objec- tives sufficiently aligned with national and state policies and strategies? ❑❑ Q7: How important are national and state policies and strategies to sector priorities and decision making? ❑❑ Q8: Are there other factors that impact sector priori- ties and decision making? ❑❑ Q9: Can the proposed objectives/development goal be clearly linked to sector-level planning and Government investment priorities? Performance of the agency ❑❑ Q10: Is the development objective dependent on a significant amount of institutional development? ❑❑ Q11: Is there sufficient leadership and senior level experience? ❑❑ Q12: Has the solution been successful in other similar environments? ❑❑ Q13: Is there a detailed implementation plan that covers how policy, culture, working practices, tech- nology, people, processes, and procedures work together for sustainable development? ❑❑ Q14: Is the development objective concentrated but expected outcomes are broad? ❑❑ Q15: Is there a clear line of sight between policy, expected outcomes, and project activity? THE IDR TOOLKIT 17 Factor Description of Factors Guiding Questions Financial viability Availability and sources of funding ❑❑ Q1: Is there a robust business case for investment? If the proposed project is of significant value, the ❑❑ Q2: Does the project represent highly involved funding arrangements for the project are complex, type/source of finance and/or multiple donor or the project is expected to deliver significant contributions? value for the money, more effort should be put into helping the IA prepare a robust business case ❑❑ Q3: Is there adequate availability of counterpart funding from the borrower? ❑❑ Q4: To what extent are decision makers account- able for the use of financial resources? Budget management ❑❑ Q5: Is there a comprehensive budget supported by robust financial modelling? ❑❑ Q6: To what extent does the IA have control of the budget? What are the specific financial manage- ment strengths and weaknesses of the IA and other relevant entities? ❑❑ Q7: How does the implementing entity derive its authority to manage the proposed funding regime? ORGANIZATIONAL ARRANGEMENTS Execution complexity Extent of change ❑❑ Q1: Is a fundamental change to the IA (significant change in the way the primary delivery agency Challenging development objectives in short time conducts its work) required in a short time frame? frames should be investigated for the capacity of the IA to accept or implement the changes ❑❑ Q2: Has the solution been tried in other states, countries? The amount of transformational change required to support new ways of working (e.g., new ❑❑ Q3: Is there a representative organizational devel- delivery models) should be evaluated. If the opment/human resource strategy in place? magnitude of the change is significant, it will need to be managed as a transformational change ❑❑ Q4: Does the IA have the capacity and will to follow through on these reforms? program, thus providing progressive development through a series of tranches ❑❑ Q5: Are there sufficient/qualified human resources to ensure for implementation? ❑❑ Q6: Does the IA have the authority to make the operational changes required for successful project implementation? ❑❑ Q7: Is the organizational culture of the IA conducive to change? Project management/expertise ❑❑ Q8: Is there a robust implementation strategy/plan covering risk management, budget, and schedule? ❑❑ Q9: Has the use of bespoke solutions been suffi- ciently interrogated for value for money? ❑❑ Q10: Does the IA and other related entities have a documented track record of implementing challeng- ing projects in short time frames? ❑❑ Q11: Does the IA have a database of projects with key data captured for review and analysis? ❑❑ Q12: How is institutional knowledge about past and current projects stored and used by the IA? ❑❑ Q13: Is there a wide project scope or challenging objectives that will only be achievable through external support? Is the development or use of a large number of specialist disciplines and skills required? Has the balance between insourcing and outsourcing resources been investigated? 18 INSTITUTIONAL DEVELOPMENT ROADMAP HANDBOOK Factor Description of Factors Guiding Questions Need for innovative/untested ❑❑  Q14: Does the proposed project involve new technologies or untested business practices or technologies? ❑❑  Q15: Has the market been sufficiently investigated? ❑❑  Q16: Has the IA flexibly responded to unex- pected discipline and skill requirements? ❑❑  Q17: Does the IA have the budget flexibility required to respond to unexpected requirements? ❑❑  Q18: Does the IA have the organizational culture to adapt to changing requirements during implementation? Step 1(b) - Assessing Context against Standards Guiding Questions Given the importance of the contextual assessment the questions provided below have been designed to aid the user gain a deeper understanding. STABILITY OF THE ENVIRONMENT If the environment is unstable (there is a high likelihood of change and low confidence in the IA's capacity to adapt to that change), then preparation activity to stabilize the environment or manage risk are required. ASSESSMENT QUESTION 1: ❑❑ Are relevant leaders committed to the reform/development objectives? ASSESSMENT LEVELS ON TYPICAL STANDARDS: • Level 1: Ownership of the project is weak, and legislation has not been enacted or effectively approved through appropriate channels. • Level 2: Adequate legal frameworks are in place and aligned with investment policies. • Level 3: Legal frameworks and investment plans are clearly articulated in a regional/national development plan and a public monitoring mechanism is in place. ASSESSMENT QUESTION 2: ❑❑ Has the necessary delegated management responsibility/autonomy been granted? ASSESSMENT LEVELS ON TYPICAL STANDARDS: • Level 1: There is no formal delegation of authority and/or the delegation is too limited to ensure smooth operations. • Level 2: The IA has a leadership team appointed and stable with sufficient authority to ensure regular day-to- day operations are effective. • Level 3: The IA leadership team makes strategic decisions and can adapt direction of the works to the needs for growth of the agency. THE IDR TOOLKIT 19 ASSESSMENT QUESTION 3: ❑❑ Is the current political, economic, and/or social environment conducive for project-led ID? ASSESSMENT LEVELS ON TYPICAL STANDARDS: • Level 1: There is evidence of high turnover in project leadership linked to new governmental appointments. • Level 2: There is public disclosure and monitoring of public investments. • Level 3: There is public disclosure and monitoring of public investments. If commitment/stability is uncertain, what preparation activity or capacity development is required? ❑❑ Robust analysis of the potential impact of the political, economic, and/or social environment and scenario planning. ❑❑ A charter/memorandum of understanding between Government/project leaders, the IA(s), and key stakeholders. ❑❑ Allocation of contingencies to manage the political risk. ❑❑ Phased investment and/or delivery is recommended. ❑❑ A reform roadmap is needed to address governance challenges. ❑❑ Other: STAKEHOLDERS AND INFLUENCERS A significant number of stakeholders with high levels of influence expose the development objective to risk from differing expectations. ASSESSMENT QUESTION 1: ❑❑ Are there effective stakeholder engagement mechanisms? ASSESSMENT LEVELS ON TYPICAL STANDARDS: • Level 1: Key stakeholders have not been identified and actions can have negative impact/cause delay to actions or programs. • Level 2: There is a clear identification of key stakeholders and an engagement plan in place. Differences in interests can be addressed; however, incentives are not aligned to support the development goals. • Level 3: Influential stakeholders demonstrate open commitment and seek to constantly bring consensus with others to achieve results. ASSESSMENT QUESTION 2: ❑❑ Is there common ground between the primary stakeholders and influencers regarding the development objectives? ASSESSMENT LEVELS ON TYPICAL STANDARDS: • Level 1: Engagements and communications are not initiated by the IA. There is no experience/authority in the IA in managing stakeholders’ engagements. • Level 2: There is a lack of experience in managing networks of stakeholders and no formal communication and engagement strategy has been defined. Actions are ad hoc and reactive. • Level 3: There is an engagement and communication strategy supporting the development goals. 20 INSTITUTIONAL DEVELOPMENT ROADMAP HANDBOOK If the impact of stakeholders is uncertain, what preparation activity or capacity development is required? ❑❑ Improve mechanisms for stakeholder engagement. ❑❑ Develop or improve mechanism for citizen engagement in potential project areas. ❑❑ Formal partnerships with key stakeholders are needed. ❑❑ Increase time/resource for pre-project activity (e.g., land acquisition). ❑❑ Other: STRATEGIC IMPORTANCE OF THE PROJECT High expectations of the project outcomes require strong project sponsorship (government commitment and strategic level ownership of the business case). Ambitious projects much have clear accountability at a senior level. ASSESSMENT QUESTION 1: ❑❑ Does the IA have a good understanding of the strategic reasons for the reform? ASSESSMENT LEVELS ON TYPICAL STANDARDS: • Level 1: The objectives of the reform are not linked to the IA's work program. There is a lack of trust /under- standing of the reform agenda amongst the IA's leadership. • Level 2: The reform is not integrated in the IA's work program. There is reluctance/significant delays in design- ing or implementing new policies or frameworks supporting the new strategy. • Level 3: The IA is a key element driving and communicating the reform through formal and public mechanisms. The risks are regularly monitored and there is demonstration of mitigation plans in place and/or communicated. ASSESSMENT QUESTION 2: ❑❑ Are project leaders accountable for the delivery of project outcomes? ASSESSMENT LEVELS ON TYPICAL STANDARDS: • Level 1: The objectives of the reform are not linked to the IA's work program. There is a lack of trust/under- standing of the reform agenda amongst the IA's leadership. • Level 2: Results of the team are monitored; however, there is no evidence of a formal engagement between the IA and authorities; outcomes/results are attributed to the project. • Level 3: There are clear lines of accountability between the IA and rewards, and sanction mechanisms are in place to support the leadership and staff motivation to obtain results. ASSESSMENT QUESTION 3: ❑❑ Is the project critical to the delivery of policy and key strategic objectives? ASSESSMENT LEVELS ON TYPICAL STANDARDS: • Level 1: The link between project outcomes and policy is limited. • Level 2: Delays in policy implementation have minor impact on delivery of the works. Challenges are observed however addressed in a passive manner. • Level 3: There is evidence that the implementation of the policy has an impact and that formal systems are in place to ensure effective delivery. THE IDR TOOLKIT 21 If the strategic importance of the project is uncertain, what preparation activity or capacity development is required? ❑❑ A review and update of legislation/policy for its ability to accommodate revision/ effective policy and legisla- tive framework is required. ❑❑ An Institutional Development Study. ❑❑ A Governance and Accountability Action Plan (GAAP). ❑❑ Other: ALIGNMENT OF OBJECTIVES Misaligned or poorly followed objectives reduce the likelihood of the project contributing to sustained development. ASSESSMENT QUESTION 1: ❑❑ Are the proposed project development objectives sufficiently aligned with national and state policy and strategies? ASSESSMENT LEVELS ON TYPICAL STANDARDS: • Level 1: The project is not aligned with national/sectoral strategy, and decision making is adversely impacted. There is a lack of resources to ensure sustainability of the project. • Level 2: There is a reasonable alignment between the project's development objectives and national/regional strategy; however, there is a lack of articulated priorities that impacts, or delays decision making. Investments are de-linked from project objectives. • Level 3: The objectives of the project are leveraged by the national/regional development plan the project outcomes influence the sector's planning and investment priorities. ASSESSMENT QUESTION 2: ❑❑ Have the institutional challenges been identified and agreed? Are there clear and jointly owned sector development targets (national, state, and/or local levels)? What activities on the critical path are not under the control of the primary IA? How would you describe the working relationships between the IA and other relevant entities? ASSESSMENT LEVELS ON TYPICAL STANDARDS: • Level 1: The implementation plan is isolated from other relevant government initiatives. There are duplication of efforts and resources within the sector/There is unclarity of ownership between different entities. • Level 2: Relations between different initiatives are established; however, there are dysfunctional or contradic- tory approaches/policies that create adverse impact on the ability of the IA to operate effectively or efficiently. The various implementation plans are note aligned creating confusion or distraction of resources. • Level 3: The interdependencies between government agencies and/or different IA are defined at a strategic level and resources are leveraged to the benefit of results for all programs. The roles and responsibilities of the different actors are clear and accepted, and relationships between the IAs are strengthening overall results and development goals. ASSESSMENT QUESTION 3: ❑❑ Is the development objective dependent on a significant amount of institutional development? 22 INSTITUTIONAL DEVELOPMENT ROADMAP HANDBOOK ASSESSMENT LEVELS ON TYPICAL STANDARDS • Level 1: The IA is not operating at expected level of results. There are significant gaps between outcomes and resources. There is a lack of experienced leadership in managing the IA. • Level 2: The IA is well managed and achieves most of its expected results. The implementation plan is updated and aligned with work practices in the sector/region/nation. Activities are regularly monitored and corrective measures are in place. Qualified and recognized leaders are in place in the IA. • Level 3: The IA is a best demonstrated practice for the sector and beyond. Work practices, policies, and staff- ing models have been replicated to support other projects. Business processes and/or innovation have had positive impacts on results beyond the project, in the sector, and/or at the region/national level. If the alignment of the objectives is uncertain, what preparation activity or capacity development is required? ❑❑ Enhanced engagement and coordination with funders and other responsible agencies (clear lines of deci- sion making, partnerships). ❑❑ The business model needs to be considered for its relevance to wider policies/strategies. ❑❑ Sector policy, institutional and legal framework development. ❑❑ Leverage joint working. ❑❑ Gather more information around the development needs. ❑❑ A review and update of legislation/policy to clarify roles and responsibilities of relevant entities in the sector. ❑❑ Other: FINANCIAL VIABILITY If the proposed project is of significant value, the funding arrangements for the project are complex or the project is expected to deliver significant value for money, effort should be put into ensuring there is a robust business case. ASSESSMENT QUESTION 1: ❑❑ Can the development objectives be reasonably delivered under the proposed funding regime? ASSESSMENT LEVELS ON TYPICAL STANDARDS: • Level 1: The project is underfunded, or resources have been misallocated. Counterpart's budget has been revised with an adverse impact on the project. Funding of the project requires significant revision of funding. • Level 2: The IA is sufficiently funded to operate within budget requirements. Activities are well provided for, and there is a transparent budget in place. Budget control function is in place internally and externally. • Level 3: The project activities are strategically funded through multiple sources that leverage each other. There are formal and public collaboration mechanisms in place providing transparent monitoring and control from the public. ASSESSMENT QUESTION 2: ❑❑ Does the IA have the independence, capacity and authority to manage the proposed funding regime? ASSESSMENT LEVELS ON TYPICAL STANDARDS: • Level 1: The proposed budget is not aligned with project activities. The IA has limited control over budget and expenditures. Financial management skills need to be strengthened. THE IDR TOOLKIT 23 • Level 2: The IA has the means to be accountable for financial management. Complex financial management and budgeting skills and experience are available in the team. Highly fragmented institutional arrangements. • Level 3: The strategic planning includes a long-term, multiyear budget aligned with regional/national plans. Streamlined planning and budgeting processes are in place. Business improvement processes are regularly implemented. If the financial viability is uncertain, what preparation activity or capacity development is required? ❑❑ Data collection for investment decision making is needed. ❑❑ Financial modelling/planning. ❑❑ Plans in place to secure additional finance if required. ❑❑ Support for assurance of estimates and expenditure. ❑❑ Other: DEPENDENCIES If the development objective is dependent on other initiatives or a significant amount of institutional development, then consideration should be given to how the proposed project can leverage joint-working with other projects. ASSESSMENT QUESTION 1: ❑❑ Is the development objective dependent on other initiatives for success? GUIDING QUESTIONS: • What activities on the critical path are not under the control of the primary IA? • Does planning sufficiently consider the timing of other initiatives? • Is the development objective dependent on a significant amount of institutional development? • Does the level of pre-project activity reflect the scale of the institutional development required? ASSESSMENT QUESTION 2: ❑❑ Is there a documented track record of successful joint projects? GUIDING QUESTIONS: • Does the government track joint projects and have metrics for assessing progress? • Are there successful projects that can be reviewed for lessons learned and best practices? • Does the government have a system in place to learn and apply lessons from past joint projects? ASSESSMENT QUESTION 3: ❑❑ Are there political consequences for project success or project failure? GUIDING QUESTIONS: • Are there clear political “winners” and “losers” related to project outcomes? • Can implementation by the IA be separated from these political consequences? 24 INSTITUTIONAL DEVELOPMENT ROADMAP HANDBOOK If dependencies exist, what preparation activity or capacity development is required? ❑❑ A sector-wide monitoring framework is required. ❑❑ Increased pre-project activity to plan ID. ❑❑ Joint-working with other projects. ❑❑ Develop a data based on past joint projects and to capture lessons learned ❑❑ Other: EXECUTION COMPLEXITY Challenging objectives in short time frames should be investigated for the capacity of the IA to accept or implement the changes (i.e., sufficient time for institutional development). ASSESSMENT QUESTION 1: ❑❑ Is there a wide project scope or are there challenging objectives that will only be achievable through external support? GUIDING QUESTIONS: • Is there a robust implementation strategy/plan covering risk management, budgets and schedules? • Does the proposed project involve new or untested business practices or technologies? • Is there evidence that a solution exists? • Has the use of bespoke solutions been sufficiently interrogated for value for money? ASSESSMENT QUESTION 2: ❑❑ Do the IA and other related entities have a documented track record of implementing challenging projects in short time frames? GUIDING QUESTIONS: • Does the IA have a database of projects with key data captured for review and analysis? • How is institutional knowledge about past and current projects stored and used by the IA? If the implementation is complex, what preparation activity or capacity development is required? ❑❑ Enhanced project management capability (PMO) needs to be developed/acquired. ❑❑ Partnering for delivery should be considered (the use of partners where the time frame/appetite is insuffi- cient for in-house development). ❑❑ Phased implementation, piloting, and/or significant training is required. ❑❑ Other: EXTENT OF CHANGE THE IDR TOOLKIT 25 The amount of transformational change required to support new ways of working (e.g., new delivery models) should be evaluated. If the magnitude of the change is significant, it will need to be managed as a transformational change program, thus providing progressive development through a series of tranches. ASSESSMENT QUESTION 1: ❑❑ Can the existing IA and other relevant entities realize the development objectives? GUIDING QUESTIONS: • Is a fundamental change to the IA (significant change in the way the primary delivery agency conducts its work) required in a short time frame? • Has the solution been tried in other states, countries? • Is there a representative organizational development/human resource strategy in place? ASSESSMENT QUESTION 2 ❑❑ Are there specific reforms required for the IA and other relevant entities to successfully implement project- led ID? GUIDING QUESTIONS: • • What reforms are required for implementing project-led ID? • • If the needed reforms are significant, does the IA have the capacity and will to follow through on these reforms? ASSESSMENT QUESTION 3: ❑❑ Does the IA have the flexibility to use new approaches that require changes in organizational and/or indi- vidual behaviors? GUIDING QUESTIONS: • • Does the IA have the authority to make the operational changes required for successful project implementation? • • Is the organizational culture of the IA conducive to change? If there is transformational change needed, what preparation activity or capacity development is required? ❑❑ Implementation of a transformational change program. ❑❑ Development of an Organizational Design and Development/Human Resource Strategy is required. ❑❑ Partnering for delivery should be considered. ❑❑ Increased pre-project activity to plan ID. ❑❑ Other: 26 INSTITUTIONAL DEVELOPMENT ROADMAP HANDBOOK Step 1(c) - Maturity levels of various areas of Contextual Assessment Key Areas Level 1: Lagging Level 2: Good Practice Level 3: Exemplar There are significant organizational Some elements of good practice There is an outcome focus, and challenges. Practices, processes, are being utilized. Systems and maximum value is achieved from the and systems are outdated or processes are appropriate for the asset and external relationships. immature. current requirement but may not be sustainable. Socio-Political Environment ❑❑ Leadership commitment ❑❑ Ownership of the project is ❑❑ Adequate legal frameworks ❑❑ Legal frameworks and investment weak, and legislation has not are in place and aligned with plans are clearly articulated in a been enacted or effectively investments policies regional/national development approved through appropriate plan, and a public monitoring channels. mechanism is in place. ❑❑ Conducive political, economic, ❑❑ There is no investment plan, or ❑❑ Limited political interference ❑❑ There is no political interference and/or social environment it is disconnected to the project and citizen consultations; on reform processes; there are objectives economic environment is regular citizen consultations fluctuating ❑❑ Delegated authority ❑❑ There is evidence of high ❑❑ IA has a leadership team ❑❑ Stable and growing economic turnover in project leadership appointed and stable, with environment linked to new governmental sufficient authority to ensure appointments regular day-to-day operations are effective ❑❑ Stakeholder analysis ❑❑ There is no formal delegation of ❑❑ There is a clear identification ❑❑ The IA leadership team makes authority and/or the delegation of key stakeholders and an strategic decisions and can adapt is too limited to ensure smooth engagement plan in place. direction of the works to the operations Differences in interests can be needs for growth of the agency addressed; however incentives are not aligned to support the development goals ❑❑ Effective engagement ❑❑ Key stakeholders have not been ❑❑ There is a lack of experience ❑❑ Influential stakeholders mechanisms identified and actions can have in managing networks demonstrate open commitment a negative impact/cause delays of stakeholders, and no and seek to constantly bring to actions or programs. formal communication and consensus with others to achieve engagement strategy has been results. ❑❑ Conflicts and difference defined. Actions are ad hoc and of interests exist amongst ❑❑ There are clear incentives in reactive place aligned with development stakeholders goals, and formal resolution ❑❑ Engagements and mechanisms exist to solve issues communications are not initiated by the IA. ❑❑ There is an engagement and communication strategy ❑❑ There is no experience/ supporting the development authority in the IA in managing goals stakeholders’ engagements continued THE IDR TOOLKIT 27 Key Areas Level 1: Lagging Level 2: Good Practice Level 3: Exemplar Policy Instruments ❑❑ Reform rationale ❑❑ The objectives of the reform ❑❑ The reform is not integrated in ❑❑ The IA is a key element driving are not linked to the IA's work the IA's work program. There and communicating the reform program. There is a lack of is reluctance/significant delays through formal and public trust /understanding of the in designing or implementing mechanisms. The risks are reform agenda amongst the IA's new policies or frameworks regularly monitored and there is leadership supporting the new strategy demonstration of mitigation plans in place and/or communicated ❑❑ Link to policy delivery ❑❑ The link between project ❑❑ Delays in policy implementation ❑❑ There is evidence that the outcomes and policy is limited have minor impact on delivery implementation of the policy of the works. Challenges has an impact and that formal are observed, however are systems are in place to ensure addressed in a passive manner effective delivery ❑❑ Accountability of the project ❑❑ There is a high turnover in ❑❑ Results of the team are ❑❑ There are clear lines of delivery leadership or inadequate level monitored; however, there accountability between the of appointment of the team and is no evidence of a formal IA and rewards, and sanction no consequences nor rewards engagement between the IA mechanisms are in place to system in place. and authorities support the leadership and staff motivation to obtain results ❑❑ Interdependencies with ❑❑ The implementation plan is ❑❑ Relations between different ❑❑ The interdependencies between other Government supported isolated from other relevant initiatives are established; government agencies and/ objectives Government initiatives. however, there are or different IA are defined at a dysfunctional or contradictory strategic level, and resources ❑❑ There are duplication of efforts approaches/policies and are leveraged to the benefit of and resources within the sector. results for all programs. creative adverse impact on ❑❑ There is unclarity of ownership the ability of the IA to operate between different entities effectively or efficiently. ❑❑ The various implementation plans are note aligned creating confusion or distraction of resources ❑❑ Alignment with regional/national ❑❑ The project is not aligned with ❑❑ There is a reasonable ❑❑ The roles and responsibilities strategy national/sectoral strategy, and alignment between the of the different actors are decision making is adversely project's development clear and accepted, and impacted. objectives and national/ relationships between the IAs are regional strategy; however, strengthening overall results and ❑❑ There is a lack of resources there is a lack of articulated development goals to ensure sustainability of the priorities that impacts or project delays decision making. Investments are de-linked from project objectives ❑❑ Performance of the agency ❑❑ The IA is not operating at ❑❑ The IA is well managed and ❑❑ The objectives of the project expected level of results. achieves the majority of the are leveraged by the national/ expected results. regional development plan. ❑❑ There are significant gaps between outcomes and ❑❑ The implementation plan is resources. updated and aligned with work practices in the sector/region/ ❑❑ There is a lack of experienced nation. leadership in managing the IA ❑❑ Activities are regularly monitored and corrective measures are in place. ❑❑ Qualified and recognized leaders are in place in the IA 28 INSTITUTIONAL DEVELOPMENT ROADMAP HANDBOOK Key Areas Level 1: Lagging Level 2: Good Practice Level 3: Exemplar ❑❑ Availability and sources of ❑❑ The project is underfunded, ❑❑ The IA is sufficiently funded ❑❑ The project outcomes influence funding or resources have been to operate within budget the sector's planning and misallocated. requirements. Activities are investment priorities well provided for, and there is a ❑❑ Counterpart’s budget has been transparent budget in place. revised with an adverse impact on the project. ❑❑ Budget control function is in place internally and externally ❑❑ Funding of the project requires significant revision of funding ❑❑ Budget management ❑❑ The proposed budget is not ❑❑ The IA has the means to be ❑❑ The project activities are aligned with project activities. accountable for financial strategically funded through management. multiple sources that leverage ❑❑ The IA has limited control over each other. budgets and expenditures. ❑❑ Complex financial management and budgeting skills and ❑❑ There are formal and public ❑❑ Financial management skills experience are available in the collaboration mechanisms in need to be strengthened place providing transparent team. monitoring and control from the ❑❑ Highly fragmented institutional public arrangements Organizational Arrangements ❑❑ Extent of change ❑❑ Reactive behavior to change; ❑❑ There is evidence of a ❑❑ The strategic planning includes undefined leadership roles reasonable appetite for change a long-term, multiyear’s budget for driving the change; lack in the IA. Change had no or aligned with regional/national of service orientation; lack of limited impact on staff turnover. plans. incentives to perform. Change had a recognized ❑❑ Streamlined planning and positive impact on service budgeting processes are in place. ❑❑ There is no evidence/track delivery. There is a public record of effective reforms. ❑❑ Business improvement processes support from leadership for There are limited staff allocated are regularly implemented change to research and innovation. Change has created high ❑❑ Changes and innovation are turnover anticipated and planned for. There is a positive track record of innovation implementation. Change has created recognized opportunities for the IA. Leadership is acknowledged as a champion for change ❑❑ Project management/expertise ❑❑ There are no information ❑❑ Systems are in place, however ❑❑ Strong systems are in place to systems in place. Project do not operate effectively. The plan, implement, and monitor management is outsourced. delivery unit is understaffed projects. There is a strong There is no clear accountability or under qualified. Project and capable delivery unit in for project management. management teams are formed, place, leading multiple complex Archiving system is outdated however, in an ad hoc manner/ projects in a programmatic and not up to date as needed basis manner. Information systems are connected, and best practices constantly reviewed and integrated in new projects ❑❑ Need for innovative/ untested ❑❑ Lack of use of innovative ❑❑ Slow and/or ad hoc ❑❑ New technologies are easily technology technology. Weak implementation of new embedded in current business implementation of current technology. Staff has limited practices with positive impacts on business practices. No budget access to innovation. Rigid service delivery. Best practices allocation to innovation and budget and decision-making are regularly analyzed. Staff have technology processes supporting allocated time for research and innovation innovation activities. Fungible business practices THE IDR TOOLKIT 29 Step 2 (a): Institutional Capability Assessment TABLE 4: FACTORS OF CAPABILITY ASSESSMENT AND GUIDING QUESTIONS Factor Description of Factors Guiding Questions PROJECT GOVERNANCE ARRANGEMENTS Effective project governance aligns projects to the Objective statement: The Government ❑❑ Q1. Does the government have a clear overall organizational/development goals, ensures has a clear vision for its development vision? that the project fulfils its purpose and is effective, objectives efficient and transparent ❑❑ Q2. Does the legal framework support development objectives? Good project governance clearly sets out decision-making responsibilities against an agreed ❑❑ Q3. Do governance arrangements support the project' s development objectives? statement of objectives providing the project team Policy Instruments the autonomy to deliver. Formal framework: The project/ ❑❑ Q4. Are the current governance arrange- implementing agency has a formal ments/framework effective? (system for governance framework that sets out delegation, process for decision-making/ decision making and delegation of change control)? authority for project preparation and implementation M&E framework: There is a robust ❑❑ Q5. Are there systems in place for data monitoring and evaluation system collection? Do reporting and accountability mechanisms need to be revised? ❑❑ Q6. Is there coordinated assurance? HUMAN RESOURCES Strategic human resources (HR) management Staffing: The project is adequately ❑❑ Q1: Is there a staffing/recruitment policy are aligned to the overall project objectives staffed (quantity and quality) based on competency and business needs and subsequent development goals. It supports in place? professional, effective, and transparent management practices that will enforce ❑❑ Q2: Are staffing processes transparent? Organisational Arrangements accountability and commitment of leaders to change Policies: The project/implementing ❑❑ Q3: Are procedures and policies communi- agency has a set of procedures, policies, cated and applied in a consistent manner? Good HR policies and practices create positive and practices that supports results and work ethics and help management attract, recognition of results. Staffing decisions ❑❑ Q4: How is this tracked and recorded? develop, and motivate staff to achieve results and are made based on objective and ❑❑ Q5: Is management accountable for good deliver qualitative services to the public transparent criteria HR decisions and strategies? Training: There is a competency-based ❑❑ Q6: Is the training plan/policy communicated training plan in place. It is adequately to staff individually? budgeted and funded, regularly monitored, and updated ❑❑ Q7: Are staff competencies measured as an outcome of the training plan? 30 INSTITUTIONAL DEVELOPMENT ROADMAP HANDBOOK Factor Description of Factors Guiding Questions ENVIRONMENTAL AND SOCIAL SAFEGUARDS Environment and social assessments conducted Procedures and processes: The ❑❑ Q1: "Are there robust procedures for land will help identify gaps in policies and procedures implementing agency has a set of acquisition, social safeguards? to determine related risks and impacts procedures, policies, and framework in place that supports land acquisition, ❑❑ Q2: Is there a comprehensive operations manual that is regularly updated? resettlements, etc. ❑❑ Q3: Are there robust procedures and frame- work for environmental safeguards? Labor: The IA fully understands labor ❑❑ Q4: Are labor laws implemented in a trans- laws and implements them transparently parent and fairly? ❑❑ Q5: Are employment processes (contracts etc..) in the communities where roads are being built inclusive of community members? ❑❑ Q6: In the event of labor influx to a construc- tion site how does the IA engage with the community? Road safety: The IA has adequate ❑❑ Q7: Are there sufficient resources (human resources (human and financial) to and financial) to manage roads safety Organisational Arrangements address road safety reporting issues as reporting? well as manage traffic enforcement ❑❑ Q8: Is there a process to report safety issues? Is it effective? ❑❑ Q9: Have line management arrangements considered any improvements? ASSET MANAGEMENT APPROACH Asset management is the provision and Life cycle planning: There is a 'whole ❑❑ Q10: Are new approaches to lifecycle maintenance of services to customers. Good asset life' approach to asset management planning, programmed maintenance and management utilizes a system of assets to achieve environmental management required? an outcome ❑❑ Q11: Is there outcome performance This is founded on a robust, formalized approach monitoring? to asset management that includes outcome performance measurement linked to customer ❑❑ Q12: Is customer engagement, asset condi- tion surveys, asset performance monitoring expectations. Good asset management practice required? is evidenced by informed planning for capital delivery, network, improvement and maintenance operations Quality of information: There is good ❑❑ Q13: Does the IA have the information to quality asset information to support challenge specialist requirements? Are the decision making required information management systems established? ❑❑ Q14: Is the IA focused on policy translation and risk-based prioritization? Is outcome performance monitoring linked to customer expectations? Network planning: There is adequate ❑❑ Q15: Is there an operational readiness plan? planning for network improvement and maintenance operations ❑❑ Q16: Are there funds to sustain operating costs? THE IDR TOOLKIT 31 Factor Description of Factors Guiding Questions BUSINESS PRACTICES AND IMPLEMENTATION PROCEDURES Application of modern industry/business practices Fiduciary capabilities (FM and ❑❑ Q1: Does the IA have the capability to plan is critical to sustainable development procurement): The implementing and supervise/coordinate activities (financial agency has strong project delivery management, procurement capability)? capability ❑❑ Q2: Have project management and risk management arrangements been considered? ❑❑ Q3: Is there a defined set of activities accompanied by a budget, timeline, and assigned personnel? Organisational Arrangements ❑❑ Q4: Has the distribution of in-house/external resources been considered? Key functions and systems: They ❑❑ Q5: Are there adequate systems and proce- support the development goal and dures for: procurement, financial manage- have been identified and resourced. ment, contract management, road safety The implementing agency has strong management, environmental and social Implementation procedures safeguards? ❑❑ Q6: Are there existing solutions that could be leveraged? Have line management arrangements for implementation been considered? Is there a comprehensive operations manual that is regularly updated? Market understanding: The market ❑❑ Q7: How will supplier quality be ensured? related to the achievement of the development objectives is well ❑❑ Q8: Have appropriate measurement, metrics, and targets been established? understood ❑❑ Q9: Is there a plan for achieving increased value from the competition process? ❑❑ Q10: Is there a well-defined packaging strat- egy? How will the market be incentivized? 32 INSTITUTIONAL DEVELOPMENT ROADMAP HANDBOOK Step 2 (b): Maturity level of Capability Assessment Key Areas Level 1: Lagging Level 2: Good Practice Level 3: Exemplar There are significant organizational Some elements of good practice There is an outcome focus, and challenges. Practices, processes, are being utilized. Systems and maximum value is achieved from the and systems are outdated or processes are appropriate for the asset and external relationships. immature. current requirement but may not be sustainable. Project governance arrangements ❑❑ Objectives statement ❑❑ Ownership of the asset is ❑❑ The project/IA has a clear ❑❑ Governance arrangements fragmented, and development statement of the objectives provide clear accountability to is subject to conflicting priorities from project leaders sponsoring organization ❑❑ Formal framework ❑❑ Unclear/ineffective governance ❑❑ The project/ IA has a formal ❑❑ Maximum value is achieved from arrangements governance framework that external relationships sets out decision making and delegation of authority ❑❑ M&E framework ❑❑ A lack of devolved responsibility ❑❑ There is a robust monitoring ❑❑ Establishes appropriate targets and diffuse accountability and evaluation system for success Human resources ❑❑ Staffing ❑❑ Weak/no link between ❑❑ Established link between ❑❑ There is a clear organizational recruitments and business recruitment and business needs chart with job descriptions and needs but weak/no transparency in the all recruitments are based on process. business needs ❑❑ Policies ❑❑ Weak implementation of staffing ❑❑ Staffing policies are ❑❑ Full transparency in the policies and no accountability in implemented but management implementation policies and HR decisions and strategies lacks accountability on HR procedures and management is decisions and strategies accountable to HR decisions ❑❑ Training ❑❑ Ad hoc/no training plan for staff ❑❑ Training plans exist but ❑❑ Training plans are fully funded implementation is weak due to and monitored/full transparency inadequate funding in the implementation of training plans Environmental and social safeguards ❑❑ Procedures and processes ❑❑ Weak to no implementation ❑❑ Social and environmental ❑❑ Implementation of social of social and environmental policies are in place and being and environmental policies safeguards policies and implemented, but processes are and procedures are fully procedures cumbersome and inefficient transparent, and there is a sound accountability to users ❑❑ Labor ❑❑ Weak implementation of labor ❑❑ Labor laws implementation ❑❑ Implementation of labor laws are laws and lack of inclusion of processes are in place, and advanced and fully transparent local communities local inclusion processes are to contractors and community weak members ❑❑ Road safety ❑❑ Weak to fragmented ❑❑ Institutional arrangements ❑❑ Institutional arrangements and arrangement on road safety and are in place but inefficiency in internal processes function traffic reporting implementation exists optimally to improve road safety reporting and traffic enforcement THE IDR TOOLKIT 33 Key Areas Level 1: Lagging Level 2: Good Practice Level 3: Exemplar Asset management approach ❑❑ Life cycle planning ❑❑ Poor decision making ❑❑ There is good quality asset ❑❑ Asset management approach undermines asset management information to support decision leads to overall value for money making and efficient service delivery ❑❑ Quality of information ❑❑ Limited application of ❑❑ There is good quality analysis of ❑❑ There is an understanding of modern planning for network asset information and application of whole life cost improvement and maintenance and environment management operations principles. ❑❑ Network planning ❑❑ No appropriate asset ❑❑ There is adequate planning ❑❑ Intelligent use of data. information for network improvement and maintenance operations Business practices and implementation procedures ❑❑ Fiduciary capabilities ❑❑ Processes, systems, and ❑❑ The IA has strong asset ❑❑ Application of modern industry/ technology do not support management and project business practices. effective asset management or delivery capability delivery ❑❑ Key functions to support the development objectives have been identified and resourced ❑❑ The IA has strong Implementation procedures ❑❑ Key functions and systems ❑❑ Absence of the required ❑❑ The core processes and ❑❑ Effectively balances in-house and specializations and high staff systems meet current and external resources. turnover future asset management and project delivery needs ❑❑ Fractured and non-transparent procurement processes; weak supplier selection, contract award, and management; outdated management and engineering practices ❑❑ Processes and systems do not support effective asset management and/or service delivery ❑❑ Market understanding ❑❑ Limited understanding of actual ❑❑ The market related to ❑❑ Leverages international suppliers market capacity, capability or the achievement of the for continuous improvement appetite development objective is well understood ❑❑ No investment in staff development ❑❑ The IA is committed to continuous staff improvement and sustained development 34 INSTITUTIONAL DEVELOPMENT ROADMAP HANDBOOK Step 3: Design of ID activity Activity Planning Log Project Preparation Agreed Activity Status Activity What Who When Agreed Development Areas (scope of ID) THE IDR TOOLKIT 35 Step 4: Implementation, Monitoring and Evaluation IDRt Scoring Tool Card Contextual Assessment Self-assessment Expert Socio-Political Environment Stability of the Leadership commitment environment Conducive political, economic, and/or social environment Delegated authority Stakeholders and Stakeholder analysis influencers Effective engagement mechanisms Policy Instruments Strategic importance of the Reform rationale project Link to policy delivery Accountability of the project delivery Alignment of objectives Interdependencies with other Government supported objectives Alignment with regional/national strategy Performance of the agency Financial viability Availability and sources of funding Budget management Organizational Arrangements Execution complexity Extent of change Project management/expertise Need for innovative/untested technology 36 INSTITUTIONAL DEVELOPMENT ROADMAP HANDBOOK Capability Assessment Self-assessment Expert Governance arrangements Objective statement Formal framework M&E Human resources Adequate staffing Aligned policies Training plan Institutional framework Asset management policy Asset management strategy Financing methods Asset management Lifecycle planning approach Quality information Network planning Business practice and Delivery capabilities implementation procedure Key functions Procedures/core processes and systems  37 38 INSTITUTIONAL DEVELOPMENT ROADMAP HANDBOOK Annexes ANNEXES 39 Annex A: Preparation for the Road Agency (RA) institutional and functional review: Questions, information needs and preliminary actions to be taken 1. CONDUCIVENESS OF SOCIOPOLITICAL ENVIRONMENT 1.1. Why was the RA formed; what was the motivation behind it; has the intended outcome been achieved and, if yes, to what extent? 1.2. How consistent are RA’s vision; objective; strategy; policy and legislative framework with the government priorities? If Yes/Not: 1.2.1. List the linkages between RA and the government priorities. 1.2.2. List the disconnect between RA and Government’s priorities and how you intend to achieve the same outcomes. 1.3. Is there a founding legislation of RA? If yes: 1.3.1. What does it say with respect to its mandate, delegations, funding and governance structure? 1.3.2. Provide a copy and or a link. 1.4. Is the current political, economic, and/or social environment conducive for RA’s objectives? If not: 1.4.1. Are there contingencies/plan to manage the risk? 1.5. Are relevant leaders committed to the reform/objectives? 1.5.1. What are the steps/plans that show the ownership and commitment? 1.6. Is there a robust implementation strategy/plan covering risk management, budget and schedule? 1.7. Is there an engagement/partnership with the stakeholders? 1.7.1. What is the relationship with the Local Government Authorities? How often RA meets with them? 1.7.2. What is the relationship/partnership with the Ministries of Transport, Finance, Forest, Health, Education, Police Department, other? 1.7.3. Is there a clear distinction (legislated) between the roles of RA and other stakeholders such as the LGAs, Transport, Finance, Forest, Health, Education, Police, other? If there’s a document establishing this distinction, please share a copy/link. 1.7.4. Is there some form of engagement with Non-Governmental Organizations, Community Based Organizations, etc..? 1.8. What is the differentiated treatment between urban and rural matters and how is that reflected in the organizational structure, functions, approaches, etc? 2. GOVERNANCE/ORGANIZATIONAL ARRANGEMENTS 2.1. Does RA have an annual business plan? If yes: 40 INSTITUTIONAL DEVELOPMENT ROADMAP HANDBOOK 2.1.1. Are annual objectives clearly established? 2.1.2. Performance indicators for the agency? Are this by units and/or personnel? If yes: 2.1.3. Does it include definitions, methodology for data collection, target values and guidance on measures to take if targets are met/not met. 2.1.4. Does it have indicators of progress? 2.1.5. Does RA have a system (informal or formal) to periodically report the progress of its work against the objectives? 2.1.6. Does it include activities with budget, timeline and responsible personnel or unit? What is the constraints, gaps to perform successfully? 2.1.7. Does it differentiate between rural and urban? 3. FINANCIAL VIABILITY AND PROBITY 3.1. Does RA have the funds to sustain its operating costs? 3.2. Does RA issue annual income and expenditure reports? 3.3. Are RA’s funds allocated transparently, efficiently and strategically? 3.3.1. Are there publications and reports where funds allocations are published? 3.4. How the maintenance and development of roads subordinated to RA organized administratively and financially and are there written regulations for these processes? 4. HR INFORMATION 4.1. Number and titles of positions (total and filled by units, offices, etc.); differentiating those from HQ, regional offices and district offices 4.2. Functional responsibilities for all positions; 4.3. Number of people in each position. Please include the personnel available at HQ, Regional and District level on key roles such as Road Safety Specialists, Communications Officers, Gender Specialists, Environmental and Social Safeguards, Procurement and FM Units. Does RA have HR management system in place: policy/manual/ guidelines in place? If yes, please provide copies and or link. 4.4. What are the challenges, constraints, gaps? 4.3.1. Salary and other cash and in-kind payments structure by position/location; 4.3.2. HR plans and projections, if any; 4.3.3. Performance evaluation rules and arrangements; 4.3.4. Salary evaluation rules. 4.3.5. Staff development programs (trainings, capacity building, etc.) 4.3.6. Turnover rate 5. ICT AND DATA MANAGEMENT: 5.1. Number of computers that belong to RA. Working condition and capability to access internet  41 5.2. Internet accessibility in all RA offices and personnel locations. 5.3. Any information on computer literacy of the staff, if any. For example: excel, GIS platforms, etc. 6. COMMUNICATION STRATEGY: 6.1. Does RA have a communications strategy? If yes, please provide a copy and /or link. Is there a differentiation between the treatment of urban vs. rural? 6.2. What are the current communication goals and objectives? 6.3. How many people are in communications function? What are their duties and responsibilities? 6.4. Is there an annual communications budget? 6.5. Has RA done a communications audit? 7. PROCUREMENT AND CONTRACT MANAGEMENT. 7.1. What is the annual budget of RA? Please share numbers for HQ, Regional and District Offices 7.2. What is the volume of procurement of TARURA? Likewise, share information from HQ, Regional and District Offices. 7.3. Share the average number of contracts that HQ, Regional and District offices process annually 7.4. Does RA have Contract Supervision or Contract Management manual/guideline? 7.4.1. If yes, please share a copy with us. 7.4.2. If no? If Yes, what is the plan? 7.5. Do you have enough working and documents storage space for the Project Implementation Unit/Project Management Unit (PIU/PMU) and Delegated PMU? 7.5.1. If not, what are the challenges? How are you intending to overcome these challenges? 7.6. Contract Management – what are the challenges RA is facing in terms of 7.6.1. Resources: quality checking instruments/tools, supervision vehicles, laboratories etc) and human capacity (number of engineers/technicians, knowledge of contract management, etc). 7.6.2. What are the plans to overcome these challenges? 7.7. Does RA has a Tender Board in HQ, and Regional? If yes, please provide their name, position, Degree, Years of Experience and experience with donor financed projects. 7.8. Provide name, position, Degree, Years of Experience and experience with donor financed projects of Procurement Management Units and Internal Audit Units, Technical Departments, in HQ, Regional and District level. 7.9. How many complains, RA received per year (or last year). 7.9.1. How many of there have been referred to the Public Procurement Appeal Authority? 7.9.2. What are the source of those complains? [type of contracts] 8. FINANCIAL MANAGEMENT: 42 INSTITUTIONAL DEVELOPMENT ROADMAP HANDBOOK 8.1. Does RA have a Financial, budget, accounting strategy/policy/guidelines etc? 8.1.1. If yes, is it consistent with National legislative framework? Please share a copy and/or link 8.1.2. If not, what are the gaps and what is RA’s plan for consistency? 8.2. What are RA’s sources of revenue? 8.2.1. Please state funding institution, amount and if earmarked. What is RA revenue growth strategy? 8.3. Describe the budgeting process. Share any documentation relevant to this. 8.4. What are the Financial Management/Accounting systems used? 8.5. How's the Internal Audit Committee composed and what are its functions and structure? 8.6. Staffing and reporting lines of the Finance and Internal Audit units including levels, experience and qualifications (both professional and academic) 9. ENVIRONMENT AND SOCIAL 9.1. Does RA have an Environmental and Social Policy? If yes, please share a copy 9.2. Does RA have an Occupational, Health and Safety policy? If yes, please share a copy 9.3. Number of Environmental and Social specialists in RA (HQ, Regional and District Offices) 9.3.1. Do they have the required experience and expertise? Are they sociologists or anthropologists for example? [The WB team will follow up with tables to fill] 9.4. Current projects supervised (HQ, Regional and District Offices) 9.5. How are the environmental and social risks and impacts monitored? 9.5.1. How are the results of monitoring and supervision reviewed and taken into account? 9.6. Grievance Redress Mechanism, share a copy please 9.6.1. To what extent there is an understanding and appreciation of RA of the need to engage in meaningful dialog with stakeholders on an informed and on-going basis?    43 Annex B: Contextual Assessment for the Identification of Pre-Project Activity Understanding the Assessing the Context Required Pre-project Planning Monitoring Context Activity Review the contextual Asses the context by answering the contextual What pre-project activity Describe how the pre-project activity will be conducted. How will factors and why increased considerations related to each of the factors. Where is required to support the Note: it is not expected that all challenges can be addressed you know pre-project activity may be challenges to the achivement of the development development goal? through pre-project activity. Focus should be on managing risk when project required to manage strategic goal raised and identify the potential cause of the where possible. preparation is risk to the development goal. challenge. sufficient? Factor Contextual Considerations Y/N Suggested Actions Agreed Actions Who When Activity Status Are relevant leaders are commited to the ❑❑ Support from an established reform/ development goal? regulatory agency is required meet the Supplementary Considerations: development goal. • Is there evidence of bureaucratic ❑❑ A Charter/ memorandum support for the development goal of understanding between (required legislations, expressed public Government/ Project support)? Leaders, the Implementing • Is commitment reflected in investment Agency(s) and key priorities? stakeholders is needed. Stability of the Environment • Is support protected against political ❑❑ Other: Socio-Political Environment change? If the delivery environment is unstable (there is a high likelihood of change and low Has there been robust analysis of the ❑❑ Scenario Planning needs to confidence in th Implementing potential impact of the external (socio- be conducted. Agency's capacity to adapt political) environment? ❑❑ Contingencies should be to that change) then pre- Supplementary Considerations: allocated. project activity to stablise the • Have appropriate risk management ❑❑ Phased investment and/or environment or manage risk been conducted? delivery is recommended. are required. ❑❑ Other: Has the necessary delegated ❑❑ Formal delegated authority management responsibility/ autonomy is required (e.g. Project has been granted. Development Agreement). Supplementary Considerations: ❑❑ Other: • Does the Implementing Agency have owership over its policies, strategic and structure? 44 INSTITUTIONAL DEVELOPMENT ROADMAP HANDBOOK Understanding the Assessing the Context Required Pre-project Planning Monitoring Context Activity Review the contextual Asses the context by answering the contextual What pre-project activity Describe how the pre-project activity will be conducted. How will factors and why increased considerations related to each of the factors. Where is required to support the Note: it is not expected that all challenges can be addressed you know pre-project activity may be challenges to the achivement of the development development goal? through pre-project activity. Focus should be on managing risk when project required to manage strategic goal raised and identify the potential cause of the where possible. preparation is risk to the development goal. challenge. sufficient? Factor Contextual Considerations Y/N Suggested Actions Agreed Actions Who When Activity Status Are there effective engagement ❑❑ Mechanisms for stakeholder mechanisms? engagement need improvement. Supplementary Considerations: ❑❑ Increased time/resource • Are there regular, formal engagement for pre-project activity (e.g. and reporting channels? land acquisition) is required. Socio-Political Environment • Does the Implementing Agency have ❑❑ Formal partnerships with Stakeholders and Influencers experience working with the project key stakeholders are stakeholders and/or in the targeted A significant number of needed. communities? stakeholders with high levels ❑❑ Other: of influence expose the • Are stakeholders likely to change? development goal to risk from Is the process for defining and ❑❑ Policy instruments should differring expectations. achieving the development goal clearly be reviewed. communicated and transparent? ❑❑ Mechanisms for stakeholder Supplementary Considerations: communication are required. • Is data used to support decision- making and lead stakeholders? ❑❑ Other: • Is relevant information shared appropriately?  45 Understanding the Assessing the Context Required Pre-project Planning Monitoring Context Activity Review the contextual Asses the context by answering the contextual What pre-project activity Describe how the pre-project activity will be conducted. How will factors and why increased considerations related to each of the factors. Where is required to support the Note: it is not expected that all challenges can be addressed you know pre-project activity may be challenges to the achivement of the development development goal? through pre-project activity. Focus should be on managing risk when project required to manage strategic goal raised and identify the potential cause of the where possible. preparation is risk to the development goal. challenge. sufficient? Factor Contextual Considerations Y/N Suggested Actions Agreed Actions Who When Activity Status Is the project critical to the delivery of ❑❑ A review and update of policy and key strategic objectives? legislation/policy for its ability to accommodate Supplementary Considerations: revision is needed. • Have challenges to delivery have been ❑❑ An effective policy and identified at the system level (policy, legislative framework is legal and regulatory)? required. • What relevant official decisions have ❑❑ An Institutional Strategic Importance of the been made (public record)? Development Study should Project be completed. Policy Instruments ❑❑ Other: High expectations of the project outcomes require strong project sponsorship Are project leaders accountable for the ❑❑ A Governance and (government commitment and delivery of project outcomes? Accountability Action Plan strategic level ownership of (GAAP) is needed. Supplementary Considerations: the business case). Ambitious ❑❑ Other: projects much have clear • Is there evidence of agreed, realistic accountability at a senior level. project outcomes and representative targets? Does the Implementing Agency does ❑❑ Assitance with policy have a good understanding of the translation/strategy strategic reasons for the reform? development is critical. Supplementary Considerations: ❑❑ Other: • Has the risk of the introduction of new policy/strategy been investigated? 46 INSTITUTIONAL DEVELOPMENT ROADMAP HANDBOOK Understanding the Assessing the Context Required Pre-project Planning Monitoring Context Activity Review the contextual Asses the context by answering the contextual What pre-project activity Describe how the pre-project activity will be conducted. How will factors and why increased considerations related to each of the factors. Where is required to support the Note: it is not expected that all challenges can be addressed you know pre-project activity may be challenges to the achivement of the development development goal? through pre-project activity. Focus should be on managing risk when project required to manage strategic goal raised and identify the potential cause of the where possible. preparation is risk to the development goal. challenge. sufficient? Factor Contextual Considerations Y/N Suggested Actions Agreed Actions Who When Activity Status Are there multiple funding organisations ❑❑ Enhanced engagement and and/or responsible Implementing coordination with funders Agencies? and other responsible agencies is required (clear Supplementary Considerations: lines of decision-making, • Have the institutional challenges been partnerships). identified and agreed? ❑❑ Sector policy, institutional • Are there clear and jointly owned and legal frameworks need sector development targets (national, development. state and/or locel level)? ❑❑ More information is needed around the development needs. ❑❑ Other: Policy Instruments Alignment of Objectives Misaligned objectives reduce Are the proposed project development ❑❑ The business model needs the likelihood of the project objectives sufficiently aligned with to be considered for its contributing to sustained national and state policy and strategies? relevance to wider policies/ development. strategies Supplementary Considerations: ❑❑ Other: • Can the proposed objectives/ development goal be clearly linked to sector-level planning and Government investment priorities? Are other Government supported ❑❑ Joint working should be objectives mutually reinforcing of the leveraged. development goal? ❑❑ Other: Supplementary Considerations: • Is there evidence that a solution exists? • Has the solution been tried in other states, countries?  47 Understanding the Assessing the Context Required Pre-project Planning Monitoring Context Activity Review the contextual Asses the context by answering the contextual What pre-project activity Describe how the pre-project activity will be conducted. How will factors and why increased considerations related to each of the factors. Where is required to support the Note: it is not expected that all challenges can be addressed you know pre-project activity may be challenges to the achivement of the development development goal? through pre-project activity. Focus should be on managing risk when project required to manage strategic goal raised and identify the potential cause of the where possible. preparation is risk to the development goal. challenge. sufficient? Factor Contextual Considerations Y/N Suggested Actions Agreed Actions Who When Activity Status Can the development goal be reasonably ❑❑ Data collection for delivered under the proposed funding investment decision making regime? is needed. Supplementary Considerations: ❑❑ Financial modelling/ planning is required. • Is there a robust business case for investment? ❑❑ Other: Financial Viability • Is there a comprehensive budget If the proposed project is of supported by robust financial significant value, the funding modelling? arrangements for the project are complex or the project is Does the project represent a highly ❑❑ There is a need to put plans expected to deliver significant involved type/source of finance and/or in place to secure additional value for money more effort multiple donor contributions? finance if required. should be put into helping the Supplementary Considerations: ❑❑ Support for assurance of Implementing Agency prepare Policy Instruments estimates and expenditure a robust business case. • To what extent does the Implementing needed. Agency have control of the budget? ❑❑ Other: • Is there adequate availability of counterpart funding from the borrower? • To what extent are decision makers accountable for the use of financial resources? Does the project / programme span many ❑❑ A joint governing board is boundaries (organizational, political and required. Interfaces and Relationships regional)? ❑❑ Other: If project success is dependent Supplementary Considerations: on factors mainly outside of • Are governance arrangements overly the control of the primary complex or lacking? Implementing Agency then consideration should be given • Are there potentially conflicting to how power will be delegated policies? and relationships managed. • Is there confience in the Primary Implementing Agency's ability to lead partners through the proposed reform? 48 INSTITUTIONAL DEVELOPMENT ROADMAP HANDBOOK Understanding the Assessing the Context Required Pre-project Planning Monitoring Context Activity Review the contextual Asses the context by answering the contextual What pre-project activity Describe how the pre-project activity will be conducted. How will factors and why increased considerations related to each of the factors. Where is required to support the Note: it is not expected that all challenges can be addressed you know pre-project activity may be challenges to the achivement of the development development goal? through pre-project activity. Focus should be on managing risk when project required to manage strategic goal raised and identify the potential cause of the where possible. preparation is risk to the development goal. challenge. sufficient? Factor Contextual Considerations Y/N Suggested Actions Agreed Actions Who When Activity Status Is there a robust implementation ❑❑ Enhanced project strategy/plan covering risk management, management capability budget and schedule? (PMO) needs to be developed/acquired. Supplementary Considerations: ❑❑ Partnering for delivery • Is there a wide project scope or Organizational Arrangements should be considered (the challenging objectives that will only be Execution Complexity use of partners where achieveable through external support? the timeframe/appetite is Challenging objectives in insufficient for in-house short timeframes should be development). investigated for the capacity of the Implementing Agency ❑❑ Other: to accept or implement the changes. Does the proposed project involve ❑❑ Phased implementation, new or untested business practices or piloting and/or significant technologies? training is required. Supplementary Considerations: ❑❑ Other: • Have the use of bespoke solutions been sufficient interrogated for value for money? Is the development goal dependent on ❑❑ A sector-wide monitoring other initiatives for success? framework is required. Supplementary Considerations: ❑❑ Other: Dependencies • What activities on the critical path are If the development goal is not under the control of the Primary Policy Instruments dependent on other initiatives Implementing Agency? or a significant amount of • Does planning sufficiently consider the institutional development then timing of other initiatives? consideration should be given to how the proposed project Is the development goal dependent ❑❑ Increased pre-project can leverage joint-working with on a significant amount of institutional activity to plan ID. other projects. development? ❑❑ Joint-working with other Supplementary Considerations: projects should be considered. • Does the level of pre-project activity reflect the scale of the ID? ❑❑ Other:  49 Understanding the Assessing the Context Required Pre-project Planning Monitoring Context Activity Review the contextual Asses the context by answering the contextual What pre-project activity Describe how the pre-project activity will be conducted. How will factors and why increased considerations related to each of the factors. Where is required to support the Note: it is not expected that all challenges can be addressed you know pre-project activity may be challenges to the achivement of the development development goal? through pre-project activity. Focus should be on managing risk when project required to manage strategic goal raised and identify the potential cause of the where possible. preparation is risk to the development goal. challenge. sufficient? Factor Contextual Considerations Y/N Suggested Actions Agreed Actions Who When Activity Status Extent of Change Can the existing Implementing Agency ❑❑ Development of an realise the development goal? Organizational Design The amount of transformational and Development/Human change required to support Supplementary Considerations: Resource Strategy is new ways of working (e.g. new • Is a fundamental change to the required. delivery models) should be Implementing Agency (significant evaluated. If the magnitude ❑❑ Partnering for delivery change in the way the primary delivery of the change is significant, should be considered. agency conducts its work) required in a it will need to be managed short timeframe? ❑❑ Increased pre-project as a transformational change activity to plan ID. Organizational Arrangements programme, thus providing • Is there a represenative organizational progressive development development/human resource strategy ❑❑ Other: through a series of tranches. in place? Is the development/ use of a large ❑❑ Market research/ number of specialist disciplines and skills engagement is required. required? ❑❑ Partnering for delivery Range of Disciplines and Supplementary Considerations: should be considered. Skills Required to Achieve the Development Goal • Have the main implementing agencies ❑❑ Pre-project market shaping and their functions been mapped out? is needed. If there is potential for strain on the supply chain then the • Has the market been sufficiently ❑❑ Increased pre-project resource needs to support investigated? activity to plan ID. project delivery should be • Has the balance between insourcing ❑❑ Other: addressed during pre-project and outsourcing resource been activity. investigated? • Has an appropriate delivery model been selected? 50 INSTITUTIONAL DEVELOPMENT ROADMAP HANDBOOK Understanding the Assessing the Context Required Pre-project Planning Monitoring Context Activity Review the contextual Asses the context by answering the contextual What pre-project activity Describe how the pre-project activity will be conducted. How will factors and why increased considerations related to each of the factors. Where is required to support the Note: it is not expected that all challenges can be addressed you know pre-project activity may be challenges to the achivement of the development development goal? through pre-project activity. Focus should be on managing risk when project required to manage strategic goal raised and identify the potential cause of the where possible. preparation is risk to the development goal. challenge. sufficient? Factor Contextual Considerations Y/N Suggested Actions Agreed Actions Who When Activity Status Implementing Agency Has the Implementing Agency had any ❑❑ Partnerships for facilitating/ Performance to Date issues meeting objectives under similar implementing the arrangements? development should be If the Implementing Agency has considered. not demonstrated the capability Supplementary Considerations: to manage implementation of ❑❑ Synergies with past • Is there sufficient leadership and senoir (national or local) strategies, solutions should be level experience? plans and programmes of investigated. the same scale or type then • Has the solution been successful in Organizational Arrangements ❑❑ Other: consideration should be given other similar environments? as to whether the required development is appropriate for the organization. Is there a detailed Implementation Plan ❑❑ An Organizational that covers how policy, culture, working Development technical Sustainability of Intervention practices, technology, people, processes expert should be engaged Understanding how all parts and procedures work together for for ID design. of an organization need to be sustainable development? ❑❑ Other: developed is key to long term Supplementary Considerations: institutional strengthening. Strategic risk to the project • Is the development goal concentrated can emerge if elements but expected outcomes are broad? are developed as discrete • Is there a clear line of sight between components. policy, expected outcomes and project activity?  51 Annex C: Capability Assessment for ID Design Core Capability Area Assessing Good Practice Required Enablers of Institutional Development ID Components Monitoring & Evaluation Institutional Development Review the core capability Consider the evidence of good practice related What institutional What activities are required to enable the institutional How can the How will you Describe how areas and reflect on to each of the core capability. Where good development development (people, systems and process) identified know when the the ID works the the importance for the practice is lacking or questionable, describe the is required to activities gaps have been development achievement of the potential gap. Use the prompts provided. support the be sensibly closed (consider goal development goal development packaged? the good practice goal? statements)? Good Practice Evidence of Good Is there a potential gap? Gaps to be People Systems Process Workstreams ID Intermediate ID PDO Practice Closed Outcomes The project has a • Is there a legal clear statement of mandate? the objectives from Project Leaders. Project Governance Arrangements Effective project governance aligns projects to the overall Policy Instruments organisational/development The project/ • Is a new governance goals, ensures that the Impelementing framework needed project fulfils its purpose Agency has a (system for delegation, and is effective, efficient and formal governance process for decision- transparent. framework that sets making/change out decision-making control)? Good project governance and delegation of clearly sets out decision- authority. making responsibilities against an agreed statement of There is a robust • Do reporting and objectives providing the project monitoring and accountability team the autonomy to deliver. evaluation system. mechanisms need to be revised? • Is there coordinated assurance? 52 INSTITUTIONAL DEVELOPMENT ROADMAP HANDBOOK Core Capability Area Assessing Good Practice Required Enablers of Institutional Development ID Components Monitoring & Evaluation Institutional Development Review the core capability Consider the evidence of good practice related What institutional What activities are required to enable the institutional How can the How will you Describe how areas and reflect on to each of the core capability. Where good development development (people, systems and process) identified know when the the ID works the the importance for the practice is lacking or questionable, describe the is required to activities gaps have been development achievement of the potential gap. Use the prompts provided. support the be sensibly closed (consider goal development goal development packaged? the good practice goal? statements)? Good Practice Evidence of Good Is there a potential gap? Gaps to be People Systems Process Workstreams ID Intermediate ID PDO Practice Closed Outcomes Published Asset • Are new policies Management Policy required? that adequately addresses the development goal. The asset • Can existing strategies Institutional Frameworks for investment strategy adequately address Policy Instruments Development (funding and the development There is demonstrable resource) is aligned goal? Implementing Agency to the development commitment to the goal. development goal, the required institutional development There are integrated • Are new strategies and future opterations and strategies linked to consistent/compatible maintenance. current capability with existing ones? and resource. There are • Does development appropriate goal does require new pre-conditions methods of financing to support the or contracting? development goal.  53 Core Capability Area Assessing Good Practice Required Enablers of Institutional Development ID Components Monitoring & Evaluation Institutional Development Review the core capability Consider the evidence of good practice related What institutional What activities are required to enable the institutional How can the How will you Describe how areas and reflect on to each of the core capability. Where good development development (people, systems and process) identified know when the the ID works the the importance for the practice is lacking or questionable, describe the is required to activities gaps have been development achievement of the potential gap. Use the prompts provided. support the be sensibly closed (consider goal development goal development packaged? the good practice goal? statements)? Good Practice Evidence of Good Is there a potential gap? Gaps to be People Systems Process Workstreams ID Intermediate ID PDO Practice Closed Outcomes There is a 'whole • Are new approaches life' approach to to lifeycle planning, asset management. programmed maintenance and environmental management required? There is good • Is there outcome quality asset performance information monitoring? to support Asset Management Approach • Is customer decision-making. engagement, asset Asset management is the condition surveys, provision and maintence of asset performance services to customers. Good Organisational Arrangements monitoring required? asset management utilises a system of assets to achieve an • Does the outcome. Implementing Agency have the information This is founded on a robust, to challenge specialist formalised approach to asset requirements? amangement that includes outcome performance There is good • Are there the measurement linked to quality analysis of required information customer expectations. Good asset information. management systems? asset management practice • Is the implementing is evidenced by informed agency focused on planning for capital delivery, policy translation network improvement and and risk-based maintenance operations. prioritisation? • Is outcome perforemance monitoring linked to customer customer expectations? There is adequate • Is there an operational planning for network readiness plan? Are improvement there funds to sustain and maintenance operating costs? 54 operations. INSTITUTIONAL DEVELOPMENT ROADMAP HANDBOOK Core Capability Area Assessing Good Practice Required Enablers of Institutional Development ID Components Monitoring & Evaluation Institutional Development Review the core capability Consider the evidence of good practice related What institutional What activities are required to enable the institutional How can the How will you Describe how areas and reflect on to each of the core capability. Where good development development (people, systems and process) identified know when the the ID works the the importance for the practice is lacking or questionable, describe the is required to activities gaps have been development achievement of the potential gap. Use the prompts provided. support the be sensibly closed (consider goal development goal development packaged? the good practice goal? statements)? Good Practice Evidence of Good Is there a potential gap? Gaps to be People Systems Process Workstreams ID Intermediate ID PDO Practice Closed Outcomes The Implementing • Does the Agency has strong Implementing project delivery Agency have the capability. capability to plan and supervise/coordinate activities (financial management, procurement capability)? • Have project management and risk management arrangements been Organisational Arrangements considered? • Is there a defined Business Practices and set of activities Implentation Procedures accompanied by a budget, timeline, and Application of modern industry/ assigned personnel? business practices is critical to sustainable development. • Has the distribution of in-house/ external resource been considered? Key functions • Is there adequate to support the staffing with technical development goal and administrative have been identified skills to meet the and resourced. development goal? • Is there a plan for resourcing key functions? • Has the distribution of in-house/ external resource been considered?  55 Core Capability Area Assessing Good Practice Required Enablers of Institutional Development ID Components Monitoring & Evaluation Institutional Development Review the core capability Consider the evidence of good practice related What institutional What activities are required to enable the institutional How can the How will you Describe how areas and reflect on to each of the core capability. Where good development development (people, systems and process) identified know when the the ID works the the importance for the practice is lacking or questionable, describe the is required to activities gaps have been development achievement of the potential gap. Use the prompts provided. support the be sensibly closed (consider goal development goal development packaged? the good practice goal? statements)? Good Practice Evidence of Good Is there a potential gap? Gaps to be People Systems Process Workstreams ID Intermediate ID PDO Practice Closed Outcomes The Implementing • Have line Agency has strong management Implementation arrangements for procedures. implementation sbeen considered? • Are there robust procedures for land acquisition, social safeguards? • Is there a plan for Organisational Arrangements staff development and training? • Does the Implementing Agency (Continue) Business Practices have the expertise to and Implentation Procedures objectively challenge cost estimates? The core processes • Are there adequate and systems meet systems and current and future procedures for: asset management Procurement, and project delivery Financial needs. Management, Contract Management, Road Safety Management, Environmental Management, Social Safeguards? • Are there existing solutions that could be leveraged? 56 INSTITUTIONAL DEVELOPMENT ROADMAP HANDBOOK Core Capability Area Assessing Good Practice Required Enablers of Institutional Development ID Components Monitoring & Evaluation Institutional Development Review the core capability Consider the evidence of good practice related What institutional What activities are required to enable the institutional How can the How will you Describe how areas and reflect on to each of the core capability. Where good development development (people, systems and process) identified know when the the ID works the the importance for the practice is lacking or questionable, describe the is required to activities gaps have been development achievement of the potential gap. Use the prompts provided. support the be sensibly closed (consider goal development goal development packaged? the good practice goal? statements)? Good Practice Evidence of Good Is there a potential gap? Gaps to be People Systems Process Workstreams ID Intermediate ID PDO Practice Closed Outcomes The market related • How will supplier to the achievement quality be ensured? of the development • Have appropriate goal is well measurement, metrics understood. and targets been established? Organisational Arrangements • Is there a plan for achieving increased value from the competition process? (Continue) Business Practices • Is there a well defined and Implentation Procedures packaging strategy? How will the market be incentivised? The implementing • Have appropriate Agency is commited measurement, metrics to continuous and targets been improvement established? and sustained • Will international development. suppliers be leveraged to build capacity?  57 58 INSTITUTIONAL DEVELOPMENT ROADMAP HANDBOOK Annex D- Pilot application of the Institutional Development Roadmap  59 Project Information Project Name Rajasthan State Highways Development Program (RSHDP) Date April 2018 Country / State India TTL Mesfin Wodajo Jijo Sector Roads Expert Radia Benamghar Project Development Objective (PDO) as per concept note The PDO to provide safe, reliable and informed journeys for road users on priority corridors, and strengthen institutional capacity to develop, fund, manage and operate the strategic road network of Rajasthan. Key Results as per concept note i. Increased road user satisfaction (gender disaggregated data) on priority corridors (develop parameters and measure as %) ii. Reduction in average roughness of the strategic network (IRI) iii. Setting up and making the Highway administration functional (% KPIs for HA achieved) iv. Development and use of Strategic asset management approach (percentage of projects selected and delivered) v. Increased private financing in the road sector in Rajasthan ($ or % raised from private sources) Concept Description The Bank adopts a two pronged strategy to support the State of Rajasthan through the proposed project RSHDP II: (i) to efficiently manage the state’s strategic road network, through building modern highway authority equipped with modern management systems and capability to raise sufficient road financing from own revenues and the commercial market. This support will have a strategic longer term impact and; (ii) support investment needs for improving priority highway corridors included in the Rajasthan State Highways Development Program (RSHDP) – a program launched in 2014 to develop and sustainably manage about 20,000 km of state road network. Financing and Instrument TBD Data sources Interviews Confidential 60 INSTITUTIONAL DEVELOPMENT ROADMAP HANDBOOK ANNEX  D- PILOT APPLICATION OF THE INSTITUTIONAL DEVELOPMENT ROADMAP 61 Step 1 - Contextual Assessment Understanding the Assessing the Context Scoring the Context Planning Context Activities Review the contextual factors Assess the context by answering the Assess the context by evaluating the contextual situation against typical Self Expert Typical activity Comments and Notes and why increased pre-project contextual considerations related to each standards and inform scoring Assessment to support the activity may be required to of the factors. Where challenges to the development goal manage strategic risk to the achievement of the development goal are development goal. raised and identify the potential cause of the challenge. Factors Guiding questions Level 1 Level 2 Level 3 Examples of Suggested Actions There are significant Some elements of good There is an outcome focus organizational challenges. practice are being utilized. and maximum value is Practices, processes and Systems and processes achieved from the asset systems are outdated or are appropriate for the and external relationships immature. current requirement but may not be sustainable Stability of the Environment Leadership • Are relevant ❑❑ Ownership of the ❑❑ Adequate legal ❑❑ Legal frameworks 2 1 ❑❑ Support to Act was passed. Commitment leaders commited project is weak frameworks are in and investment establish Initiated by the The environment stability has to the reform/ and legislation has place and aligned plans are clearly regulatory agency governement. an impact on the Implementing development goal? not been enacted with investments articulated in a is required to meet Many meeting and Agency’s capacity to deliver or effectively policies. regional/national the development engagements. and will influence how activities • Is there evidence approved through development goal Commitments in and risks will be managed. of bureaucratic ❑❑ no implementation Socio-Political Environment appropriate plan and a public speeches. Too long The likelihood of change and support for the ❑❑ A Charter/ channels. monitoring not reach the point for confidence in the Implementing development memorandum of mechnism is in implementation. They Agency’s capacity to adapt to that goal (required ❑❑ There is no understanding place. are committed but don’t change will inform pre-project legislations, investment plan or between have the time. Financial activity needed to mitigate expressed public it is disconnected Government/ aspects are not there. the risks and stabilise the support)? to the project Project Leaders, Leadership will play environment. objectives    the Implementing • Is commitment a critical role -self Agency(s) and key reflected in motivation stakeholders is investment needed. priorities? • Is there support against political change? 62 INSTITUTIONAL DEVELOPMENT ROADMAP HANDBOOK Understanding the Assessing the Context Scoring the Context Planning Context Activities (Continue) Stability of the Conducive • Have there been ❑❑ Local / national ❑❑ local / national ❑❑ Local/national 2 2 ❑❑ Robust analysis Elections will accelerate Environment political, economic, any recent political, political events political events political events of the potential elections will have and/or social economic, or are negatively are disturbing are conducive to impact of the no impact. Elections environment social issues that impacting or activities of public important reforms political, economic makes things move until have impacted slowing the pace and private sector in sector or in and/or social elections. the general of reforms. general environment and ❑❑ some social issues governance scenario planning. ❑❑ no or limited are negatively ❑❑ regular citizen environment? citizen consultation impacting regular consultations ❑❑ A Charter/ • Are there any / social events activities support reforms memorandum of political, economic, are negatively understanding ❑❑ economic ❑❑ Stable and Socio-Political Environment or social issues that impacting reforms between environment is growing economic have impacted or in the sector Government/ fluctuating environment disrupted planning Project Leaders, ❑❑ important econmic and service the IA(s) and key crisis delivery in the stakeholders. transport sector? ❑❑ Allocation of • What are the key contingencies contextual factors to manage the that have impacted political risk. meaningful ❑❑ Phased investment reforms in the and/or delivery is sector (including recommended. community, state [for federal ❑❑ A reform roadmap systems], national, is needed regional, and to address global factors)? governance challenges. ANNEX D- PILOT APPLICATION OF THE INSTITUTIONAL DEVELOPMENT ROADMAP 63 Understanding the Assessing the Context Scoring the Context Planning Context Activities (Continue) Stability of the Delegated • Has the necessary ❑❑ There is no formal ❑❑ The IA has a ❑❑ The IA leadership 1 1 ❑❑ Formal delegated Environment Authority delegated delegation of leadership team team makes authority management authority and/ appointed and strategic decisions is required responsibility/ or the delegation stable with and can adapt (e.g. Project autonomy been is too limited to sufficient authority direction of the Development granted. ensure smooth to ensure regular works to the needs Agreement). operations. day-to-day for growth of the • Does the ❑❑ Corporate operations are agency. Implementing ❑❑ There is evidence governance effective. Agency have of high turnover in ❑❑ Transparent, analysis ownership over its project leadership ❑❑ Limited but efficient and ❑❑ citizen policies, strategic linked to new regular political effective engagement and structure?What governemental interference decision making activities activities on the appointments. in day-to-day mechanisms Socio-Political Environment critical path are not management of are regularly ❑❑ Management ❑❑ signiifcant political under the control the IA monitored and training interferance of the Primary reported against in day-to-day ❑❑ Operaitonal ❑❑ Accounatbility Implementing management of decisions are charter/framework Agency? the IA implemented in a • Does planning timely manner and sufficiently consider communicated the timing of other to citizens / initiatives?• What stakeholders are the formal and informal channels for decision making? • In addition to the IA, what other government entities have decision-making authorities that would impact project activities? 64 INSTITUTIONAL DEVELOPMENT ROADMAP HANDBOOK Understanding the Assessing the Context Scoring the Context Planning Context Activities Stakeholders and Influencers Stakeholders • Who are ❑❑ Key stakeholders ❑❑ There is a clear ❑❑ Influential 2 1 ❑❑ Stakeholder There is not much analysis the relevant have not been identification of stakholders engagement communication outside. A significant number of stakeholders identified and key stakeholders demonstrate open analysis and Within PWD people are stakeholders with high levels of involved in shaping actions can have and an commitment and startegy to be put very much aware about influence expose the development the development negative impact engagement plan seek to constantly in place it. Staff are impacted goal to riskd from differring objectives?Are / cause delay in place. bring consensus in their work program. expectations; and may have ❑❑ Formal stakeholders likely to actions or with others to Chief Secretary and grounds to object to the operation, ❑❑ Differences in partnerships with to change? programs. achieve results Chief Minister. No implementation or objective, interests can key stakeholders communication. Trust is that may affect its successful • Are stakeholders ❑❑ Conflicts and be addressed ❑❑ There are clear are needed. the main issue because completion by delaying or halting likely to change? Is difference of however incentives in ❑❑ Communication Bankers are not its implementation. Stakeholders there a consensus interests amomgst incentives are not place aligned strategy investing. could include civil society, private among these stakeholders aligned to support with development sector organizations, labor unions, stakeholders the development goals and formal ❑❑ Citizen ❑❑ Competing governments of other countries, and influencers goals. resolution engagement agencies other donors and other members regarding the mechanisms exists approach and tools ❑❑ roles and of the general public importance of to solve issues. to be implemented responsibilities the development of agencies are ❑❑ Roles and objectives? Socio-Political Environment established but responsibilities of • If not, what are the unclear or not agencies are clear, differences and communicated communicated why? appropriately and regularly monitored by the • Can these public differences be addressed or are they irreconcilable? • What are the key actors and/ or stakeholders that are outside of the control of the primary IA? • What are the incentives (if any) required for the actors and/ or stakeholders to productively partner with the primary IA to successfully implement project activities? ANNEX D- PILOT APPLICATION OF THE INSTITUTIONAL DEVELOPMENT ROADMAP 65 Understanding the Assessing the Context Scoring the Context Planning Context Activities (Continue) Stakeholders and Effective • Is the process for ❑❑ Engagements and ❑❑ There is a lack ❑❑ There is an 1 1 ❑❑ Policy instruments Suggestion: monthly Influencers engagement defining the project communications of experience engagement and review bulletin from the dept. mechanisms objectives clearly are not existant in managing communication head Not ncessary to ❑❑ Mechanisms communicated and or not initiated by networks of strategy supporting communicate more. The for stakeholder transparent? the IA. stakeholders the development Bank mission refreshed communication are and no formal goals. their minds at the PS • Are there regular, ❑❑ There is no required. communication level but not at Chief formal engagement experience/ ❑❑ There are regularl and engagement ❑❑ A joint governing Minister because he and reporting authority in the feedback loops in strategy has been board is required. is new. Inside PWD channels? IA in managing place to ensure defined. there are meetings and stakeholders continuous ❑❑ Clarification • Does the IA/ informal channels. No engagements ❑❑ Actions are ad hoc engagment and of roles and management have real communication. and reactive. improvement responsibilities of experience working Need for engagement. of the IA’ the IA and other with the project ❑❑ there is no relevant entities Socio-Political Environment performance stakeholders and/ communication required. or in the targeted dept. communities? • Does IA have adequate human resource capacity to effectively engage key stakeholders? • Is data used to support decision- making and lead stakeholders? • Does the IA have systems in place for citizen engagement? • Is relevant information shared appropriately? 66 INSTITUTIONAL DEVELOPMENT ROADMAP HANDBOOK Understanding the Assessing the Context Scoring the Context Planning Context Activities Strategic Importance of the Reform rationale • Does the ❑❑ The objectives of ❑❑ The reform is ❑❑ The IA is a 3 2 ❑❑ Assistance with Additional finance Project Implementing the reform are not integrated in the key elements policy translation/ sources will help the Agency does linked to the IA’s IA’s work program. driving and strategy overall road strategy. High expectations of the project have a good work program. communicating the development is There is a high need for outcomes require strong project ❑❑ There is reluctance understanding Reform through critical. this authority because sponsorship (government ❑❑ There is a /significant delays of the strategic formal and public there will be more commitment and strategic level lack of trust / in designing or ❑❑ Change reasons for the mechanisms. autonomy. ownership of the business understanding of implementing management reform? case). Ambitious projects much the Reform agenda new policies ❑❑ The risks program to be have clear accountability at a • Has the risk of amongst the IA’s or frameworks are regularly developped to senior level.The project must be the introduction leadership supporting the monitored support the project aligned with national/regional of new policy/ new strategy. and there is ❑❑ Institutional development objectives and strategy been demonstration of assessment of investigated? Are mitigation plans different models there potentially in place and/or for network conflicting policies? communicated. management • Is there confidence Policy Instruments in the primary IA’s ability to lead partners through the proposed reform? Link to policy • Is the project ❑❑ The link between ❑❑ Delays in policy ❑❑ There is 3 1 ❑❑ A review and delivery critical to the project outcomes implementation evidence that the update of delivery of policy and policy is have minor impact implementation of legislation/policy and key strategic limited. on delivery of the the policy has an for its ability to objectives? works. impact and that accommodate ❑❑ There is a formal systems are revision is needed. • Have challenges regulatory vacum ❑❑ Challenges are in place to ensure to delivery been to support the observed however ❑❑ An effective policy effective delivery. identified at the reform addressed in a and legislative system level passive manner. framework is (policy, legal and required. regulatory)? ❑❑ An Institutional Development Study should be completed. ANNEX D- PILOT APPLICATION OF THE INSTITUTIONAL DEVELOPMENT ROADMAP 67 Understanding the Assessing the Context Scoring the Context Planning Context Activities (Continue) Strategic Importance Accountability of • Have the ❑❑ There is a high ❑❑ Results of ❑❑ There are 1 1 ❑❑ Formalize IA and/ A lot of political of the Project the project delivery authorities turnover in the team are clear lines of or Leaderhsip interfeference. Once it appointed a senior leadership or monitored accountability appointment and will be established it will leader accountable indatequate level however, there between the TORs take its own course. The for the project? of appointment of is no evidence IA and rewards target is clear. ❑❑ Develop Are project leaders the team and no of a formal and sanction formal and accountable for the consequences nor engagement mechanisms are comprehensive delivery of project rewards system in between the IA in place to support reporting (M&E) outcomes? place. and authorities. the leadership and tools and staff motivation to • Are there ❑❑ outcomes /results ❑❑ outcomes/results mechanisms obtain results. regular reporting of the project are are attributed to ❑❑ Establish archiving mechanism not attributed the proejct / knowledge between the Policy Instruments management project leadership system and authorities? • Is there evidence of agreed, realistic project outcomes and representative targets? What relevant official decisions have been made (public record)? • Is there a clear record of the IA and relevant entities following key strategic objectives? 68 INSTITUTIONAL DEVELOPMENT ROADMAP HANDBOOK Understanding the Assessing the Context Scoring the Context Planning Context Activities Alignment of Objectives Interdependencies • Are there ❑❑ The ❑❑ Relations between ❑❑ The 1 1 ❑❑ Enhanced with other multiple funding implementation different initiatives interdependencies engagement Misaligned objectives reduce Government organisations and/ plan is isolated are estbalished between and coordination the likelihood of the project supported or responsible from other relevant however, there governement with funders and contributing to sustained objectives Implementing governement are dysfunctional agencies and/ other responsible development. If the development Agencies? initiatives. or contradictory or different IA agencies goal is dependent on other approaches/ are defined at is required initiatives or a significant amount • Have the ❑❑ there are policies creative a strategic level (clear lines of of institutional development then institutional duplication adverse impact and resources are decision-making, consideration should be given challenges been of efforts and on the ability of leveraged to the partnerships). to how the proposed project can identified and resources within the IA to operate benefit of results leverage joint-working with other agreed? the sector ❑❑ Sector policy, effectively or for all programs. projects institutional and • Are there clear ❑❑ there is unclarity efficiently. ❑❑ The roles and legal frameworks If project success is dependent and jointly of ownership ❑❑ the various responsibilities need development. on factors mainly outside of owned sector between different Policy Instruments impementation of the differents the control of the primary development entties ❑❑ More information plans are not actors are clear, Implementing Agency then targets (national, is needed around aligned creating accepted and consideration should be given to state and/or local the development confusion or relationships how power will be delegated and level)? What needs. distraction of between the IAs relationships managed. activities on the resources are strengthening ❑❑ Partnerships critical path are not overall results for facilitating/ under the control of and development implementing the primary IA? goals. the development • Does planning should be sufficiently consider considered. the timing of other ❑❑ Synergies with past initiatives? solutions should be • How would investigated you describe ❑❑ Joint working the working should be relationships leveraged. between the IA and other relevant entities? ANNEX D- PILOT APPLICATION OF THE INSTITUTIONAL DEVELOPMENT ROADMAP 69 Understanding the Assessing the Context Scoring the Context Planning Context Activities (Continue) Alignment of Alignment with • Are the ❑❑ ☐The project is ❑❑ There is a ❑❑ The objectives of 2 2 ❑❑ Synergies with Objectives regional / national proposed project not aligned with reasonable the project are regional / national strategy development national / sectoral alignment leveraged by the strategy and objectives strategy and between the national/regional policy to be sufficiently aligned decision making is project’s development plan demonstrated. with national and adversly impacted development ❑❑ The project ❑❑ Joint working state policy and objectives and ❑❑ There is a lack outcomes should be strategies? national/regional of resources influence the leveraged. strategy however, • How important to ensure sector’s planning there is a lack ❑❑ A sector-wide are national and sustainability of the and investment of articulated monitoring state policy and project priorities. prirorities that framework is strategies to sector Policy Instruments impacts or delays required. priorities and decision making. decision-making? ❑❑ Investments are • Are there other de-linked from factors that project obejctives impact sector priorities and decision-making? • Can the proposed objectives/ development goal be clearly linked to sector-level planning and Government investment priorities? 70 INSTITUTIONAL DEVELOPMENT ROADMAP HANDBOOK Understanding the Assessing the Context Scoring the Context Planning Context Activities (Continue) Alignment of Performance of the • Is the development ❑❑ The IA is not ❑❑ The IA is well ❑❑ The IA is a best 2 1 ❑❑ The business best officers are Objectives agency objective operating at managed and demonstrated model needs to be engaged in this project dependent on a expected level of achieves the practice for the considered for its significant amount results. majority of the sector and bey relevance to wider of institutional exepcted results. ond. policies/strategies. ❑❑ There are development? significant gaps ❑❑ The ❑❑ Work practices, ❑❑ Increased pre- • Is there sufficient between outcomes implementation policies and project activity to leadership and and resources plan is updated staffing models plan ID senoir level and aligned with have been ❑❑ There is a lack ❑❑ Performance experience? work practices in replicated to of experienced Agreement the sector/region support other • Has the solution leadership in between the IA / nation projects. been successful managing the IA and Authorities in other similar ❑❑ activities are ❑❑ Business ❑❑ There is environments? regularly processes and/or monitored innovation have • Is there a detailed Policy Instruments and corrective had positive impact Implementation measures are in on results beyond Plan that covers place. the project, in the how policy, culture, sector and/or the working practices, ❑❑ quallified and region/national technology, recognized level people, processes leaders are in and procedures place in the IA work together for sustainable development? • Is the development objective concentrated but expected outcomes are broad? • Is there a clear line of sight between policy, expected outcomes and project activity? ANNEX D- PILOT APPLICATION OF THE INSTITUTIONAL DEVELOPMENT ROADMAP 71 Understanding the Assessing the Context Scoring the Context Planning Context Activities Financial Viability Availability and • Is there a robust ❑❑ The project is ❑❑ The IA is ❑❑ The project 2 2 ❑❑ Data collection sources of Funding business case for underfunded or sufficiently activities are for investment If the proposed project is of investment? resources have funded to operate strategically decision making is significant value, the funding been misallocated within budget funded through needed. arrangements for the project are • Does the project requirements. multiple sources complex or the project is expected represent ❑❑ Counterpart’s ❑❑ Financial that leverage each to deliver significant value for highly involved budget has been ❑❑ Activities are modelling/planning other money more effort should be put type/source of revised with an well provided is required. into helping the Implementing finance and/or adverse impact on for and there is ❑❑ There are formal ❑❑ There is a need Agency prepare a robust business multiple donor the project a transparent and public to put plans in case. contributions? budget in place collaboration ❑❑ Funding of the place to secure mechanisms in • Is there adequate project requires ❑❑ budget control additional finance place providing availability of significant revision function is in place if required. transparent counterpart funding of funding internally and monitoring and ❑❑ Support for from the borrower? externally control from the assurance of • To what extent are public estimates and decision makers expenditure accountable for needed. the use of financial Policy Instruments resources? Budget • Is there a ❑❑ The proposed ❑❑ The IA has the ❑❑ The strategic 3 2 ❑❑ Pre-project Management comprehensive budget is not means to be planning includes financial modelling budget supported aligned with accountable a long term, multi including by robust financial project activities for financial years budget consultancy modelling? management. aligned with costs and change ❑❑ The IA has regional / national management • To what extent limited control ❑❑ Complex financial plans / OD activities. does the IA over budget and management and must be made for have control expenditures budgeting skills ❑❑ Streamlined subsequent of the budget? and experience planning and ❑❑ Financial What are the are avaialble in the budgeting ❑❑ Allowances for management specific financial team processes are in contingencies skills need to be management place should be strenghtened ❑❑ Highly fragmented strengths and considered institutional ❑❑ Businses weaknesses of arrangements improvement ❑❑ FM Capacity the IA and other processes Assessment relevant entities? are regularly • How does the implemented Implementing Entity derive its authority to manage the proposed funding regime? 72 INSTITUTIONAL DEVELOPMENT ROADMAP HANDBOOK Understanding the Assessing the Context Scoring the Context Planning Context Activities Execution Complexity Extend of Change • Is a fundamental ❑❑ Reactive behavior ❑❑ there is evidence ❑❑ changes and 1 1 ❑❑ Development of change to the IA to change ; of a reasonable innovation are an Organizational Challenging objectives in (significant change appetite for anticipated and Design strategy short timeframes should be ❑❑ undefined in the way the change in the IA planned for investigated for the capacity of the leadership roles for ❑❑ Partnering for primary delivery Implementing Agency to accept or driving the change ❑❑ change had no or ❑❑ there is a positive delivery agency conducts its implement the changes. limited impact on track record work) required in a ❑❑ lack of service ❑❑ Increased pre- staff turnover of innovation short timeframe? orientation project activity to The amount of transformational implementation ❑❑ change had plan ID. Pre-project change required to support new • Has the solution ❑❑ lack incentives to a recognized ❑❑ change has funds to be ways of working (e.g. new delivery been tried in other perform positive impact on created recognized used as finance models) should be evaluated. states, countries? ❑❑ there is no service delivery oportunities for for strategic If the magnitude of the change • Is there a evidence/track the IA consultancy for is significant, it will need to be ❑❑ there is a public representative record of effective developing the managed as a transformational support from ❑❑ leadership is organizational reforms non construction change programme, thus leadership for acknowledge as development/ elements of providing progressive ❑❑ there are limited change a champion for human resource the programme Organizational Arrangements development through a series of staff allocated change strategy in place? (development of tranches. to research and an organisational • Does the IA have innovation design and the capacity and ❑❑ change has development will to follow created high plan and through on these turnover in implementation reforms? qualified staff plan). • Are there sufficient / qualified human resources to ensure implementation • Does the IA have the authority to make the operational changes required for successful project implementation? • Is the organizational culture of the IA conducive to change? ANNEX D- PILOT APPLICATION OF THE INSTITUTIONAL DEVELOPMENT ROADMAP 73 Understanding the Assessing the Context Scoring the Context Planning Context Activities (Continue) Execution Complexity Project • Is there a robust ❑❑ there are no ❑❑ systems are in ❑❑ Strong systems are 1 2 ❑❑ Enhanced project Management/ implementation information place however in place to plan, management Expertise strategy/plan systems in place. do not operate implement and capability (PMO) covering risk effectively monitor projects needs to be ❑❑ project management, developed/ management is ❑❑ the delivery unit ❑❑ there is a strong budget and acquired. outsourced is under staff or and capable schedule? under qualified deliivery unit in ❑❑ Market research/ ❑❑ there are no clear • Has the use of place leading engagement is req accountability ❑❑ project bespoke solutions multiple complex uired. for project management been sufficiently projects in a management team are formed ❑❑ Capacity interrogated for programatic however in ad hoc Assessment is value for money? ❑❑ archiving system is manner manner / need required. outdated and not • Does the IA and basis ❑❑ information up to date other related systems are entities have connected and a documented best practices track record of constantly implementing reviewed and challenging integrated in new projects in short projects time frames? ❑❑ project Organizational Arrangements • Does the IA have management a database of team reports to projects with key leadership data captured for review and analysis? • How is institutional knowledge about past and current projects stored and used by the IA? • Is there a wide project scope or challenging objectives that will only be achieveable through external support?Is the development or use of a large number of specialist disciplines and skills required? Has the balance between insourcing and outsourcing resource been investigated? 74 INSTITUTIONAL DEVELOPMENT ROADMAP HANDBOOK Understanding the Assessing the Context Scoring the Context Planning Context Activities (Continue) Execution Complexity Need for • Does the proposed ❑❑ lack of use ❑❑ slow and/or ad hoc ❑❑ New technologies 1 1 innovative/ project involve of innovative implementation of are easily ❑❑ phased untested new or untested technology new technology embedded in implementation of technology: business practices current business ❑❑ weak ❑❑ staff has limited new technology or technologies? practices with implementation of access to positive impact on ❑❑ enhanced budget Organizational Arrangements • Has the market current business innovation service delivery and staff training been sufficiently practices ❑❑ rigid budget allocated to investigated? ❑❑ best practices are ❑❑ no budget and decision innovation regularly analyzed • Has the IA allocation to making processes ❑❑ knowledge flexibly respond innovation and supporting ❑❑ staff have allocated exchange activities to unexpected technology innovation time for research discipline and skill and innovation ❑❑ review/introduce requirements? activities research and development • Does the IA ❑❑ fungible business policies/practices have the budget and business flexibility required practices to respond to unexpected requirements? ANNEX D- PILOT APPLICATION OF THE INSTITUTIONAL DEVELOPMENT ROADMAP 75 Step 2 - Institutional Capability Assessment Core Capability Area Assessing Practice Scoring Planning Comments Activities Review the core capability Consider the evidence of good practice Assess the context by evaluating the contextual situation against typical Self Expert Typical activity Comments areas and reflect on related to each of the core capability. Where standards and inform scoring Assessment to support the the importance for the good practice is lacking or questionable, development goal achievement of the describe the potential gap. Use the prompts development goal. provided. Description Key Areas Guiding questions Level 1 Level 2 Level 3 Examples of Suggested Actions There are significant Some elements of There is an outcome organizational good practice are focus and maximum challenges. Practices, being utilized. Systems value is achieved from processes and systems and processes are the asset and external are outdated or appropriate for the relationships immature. current requirement but may not be sustainable Project Governance Objective • Does the ❑❑ Lack of a clear ❑❑ The project/IA has ❑❑ Adequate legal 3 2 ❑❑ Strategy • Clear objectives Arrangements Statement: goverment have a legal framework a clear statement framework and development and no other clear vision ? of objectives from clear vision Advisory services responsibilities. Effective project governance The Government ❑❑ Government lacks Project Leaders that support aligns projects to the overall has a clear vision • Does the legal a clear vision ❑❑ Legal advisory but implementation government organisational/development for its development framework support services Policy Instruments is weak development goals, ensures that the objectives development objectives project fulfils its purpose objectives ? and is effective, efficient and • Do governance transparent. arrangements Good project governance supportthe project’ clearly sets out decision- s development making responsibilities objectives? against an agreed statement of objectives providing the project team the autonomy to deliver. 76 INSTITUTIONAL DEVELOPMENT ROADMAP HANDBOOK Core Capability Area Assessing Practice Scoring Planning Comments Activities (Continue) Project Formal Framework: • Are the ❑❑ Unclear/ineffective ❑❑ The project/ IA ❑❑ Governance 2 1 ❑❑ Corporate Governance Arrangements currentgovernance governance has a formal arrangements governance The project/ arrangements/ arrangements. governance provide clear assessment Impelementing framework framework accountability Agency has a ❑❑ Organizational effective? (system that sets out to sponsoring formal governance design for delegation, decision-making organization. framework that sets intervention process for and delegation of out decision-making decision-making/ authority. ❑❑ Accountability and delegation of change control)? framework authority for project preparation and ❑❑ Advise and implementation. coaching of Board members Policy Instruments ❑❑ Capacity assessment for Senior Management M&E Framework: • Are there systems ❑❑ A lack of devolved ❑❑ There is a ❑❑ There is a robust 2 2 ❑❑ Develop M&E • Too much in place for data responsibility functioning M&E system which framework administration There is a robust collection? and diffuse monitoring and is able to establish when submitting monitoring and ❑❑ Develop Key accountability. evaluation system appropriate targets the forms for evaluation system. • Do reporting and Performance but inefficiencies for success. reimbursement accountability Indicators exist. mechanisms need ❑❑ Training to be revised? • Is there coordinated assurance? Human Resources Staffing: • Is there a staffing/ ❑❑ Weak/no link ❑❑ Established ❑❑ There are clear 1 1 ❑❑ Develop strategic • Vacancies will be recruitment between link between and complete staffing plans fulfilled Strategic Human Resources The project is policy based on recruitments and recruitment and strategic staffing management are aligned adequatly staffed ❑❑ Institional • There is enough competency and business needs ☐ business needs plan in place to the overall project (quantity and quality) development manpower. Need business needs in recruitment gaps but weak/no and aligned with objectives and subsequent assessment to hire youth aging Organisational Arrangements place? transparency in the organisation incl. development goals. It population and process job descriptions ❑❑ Competency gap supports proffessional, • Are staffing to respond to and business assessment effective and transparent processes the extend of the needs management practices that transparent? works. The gap will enforce accountability in Junior Engineer and commitment of leaders and Assistant to change. Engineers. Aging of the population Good HR policies and creates less pactices create a positive availability of HR. work ethics and help management attract, develop and motivate staff to achieve results and deliver qualitative services to the public. ANNEX D- PILOT APPLICATION OF THE INSTITUTIONAL DEVELOPMENT ROADMAP 77 Core Capability Area Assessing Practice Scoring Planning Comments Activities Description Key Areas Guiding questions Level 1 Level 2 Level 3 Examples of Suggested Actions (Continue) Human Policies: • Are procedures ❑❑ Weak ❑❑ R policies are ❑❑ Full transparency in 1 1 ❑❑ Functional review • There is no Resources and policies implementation implemented but the implementation autonomous The project/ ❑❑ Legal framework communicated of staffing Management lacks of policies and authority to recruit. Impelementing advisory services and applied in a policies and no accountability on procedures and The confusion has Agency has a set of consistent manner? accountability in HR decisions and Management is ❑❑ Management been resolved and procedures, policies HR decisions and strategies accountable to HR training the recruitment and practices that • How is this tracked strategies decisions will be able to supports results and and recorded? ❑❑ Professionalization proceed recognition of results. of HR personnel • Is management Staffing decisions • Organization set accountable for ❑❑ Creation / are made based up needs to be good HR decisions modernization of on objective and filled at 80% of the and strategies? HR unit transparent criteria. time all the time. Organisational Arrangements Lack of incentives to join the new entity even if it is a presitgious project. Link between field engineer and college. there is a considerable budget for. Need for a human resources department to help engineers. Need for design wing in PWD on innovation to generate revenue that could be redistributed to the engineer who developped as an incentive scheme. 78 INSTITUTIONAL DEVELOPMENT ROADMAP HANDBOOK Core Capability Area Assessing Practice Scoring Planning Comments Activities Description Key Areas Guiding questions Level 1 Level 2 Level 3 Examples of Suggested Actions (Continue) Human Training: • Is the Training ❑❑ Ad hoc/no training ❑❑ Training plans exist ❑❑ Training plans are 2 1 ❑❑ Competency • Training cell within Resources plan/policy plan for staff but implementation fully funded and based learning PWD design There is a communicated to is weak or monitored/Full analysis and delivey competency based staff individually? inadequate funding transparency in the of training + training plan in place. ❑❑ Training systems implementation of external agencies It is adequately • Are staff and procedures training plans and calendars budgeted and competencies ❑❑ Budget review are kept. PWD funded, regularly measured as an has budget for monitored and outcome of the ❑❑ Learning center training. There updated. training plan? ❑❑ Establish a is no culture learning function of learning in ❑❑ Knowledge PWD. Technical exchange training is needed. Professional expertise is not followed. Interest of the organization. Association Organisational Arrangements of Engineers. Weekly seminars technologies and innovation in collaboration with Chief Engineers - external organization funds their technology. International knowledge exchanges are happening. Environment and Social Procedures and • Are there robust ❑❑ Weak to no ❑❑ Social and ❑❑ Implementation 2 1 ❑❑ Social and • No direct processes: procedures for land implementation environmental of Social and environment interventions but Assessments conducted acquisition, social of social and policies are in Environmental assessment going through will help identify gaps in The implementing safeguards? environmental place and being policies and political channels policies and procedures to Agency has a set of ❑❑ Social and safeguades implemented but procedures are determine related risks and procedures, policies • Is there a environment unit • There is limited policies and processes are fully transparent impacts and an framework in comprehensive influence by PWD. procedures cumbersome and and there ❑❑ Training for staff place that supports operations manual But it depends inefficient is a sound and management land acquisition, that is regularly on political will accountability to ressetlemnts etc.. updated? ❑❑ Communication and decision. Not users strategy / plan enough training • Are there robust for contractors. procedures and ❑❑ Citizen No need for framework for engagement supervisor staff. environmental activities Implementaion safeguards? aspects are weak ANNEX D- PILOT APPLICATION OF THE INSTITUTIONAL DEVELOPMENT ROADMAP 79 Core Capability Area Assessing Practice Scoring Planning Comments Activities Description Key Areas Guiding questions Level 1 Level 2 Level 3 Examples of Suggested Actions (Continue) Environment Labor: • Are labor laws ❑❑ weak ❑❑ Labor laws ❑❑ Implementation 2 1 ❑❑ Legal advisory • not much work has and Social implemented in a implementation of implementation of labor laws are services been done since The IA fully transparent and labor laws and lack processes are advanced and 2013 understands labor ❑❑ Labor inspection fairly? of inclusion of local in place and fully transparent laws and implements • Not much training communities local inclusion to contractors ❑❑ Citizen them transparently • Are employment and awareness of processes are and community engagement processes the new proceudre weak members actiivties (contracts etc..) in beyond PPP the communities ❑❑ Communication division where roads are and engagement • Need to hire being built inclusive activities specialized skills of community ❑❑ Townhalls level of awareness members? for engineers and ❑❑ Revision of labor • In the event of deep specialised and security labor influx to a skils for policies construction site implementation how does the IA ❑❑ Security training • Post has been engage with the and awareness created but not community? campaigns hired. Road Safety: • Are there sufficient ❑❑ Insufficient ❑❑ Institutional ❑❑ Instutional 1 1 ❑❑ Establish function • Training on road resources (human resources arrangements arrangements and safety have been The IA has adequate ❑❑ Full road security Organisational Arrangements and financial) to allocation to is in place but internal processes developped resources (human assessment manage roads manage reporting inefficiency in function optimally and next year and financial) to safety reporting? implementation to improve Road ❑❑ Workshops and additional batches address road safety ❑❑ Weak to exist safety reporting seminars of people are reporting issues • Is there a process fragmented and traffic proposed for as well as manage to report safety arrangement on ❑❑ Engagement enforcement training. Road traffic enforcement issues? Is it Road safet and campaign and safety is not a big effective? traffic reporting stakeholders issue because reviews • Have line it depends on management behaviors of the arrangements public. There considered any has been a lot of improvements? development in the past 4 to 5 years especially in raising awareness. IN PWD there are trained staff • Provision of consultant but there is no recruitment • No awareness -> huge change management • Need ot have more training to the conscessioners. No follow up after design stage. 80 INSTITUTIONAL DEVELOPMENT ROADMAP HANDBOOK Core Capability Area Assessing Practice Scoring Planning Comments Activities Description Key Areas Guiding questions Level 1 Level 2 Level 3 Examples of Suggested Actions Asset Management Lifecycle planning: • Are new ❑❑ Poor decision ❑❑ There isfair quality ❑❑ Asset management 1 1 • lack of Approach approaches to making asset information approach leads Commitment to There is a ‘whole life’ lifecycle planning, undermines asset to support to overall value change Asset management is the approach to asset programmed management. decision-making. for money and provision and maintence of management. maintenance and efficient service services to customers. Good environmental delivery asset management utilises a management system of assets to achieve required? an outcome. This is founded on a robust, formalised approach to asset amangement Quality of • Is there outcome ❑❑ No appropriate ❑❑ Thereis a fair ❑❑ There is an 1 1 • Every now and that includes outcome information: performance asset information quality analysis of understanding of then there is a performance measurement monitoring? asset information.. and application circular but there There is good quality linked to customer of whole life cost is no monitoring. asset information • Is customer expectations. Good asset and environment to support engagement, asset • No self discipline management practice is management decision-making. condition surveys, in following evidenced by informed principles. asset performance through planning for capital delivery, monitoring Organisational Arrangements network improvement and required? maintenance operations. • Does the IA have the information to challenge specialist requirements?Are the required information management systems established? • Is the IA focused on policy translation and risk-based prioritization? Is outcome performance monitoring linked to customer expectations? Network planning: • Is there an ❑❑ Limited ❑❑ There is a ❑❑ Strategic se of 1 1 operational application of fair planning data. There is adequate readiness plan? modern planning for network planning for network for network improvement improvement • Are there funds to improvement and maintenance and maintenance sustain operating and maintenance operations operations costs? operations ANNEX D- PILOT APPLICATION OF THE INSTITUTIONAL DEVELOPMENT ROADMAP 81 Core Capability Area Assessing Practice Scoring Planning Comments Activities Description Key Areas Guiding questions Level 1 Level 2 Level 3 Examples of Suggested Actions Business and Fiduciary Fiduciary • Does the IA have ❑❑ Processes, systems ❑❑ Processes and ❑❑ Effectively 2 2 • IMFS Systems are Processes and Practices Capabilities (FM and the capability to and technology systems are balances in-house developped and Procurement): plan and supervise/ do not support fairly in place but and external budget allocations. Application of modern coordinate effective delivery delivery capability resource Procurement industry/business practices The Implementing activities (financial remains weak through is critical to sustainable Agency has strong ❑❑ Absence of ❑❑ The IA has strong management, e-procurement development. project delivery the required ❑❑ Limited staff project delivery procurement capability. specializations and specialization capability • Fulyntransparent capability)? High staff turnover. and cumbersome and website. • Have project external resources There will be a management and availability need for training risk management with the right level arrangements been of incentives. considered? • Is there a defined set of activities accompanied by a budget, timeline, and assigned personnel? • Has the distribution Organisational Arrangements of in-house/ external resource been considered? Key functions and • Are there adequate ❑❑ No or incomplete ❑❑ Processes and ❑❑ Application of 2 2 Systems: systems and Operational systems (including modern industry/ procedures for: manual is in place Operations business practices To support the Procurement, or implementation Manual) have been development goal ❑❑ The core Financial is not up to date/ established but have been identified processes and Management, effective. implementation is and resourced. systems meet Contract weak The Implementing current and future Management, Agency has strong ❑❑ Limited staff project delivery Road Safety Implementation capacity and no needs Management, procedures. improvement from Environmental and Management has Social Safeguards? been considered • Are there existing solutions that could be leveraged? Have line management arrangements for implementation been considered? Is there a comprehensive operations manual that is regularly updated? 82 INSTITUTIONAL DEVELOPMENT ROADMAP HANDBOOK Core Capability Area Assessing Practice Scoring Planning Comments Activities Description Key Areas Guiding questions Level 1 Level 2 Level 3 Examples of Suggested Actions (Continue) Business and Market • How will supplier ❑❑ Limited ❑❑ The market ❑❑ Clear 2 2 Fiduciary Processes and understanding: quality be ensured? understanding related to the understanding Practices of actual market achievement of of the market The market related • Have appropriate capacity, capability the development and rules and to the achievement measurement, or appetite. objective is procedures of of the development metrics and targets Organisational Arrangements understood but engagement objectives is well been established? engagment with understood. ❑❑ Leverages • Is there a plan the market is weak international for achieving suppliers for increased continuous value from the improvement. competition process? • Is there a well- defined packaging strategy? How will the market be incentivized? ANNEX D- PILOT APPLICATION OF THE INSTITUTIONAL DEVELOPMENT ROADMAP 83 Step 4 - IDRT Scoring Tool Card Contextual Assessment Self-Assessment Expert Socio-Political Stability of the Environment Leadership Commitment 2 1 Conducive political, economic, and/or social 2 2 environment Delegated Authority 1 1 Stakeholders and influencers Stakeholders analysis 2 1 Effective engagement mechanisms 1 1 Policy Instruments Strategic Importance of the Reform rationale 3 2 Project Link to policy delivery 3 1 Accountability of the project delivery 1 1 Alignment of Objectives Interdependencies with other Government 1 1 supported objectives Alignment with regional / national strategy 2 2 Performance of the agency 2 1 Financial viability Availability and sources of Funding 2 2 Budget Management 3 2 Organizational Arrangements Execution Complexity Extend of Change 1 1 Project Management/Expertise 1 2 Need for innovative/ untested technology: 1 1 84 INSTITUTIONAL DEVELOPMENT ROADMAP HANDBOOK Capability Assessment Self-Assessment Assessors Governance Arrangements Objective Statement 3 2 Policy Inst. Formal Framework 2 1 M&E 2 2 Human Resources Adequate Staffing 1 1 Aligned Policies 1 1 Training Plan 2 1 Environment and Social Process and procedures 2 1 Labor 2 1 Organisational Arrangements Road Safety 1 1 Asset Management Approach Lifecycle planning 1 1 Quality Information 1 1 Network planning 1 1 Business Practice and Fiduciary capabilities 0 0 Implementation Procedure Key Functions and systems 0 0 Market understanding 2 2  85 Step 4 - Monitoring and Evaluation Project Name Rajasthan State Highways Development Program (RSHDP) Date April 2018 Country / State India TTL Mesfin Wodajo Jijo Sector Roads Expert Radia Benamghar NEW Project Development Objective (PDO), including ID The PDO is to build capacity for better management of state highways and to improve traffic flows on selected state highways in the state of Rajasthan. ID Key Results PDO indicator: • RSHA is operational, efficient and effective Suggested intermediate results indicators: • %tage of technical staff trained that are using asset management tools • %tage of staff position filled (technical and management) • Number of citizen engagement activities completed • Number of contracts awarded within […] by the newly formed fiduciary cell • %tage of private sector investment in the State Authority Fund • Availability of the long-term investment plan to the public • Number of RSHA units with published and measured performance plans 86 INSTITUTIONAL DEVELOPMENT ROADMAP HANDBOOK RADIA BENAMGHAR JUNE 2018