69816 EDBM’s Monitoring and Evaluation Framework For Investment Climate Reforms in Madagascar 1. Introduction 1. In its development strategy - the Madagascar Action Plan (MAP), the Government of Madagascar (GoM) aims at raising the country’s economic growth rate between 7% and 10% by 2012. To achieve this growth rate, which is crucial for poverty alleviation, the government commits to have a diversified and strong private sector driven by local and international investment and trade. In particular, to boost the foreign direct investment, the GoM in the MAP sets the target to increase the foreign direct investment (FDI) from US$84 millions in 2005 to US$500 millions in 2012. 2. As is now widely recognized, an investment climate without unnecessary and excessive administrative and regulatory burdens, with transparent rules and regulations, and accountable and well-functioning public institutions, is a key to increase investment and growth. Accordingly, in the MAP the government commits to generate the fundamental conditions to support the facilitation of business. To improve Madagascar’s investment climate, the GoM in the MAP sets the target to improve the ranking of the ease of doing business indicator from 149th in the Doing Business Report, 2007 to 80 by 2012, assuming all else equal (i.e., measured against the global 2006 ranking). 3. To improve investment climate and thus to increase investment and to achieve the target of private sector led high growth economy by 2012, the GoM created the Economic Development Board of Madagascar (EDBM), a privately managed structure, under the Office of the Presidency in 2006. The EDBM is a single and autonomous body with a private-public Board and plays a role of investment promotion and business facilitation. The EDBM is funded under the Integrated Growth Poles Project (IGPP) for the first three years. 4. EDBM’s mission is to: • promote Madagascar as a preferred destination for investment; • increase levels of FDI by proactively attracting and securing investments; • provide professional and efficient investment facilitation services to investors; and • create value for investors and actively contribute to the economic development and growth of Madagascar and improvement of its business climate. 5. Accomplishment of such an energized and result-oriented mission requires setting up of a neutral and credible monitoring and evaluation (M&E) system in the EDBM. The EDBM Management has requested Bank support to launch the M&E activity. This report identifies the key M&E indicators to monitor the outcomes and impacts of the EDBM interventions and discusses the next steps to design an M&E system for the EDBM. The proposed M&E indicators and recommendations of this report have been validated with EDBM and selected stakeholders from the private and public sectors (please see Annex 3 for the list of stakeholders consulted). 6. This M&E system of EDBM should: • regularly monitor the overall investment climate in Madagascar, • regularly monitor to what extent EDBM is contributing to the improvement of investment climate, and • evaluate to what extent the improvement of the investment climate is contributing to the relevant goals of MAP in terms of expanded private sector of all sizes (i.e., small, medium, and large enterprises), increased domestic and foreign investment, trade, and growth and thereby increased employment and reduced poverty. 7. The M&E system will also include the EDBM’s performance indicators to provide feedback to EDBM management and the Board on how well EDBM is performing its day-to-day activities. However, these performance indicators for EDBM’s internal purposes are not the focus of this report as these indicators and the targets are part of EDBM’s business plan. Nonetheless, these performance indicators will be integrated into the M&E system once the M&E system is put in place. 8. This report, on the other hand, focuses on the IC indicators that EDBM should monitor because of three reasons. First, these are the indicators that are of primary interest of EDBM’s stakeholders, i.e., the private sector, the GoM, and the World Bank. Second, EDBM is viewed as the right structure in Madagascar to monitor the status of investment climate in Madagascar and to do policy advocacy for investment climate reforms. This policy advocacy should be an integral part of its key roles of investment promotion and business facilitation. Third, demonstrating EDBM’s contribution to these reforms will itself be a promotional activity for EDBM and will create an image of EDBM as a highly effective public-private body in Madagascar. 9. Monitoring reforms tend to generate reforms. This is demonstrated by a number of reform experiences around the world (i.e., the Bulldozer Reform in Bosnia and Herzegovina, the Procurement Law Reform in the Philippines, the Citizen Score Card Initiative in Indian Cities, Inspectorate Reform in Latvia, etc.) and the lessons from the World Bank Group’s Doing Business Studies. Hence, the M&E should be viewed as an integral part of EDBM’s effort to realize its stated mission. It should be put in place in the EDBM from the beginning of its operation. 10. The M&E system, however, should not be used merely as an accounting exercise. Neither should it be in place as a donor requirement to generate reports for the donors. Rather it should serve at least three critical purposes. First, the M&E system should promote the transparency and accountability of the government’s private sector development (PSD) reform process by making the 2 progress and results of the reforms public. EDBM’s mandate is to facilitate the investment climate reforms through policy advocacy. Using appropriate M&E tools, the M&E system should also monitor to what extent EDBM is reaching out to the private sector at large and is contributing to the PSD and investment climate (IC) reforms that are deemed crucial by the private sector. This in turn should also enhance EDBM’s credibility, transparency and accountability to its stakeholders (i.e., the government, private sector and the civil society at large). Second, the M&E system should serve as a feedback loop and lesson-learning tool to provide constructive feedback to all the concerned stakeholders about where reforms are needed and whether the desired results from the reforms are realized. This in turn should enable the policy makers to better design the reforms. Similarly, monitoring and evaluating how well EDBM is contributing to the reforms and making progress in accomplishing its stated mission should provide appropriate feedback to the EDBM Board and Management on the effectiveness of EDBM’s current strategy and interventions. This in turn should further improve EDBM’s strategy and interventions. Third, through regular dissemination and communication of the key M&E findings, the M&E system should facilitate the reform process by generating public awareness, public support and political will for the reforms and by weakening the illegitimate vested interests against reforms. 11. To assist the EDBM in setting-up such an M&E system, this report: • defines the goals and impacts and objectives and outcomes of the EDBM interventions (see Section 2) • proposes a list of key outcome and impact indicators to monitor the investment climate and its resultant impacts, based on the consultation with the stakeholders from the public and private sectors (see Section 2) • presents a (draft) comprehensive logical framework (logframe) with the above mentioned indicators and their potential sources of verifications (see Annex 1) • presents some of the tools to monitor and evaluate EDBM’s success in contributing to PSD and IC reforms through policy advocacy (see Section 3), and • discusses some of the key next steps in setting-up EDBM’s M&E system (see Section 4). 12. The report is organized as follows. Section 2 discusses the M&E framework for EDBM. Section 3 presents a reform tracking matrix to track the reforms facilitated by EDBM and their intended results. Section 4 concludes the report by highlighting some of the key next steps. A draft logframe is presented in Annex 1. Annex 2 provides the terms of reference (ToR) for EDBM’s M&E system design. Annex 3 provides the list of stakeholders consulted for this report. 3 2. An M&E Framework for Investment Climate Reforms in Madagascar 13. The design and operationalization of EDBM’s M&E framework for IC reforms is essentially the following eight-step process: (i) Define the goals and impacts, objectives and outcomes, outputs, activities, and inputs of EDBM (ii) Identify the objectively verifiable indicators (OVIs) and their sources to measure the IC impacts, outcome, and outputs (iii)Develop an M&E logframe for the EDBM (iv) Obtain the baseline data of the impact and outcome indicators against which progress of the IC reforms should be benchmarked (v) Update the data regularly (vi) Monitor EDBM’s organizational effectiveness and contribution to the IC reforms (vii)Analyse the data and prepare report on to what extent results are achieved against the set targets and EDBM’s contribution towards those achievement, and (viii)Disseminate the results to inform the policy makers and relevant stakeholders 14. This section focuses on steps 1 and 2. Step 3, that is, the logframe for EDBM is presented in Annex 1. Step 6 is discussed in the next section. The rest of the steps are beyond the scope of this short report and subjects of the actual design of the M&E system as discussed in Section 4. Step 1: Define the Goals and Impacts, Objectives and Outcomes, Outputs, Activities and Inputs of EDBM 15. The generic definitions of and the logical connections among the goals and impacts, objectives and outcomes, outputs, activities and inputs are presented in Box 1. These are the essential component of any programme logic framework (henceforth, logframe). Goals and Impacts 16. The goal, as defined in Box 1, is “what is intended to be changed/achieved.� The overall goal of the GoM is to reduce poverty. As is well-known, one of the key Millennium Development Goals (MDGs) as well is to halve the poverty rate by 2015. Poverty, however, is influenced by a number of factors ranging from basic food and nutrition, health 4 and sanitation, education, infrastructure, environment to investment climate. Therefore, to reduce poverty, a number of commitments are made in the MAP1. 17. One of these commitments, i.e., Commitment 6 of the MAP (presented below), is to reduce poverty through creating a better investment climate in Madagascar that in turn will facilitate growth of businesses, increased trade and more investment (both domestic and foreign) domestic and foreign investments: COMMITMENT SIX: HIGH GROWTH ECONOMY Madagascar will have a high growth economy with growth rates reaching between 7% to 10% by 2012. We will ensure that we have a diversified and strong private sector driven by local and international investment and trade. The Government will generate the fundamental conditions that will support the facilitation of business. We will encourage the pursuit of the best ideas and cutting edge techniques, technologies, and strategies to ensure that we as a nation are responsive to the challenges of globalization, productive in the workplace, and gain a competitive advantage. 18. The EDBM is created to facilitate the GoM’s overall goal of poverty alleviation through a private sector led high growth economy (Commitment # 6 in the MAP). Thus the goals of EDBM follow naturally from this commitment and from the overarching goal of the MAP, which is poverty reduction. These goals thus can be listed as follows: • To reduce poverty • To increase economic growth to 7%-10% by 2012 • To increase domestic investment • To increase foreign investment • To increase trade 19. As illustrated in Box 1, impacts are essentially the achievements of the goals. To achieve the above goals, EDBM’s objective or mandate is to improve the IC in Madagascar. 20. A conducive IC is crucial to support enterprise development in general and employment growth in particular. An improvement in economic growth along with more investment and employment should in turn reduce poverty. About half of the firms surveyed in the Madagascar Investment Climate Assessment2 report that foreign exchange regulations, corruption, anti-competitive practices are important constraints to an enabling business environment. 21. Thus the potential impacts resulting from an improved investment climate can be listed as follows: 1 There are eight commitments in the MAP: responsible governance, connected infrastructure, educational transformation, rural development and green revolution, health, family planning, and the fight against HIV/AIDS, high growth economy, cherish the environment, and national solidarity. For further details on the MAP, see http://www.madagascar-presidency.gov.mg/MAP/ 2 Madagascar Investment Climate Assessment, World Bank, June 2005 5 • Growth of registered businesses • Increased domestic investment • Increased foreign investment • Increased employment in the manufacturing and service sector • Reduced informality • Increased exports • Reduced poverty 6 Box 1: The essential components of the Logframe Goal: What is intended to be changed/achieved? Objectives: The intended results to which the project or program will contribute Activities: Actions taken or work performed through which inputs such as funds and technical assistance are mobilized to produce outputs Outputs: Direct results, e.g., products, goods and services resulting from project/program observed immediately or in a short-term period during the project timeframe Outcomes: Anticipated changes in knowledge, behaviour (new practices) and performance of direct and indirect beneficiaries resulting from outputs; these measure the achievements of Objectives. Impacts: Positive or negative, primary or secondary long term effects produced by the project/program; measures the achievement of Goals. To reduce poverty To increase economic growth To increase domestic investment To increase foreign To improve the investment investment Reforms Reduced poverty To increase trade climate recommended Increased economic to improve the Improved growth investment investment Increased trade climate climate Business growth Increased employment Increased domestic investment Increased foreign investment 7 The list of indicators of the EDBM logframe are summarized in the following table following the logical chain illustrated in Box 1. Table 1: EDBM’s Results Indicators Activities Outputs Outcomes Impacts • Desk research EDBM’s Ease of doing business Reduced poverty for preparing recommendations Process/Intermediate Indicators • Poverty Headcount Ratio the for necessary Number/list of reforms undertaken based on EDBM (Global), i.e., percentage of diagnostics, policy/legal recommendations in terms of people below $1/day solution reforms and/or policies/laws/legislations and/or institutions related • Poverty Headcount Ratio design and institutional to: (National), i.e., percentage implementati reforms to • Business Start-up of people below the national on plan for improve various • Licensing poverty line (preferably investment dimensions of • Employment disaggregated by gender) climate investment • Property Registration • Regional Poverty reforms climate (as • Credit Headcount Ratio, i.e., • Field work measured by the • Investor Protection percentage of poor in undertaken outcome • Tax different regions (preferably for preparing indicators) • Trade disaggregated by gender) the • Contract Enforcement diagnostics, solution • Business Exit Reduced Informality Result/Final Outcome Indicators: • Informal sector as a design and At the national level and in three growth poles, ease percentage of GDP implementati on plan for of doing business in terms of reduced time, cost, and • Sales amount reported by a investment processes involved in: typical firm for tax purposes climate • Business Start-up (%) reforms • Licensing • Stakeholder • Employment Increased employment consultations • Property Registration • Number of manufacturing for • Credit and service sector investment • Investor Protection employment (gender climate • Tax disaggregated) national and reforms • Trade in the three growth poles • Public-private • Contract Enforcement • Number of employment in SMEs (gender dialogues on • Business Exit disaggregated) i t t 8 Activities Outputs Outcomes Impacts investment Better bureaucracy disaggregated) climate Process/Intermediate Indicator: • reform issues • Number of relevant government organizations Increased export in priority • Development publishing their respective regulatory policies sectors of Action and procedures in their respective websites • Volume and value of export Plans and based on EDBM recommendation. from tourism, mining and Reform • Number of government organizations with e- agri-business ) national and Strategies for government options based on EDBM in the three growth poles investment recommendations/interventions • Export from tourism, climate • Measures undertaken by EDBM for business mining and agri-business as reforms facilitation a percentage of total export, • Advising the Result/Final Outcome Indicator: national and in the three GoM on • Percentage of senior management time spent in growth poles undertaking dealing with requirements of government necessary regulation; preferably disaggregated by regions Growth of business reforms to • Percentage of firms view that officials’ • Number of new firms improve the interpretations of regulations affecting the firm registered for Business investment are consistent/predictable; preferably • Number of firms filing tax climate in disaggregated by regions returns with the tax Madagascar • Average time firms spent in meetings with tax authority officials (days); preferably disaggregated by regions Growth of SMEs • Number of new SMEs Reduced crime and improved security • Number of SMEs filing tax Process/Intermediate Indicator returns with the tax • Number of measures undertaken by the GoM to authority reduce crime and improve security Result/Final Outcome Indicator: Increased foreign investment • Security costs (% of sales) • Net inflow of FDI; national and in the three growth • Losses due to theft, robbery, vandalism, and poles arson against the firm (% of sales) Increased domestic investment Improved access to finance • Volume of Gross Fixed Process/Intermediate Indicator: Capital Formation (GFCF), • Number of measures undertaken by the GoM to private sector;, national and improve the firms’ access to finance in the three growth poles • Number of measures undertaken by EDBM to • Volume of investment of improve the firms’ access to finance as a part of SMEs, national and in the its business facilitation role three growth poles Result/Final Outcome Indicator: • Percentage of firms view that it is a major Cost-savings of the firms constraint • Aggregate cost-savings of 9 Activities Outputs Outcomes Impacts Better public service delivery to enterprises the firms due to simplified Process/Intermediate Indicator: regulatory regime • Number of measures undertaken by the concerned government organizations to improve the service delivery to the businesses Result/Final Outcome Indicator: ( national level and in the three growth poles) • Delay in obtaining an electrical connection (days) • Number of electrical outages (days) • Value lost due to electrical outages (% of sales) • Number of water supply failures (days) • Delay in obtaining a mainline telephone connection (days) • Firms using the Web in interaction with clients/suppliers (%) Streamlined Custom Process/Intermediate Indicator: • Number of measures undertaken by the Custom Administration and Tax Authority to streamline custom administration and to facilitate trade recommendations/interventions Result/Final Outcome Indicators: • Average time to clear direct exports through customs (days) • Longest time to clear direct exports through customs (days) • Average time to claim imports from customs (days) • Longest time to claim imports from customs (days) • Firms that export directly (%) 10 22. Poverty has a gender and geographical profile and some regions can fall behind in the poverty trap compared to the other regions in the same country. EDBM has priority sectors (i.e., agribusiness, tourism, and internet and communication technology (ICT)) and will be primarily active in the three growth poles3. Therefore, disaggregated data on the following indicators will enable the EDBM and the GoM to monitor some of the key development impacts in the poles and in the priority sectors:: • manufacturing and service sector employment in the poles, gender-disaggregated • employment in the SMEs in the poles, gender-disaggregated • number of SMEs in the poles • export volumes from the priority sectors and from the poles • foreign investment in the priority sectors and in the poles • domestic investment in the priority sectors and in the poles • volume of investment of the SMEs in the poles 23. Moreover, this disaggregation of the impact indicators will fit nicely with EDBM’s effort to improve the regional competitiveness as discussed below. Objective and Outcomes 24. As discussed above, the GoM has created the EDBM to facilitate the government’s overall goal of poverty reduction through a high growth economy by improving the investment climate in Madagascar. The objective, as defined in Box 1, is the intended result to which the project or program is going to contribute. Hence, the key objective of the EDBM is to improve the investment climate in Madagascar, which is crucial to achieve EDBM’s stated mission of increased domestic and foreign investment. 25. The outcomes, as defined in Box 1, measure the achievement of the objectives. Hence EDBM’s outcomes will be measured in terms of improvement in the investment climate in Madagascar (see below the discussions on the indicators to measure various dimensions of investment climate). 26. An improvement in overall investment climate in Madagascar will require improvement in investment climate of different regions within the country. This in turn will contribute to the goal of reducing regional inequality in terms of the above mentioned development impacts. As EDBM is going to be primarily active in the three growth poles, this in turn should contribute to the improvement of the investment climate in the poles. 27. The objective of improving the investment climate at the regional/sub-national level has at least three more merits. The first merit is the intra-regional competition to improve the investment climate and competitiveness. As the experiences from sub- national Doing Business studies in Latin America, Municipal Scorecards in Latin America, and Provincial Competitiveness Index (PCI) in Vietnam show, focusing on the investment climate of different regions in the same country generate enormous intra- 3 These three growth poles are Tolagnaro, Nosy Be, and Antananarivo-Antsirabe. regional competition to improve the investment climate and competitiveness, particularly by the laggard regions. This in turn improves the overall investment climate and competitiveness in the country (see Box 2 on Vietnam PCI). 28. The second merit is the identification of the local best practices as opposed to international best practices. Rather than looking for international best practices which may often be difficult to adopt locally given the local environment and constraints, the focus on sub-national level identifies regional or local best practices, which is much more convincing to the government and policymakers. Highlighting the regional variation in regulatory regimes and investment climate, thus tend to provide further motivation to improve the investment climate in the laggard regions. Box 2 Vietnam Provincial Competitiveness Index Vietnam is one of the pioneering countries in developing sub-national competitiveness. In 2005 the Provincial Competitiveness Index (PCI) was developed by the Vietnam Chamber of Commerce and Industry (VCCI) and the USAID-funded Vietnam Competitiveness Initiative (VNCI). In the first year, the PCI assessed and ranked 42 provincial governments and was subsequently expanded to 64 provinces in response to attention of the government, business community, donors, and the media. The provincial authorities responded to the index with a range of initiatives to improve their business environments. VNCI and VCCI organized diagnostic workshops to outline the key strengths and weaknesses of the respective PCI performance and to discuss recommendations and action plans for reform. A number of important commitments and initiatives resulted from these workshops in various provinces (e.g., new measures to promote private sector consultation and feedback in the policy and planning process, efforts to streamline and rationalize business licensing and land access procedures, improved mechanisms for the dissemination of investor-related information and more coordinated institutional response to improve the treatment of entrepreneurs. The PCI in many ways now has become an integral part of the private sector policy and regulatory reform landscape in Vietnam. It was at the center in the Prime Minister’s address to the business community in January 2006. Provincial leaders use it in preparing for the Provincial Party Congresses. The media focus on PCI when reporting on business environment. Donors such as IFC-MPDF, DANIDA, and GTZ mainstreamed the PCI into their provincial business enabling environment 29. The third merit of the objective of improving the investment climate at the sub- national level is the ability to calculate “investment climate premium� to further motivate the government and policy makers to improve the investment climate and competitiveness of different regions and thus of the whole nation. The “investment climate premium� can be defined as the improvements in living standards through better investment climate, given the same level of endowments. This is further illustrated in Figure 1 below based on the Vietnam PCI. Figure 1 shows the relative benefit from better investment climate measured by the Vietnamese Provincial Competitiveness Index (PCI)4. Provinces with high scores on the PCI have a higher standard of living at every level of structural endowments. Put simply, provinces with better investment climate (i.e. high PCI) are able to generate 4 In the Vietnam Competitiveness Index Report, this is termed as “governance premium�. However, the idea is essentially the same. And similar exercise can be conducted using sub-national Doing Business or other measures of investment climate instead of the PCI. 12 higher living standards from the same level of development. The gap between the two lines can be thought of as an ‘investment climate premium’. Perhaps more importantly, Figure 1 shows that the slope of the high-PCI line is higher than the slope of the low-PCI line, indicating that this ‘investment climate premium’ actually increases as the province develops. This level of analysis motivates the sub-national authorities to adopt best possible practices to improve the investment climate in order that the local population can draw the greatest benefit from economic growth and development. Figure PCI Performance and Economic Welfare 15 2004 GDP per capita (in Millions of Constant 1994 VND) “The Investment Climate Premium� Provinces with better IC (high PCI) are able to generate higher living 10 standards from the same level of development 5 0 0 20 40 60 80 100 Structural Endowments (Infrastructure, Human Capital, Proximity to Markets) Low PCI High PCI Source: Vietnam Competitiveness Index, Summary Report (www.vnci.org) Outputs, Activities, and Inputs 32. Identification and discussion of EDBM’s actual outputs, activities and inputs (notice that these are primarily the performance indicators of EDBM as mentioned in the introduction) are not the subject matter of this report. These should be covered in EDBM’s business and work plan for the next several years. 33. However, the important thing is to ensure that EDBM’s detailed plan of actual activities and outputs (and the resource allocation, i.e., the inputs for these) are aligned with the goals and impacts, and objectives and outcomes as discussed above. Thus generically speaking, EDBM’s outputs should be the key reform recommendations and action plans (coming out of the projects that EDBM will be undertaking by itself and in consultation with the World Bank Group and other partners) to improve different 13 dimensions of the investment climate (see the discussion below on indicators to measure the different dimensions of investment climate) at the regional/sub-national and national level. These recommendations can be broadly disaggregated into two types: • Recommendations and action plans related to legal/policy reforms • Recommendations and action plans related to institutional/organizational reforms 34. Some of the generic activities of EDBM projects to generate the recommendations and actions plans for necessary IC reforms will thus be as follows: • Desk research for preparing the diagnostics, solution design and implementation plan • Field work undertaken for preparing the diagnostics, solution design and implementation plan • Stakeholder consultations • Public-private dialogues on PSD Reform Issues • Development of Action Plans and Reform Strategies • Advising the GoM on undertaking necessary reforms 35. Finally, inputs are going to be the monetary amount of the resources used for different projects to generate the desired outputs. Step 2: Identify the objectively verifiable indicators (OVIs) and their sources to measure the IC impacts, outcome, and outputs 36. Measurable indicators and their corresponding sources need to be identified to measure the above mentioned impacts, outcomes and outputs of EDBM intervention. These indicators should form the basis of EDBM M&E system. Data on these indicators need to be regularly collected and analyzed to understand the result and effectiveness of various IC reforms proposed by the EDBM. Impact Indicators 37. Based on the above mentioned impacts, a proposed list of impact indicators and the corresponding potential sources is as follows: (Intended) Impact Objectively Verifiable Indicator (Potential) Sources of (OVI) Verification (SoV) Reduced poverty • Poverty Headcount Ratio World Development Indicators (Global), i.e., percentage of (WDI) people below $1/day • Poverty Headcount Ratio INSTAT (National), i.e., percentage of people below the national poverty line (preferably disaggregated by gender) 14 (Intended) Impact Objectively Verifiable Indicator (Potential) Sources of (OVI) Verification (SoV) • Madagascar Human INSTAT Development Index Reduced Informality • Informal sector as a Doing Business, World Bank percentage of GDP To be confirmed, INSTAT5 • Sales amount reported by a typical firm for tax purposes ICA/EDBM enterprise survey6 (%) Increased employment • Number of manufacturing INSTAT, PIC7 and service sector employment (gender disaggregated) national and in the three growth poles • Number of employment in SMEs (gender disaggregated) ) national and in the three growth poles Increased export in priority • Volume and value of export PIC, Central Bank, INSTAT, sectors in EDBM priority sectors8 ( EDBM ICT and agri-business ) (national and in the three growth poles ) • Export from ICT and agro- business as a percentage of total export ( national and in the three growth poles ) Growth of business • Number of new firms INSTAT, Tax Authority, EDBM registered for Business • Number of firms filing tax returns with the tax authority Growth of SMEs • Number of new SMEs, INSTAT, EDBM, Tax Authority, national and in the three PIC, SME Solution Center (SME growth poles database) • Number of SMEs filing tax returns with the tax authority, national and in the three growth poles Increased domestic investment • Volume of Gross Fixed INSTAT, Central Bank, EDBM, Capital Formation (GFCF), PIC private sector; national and in the three growth poles 5 If the data is not regularly available from Doing Business study, INSTAT may consider to measure the size of informality in Madagascar 6 Wherever in the logframe ICA is mentioned implies that the data is available in the ICA but the ICA survey is conducted once in every four years. So data on these indicators need to be updated on an annual basis through surveys conducted by EDBM 7 PIC is the French acronym of the Integrated Growth Pole Project (IGPP). 8 As far as EDBM mandate is concerned the priority sectors are agro-business, tourism and ICT 15 (Intended) Impact Objectively Verifiable Indicator (Potential) Sources of (OVI) Verification (SoV) • Volume of GFCF in EDBM priority sectors • Volume of investment of SMEs, national and in the three growth poles Outcome Indicators 38. As the key outcome is an improved investment climate, indicators are required to measure the different dimensions of the investment climate. Thus the outcome indicators can further be classified into two types: • Process/intermediate outcome indicators, which aim to track the necessary legal/policy and/or institutional reforms undertaken to improve certain aspects of investment climate (i.e., business registration, licensing, contract enforcement, etc.) • Result/final outcome indicators, which aim at measuring the result obtained in terms of simplified and less costly business regulations and/or better quality regulations from implementing the necessary reforms 39. The process/intermediate outcome indicators are the ones that are more closely linked with the outputs (i.e., recommendations for relevant legal or institutional reforms forthcoming from the corresponding project on say, business registration or licensing). The data or information for these process indicators will be coming from the concerned line ministries and government sources. 40. It should be noted that many of the above legal/policy and institutional/organization reforms actually refer to the reforms of other institutions in the government. EDBM’s role here is to mobilize these reforms in other institutions, and much of the work on solution design and implementation will be done not exclusively by EDBM but in concert with other ministries and agencies. EDBM will play the role of the main instigator and motivator of reform within a complex government/administrative structure. EDBM’s success in this connection will be tracked by the “reform tracking matrix� as presented in the next section. 41. To measure various dimensions of the investment climate two sets of result/final outcome indicators are proposed. The first set of indicators is the so-called Doing Business Indicators based on the World Bank Group Doing Business Study and Methodology. The basis for the second set of indicators is the Madagascar ICA, 2005. 42. However, the ICA survey is likely to be conducted once in every four years. Moreover, the current methodology of the ICA survey does not adequately cover the firms from the three growth poles. On the other hand, the EDBM needs to monitor these indicators on a regular basis (at least annually). 16 43. During our consultation with the EDBM, it proposes to conduct an enterprise survey on an annual basis, particularly on the enterprises that will be registered with them. This will enable EDBM to monitor the growth and expansion of these enterprises over time and the state of the investment climate. As discussed in Section 4, the design and implementation of this survey as well as issues related to this survey should be given due consideration during the M&E system design for EDBM. 44. The rationale for having Doing Business indicators (based on Doing Business methodology) and the data from the enterprise survey are as follows. Broadly speaking, the Doing Business methodology provides official version of business regulations, i.e., how the regulations supposed to work in the books. However, often there is a huge gap between what is in the book and what actually gets implemented on the ground. To bridge this gap, indicators from the enterprise survey will be helpful as it will enable the EDBM to understand (i) how the regulations work in practice, (ii) how different types of firms given their attributes (such as size, ownership, location and so on) face the burden of regulations, and (iii) what are the underlying dynamics of the political economy of investment climate reforms. Moreover, some important dimensions of investment climate such as the quality of government regulations, access to information on relevant business regulations, corruptions, and so on, which are subject matter of the enterprise survey but are not captured through the Doing Business methodology. 45. However, some developments are needed for both the Doing Business and enterprise survey indicators. Both the Doing Business and enterprise survey needs to be undertaken at the regional/sub-national level, or more specifically in the growth poles to monitor the situations in the poles - the primary focus of EDBM. In the future, this can be further expanded to other regions to boost the healthy inter-regional competition to improve the local investment climate. Moreover, as far as enterprise survey is concerned a panel of firms needs to be established that will be surveyed on an annual basis. This in turn will enable EDBM to clearly establish the link between investment climate, productivity and growth controlling for other observed and unobserved phenomenon. Doing Business Indicators 46. The government in the MAP sets the target to improve the ranking of the ease of doing business indicator from 149th in the Doing Business Report, 2007 to 80 by 2012, assuming all else equal (i.e., measured against the global 2006 ranking). 47. The ease of doing business indicator is a summary indicator based on the time, cost, and processes involved in: • Starting a Business • Dealing with Licenses • Employing Workers • Registering Property • Getting Credit • Protecting Investors 17 • Paying Taxes • Trading Across Borders • Enforcing Contracts • Closing a Business 48. The detailed methodology of each of these ten dimensions of investment climate is available in www.doingbusiness.org 49. To measure the progress against the government’s set target in terms of the ease of doing business ranking, the time, cost, and process indicators of each of the above ten dimensions of investment climate needs to be monitored. 50. Moreover, as already mentioned to monitor the investment climate in the poles, these indicators need to be developed for each of the poles (and later can be developed for other regions) using the sub-national Doing Business methodology available in FIAS and the Doing Business Team in the WBG. Enterprise Survey Indicators 51. The doing business indicators need to be complemented by the enterprise survey indicators, which are currently based on ICA but need to be updated on an annual basis through the proposed EDBM survey or other means (as discussed in Section 4). These indicators will capture how the burden of regulations are faced and the investment climate is perceived by the firms of different sizes, industries, and locations. Some of the key indicators in this respect are thus as follows: (Intended) Outcome Objectively Verifiable Indicator (OVI) Better bureaucracy • Percentage of senior management time spent in dealing with requirements of government regulation; national level and in three growth poles • Percentage of firms view that “circulaires� affecting their business are implemented with no further interpretation from relevant authorities; national level and in three growth poles Reduced corruption • Unofficial payments for typical firm to get things done (% of sales); national level and in three growth poles Well-functioning court system • Percentage of firms having confidence in the court system; national level and in three growth poles • Time spent in resolving disputes in the court (weeks); national level and in three growth poles Better public service delivery to • Delay in obtaining an electrical connection (days); national level enterprises and in three growth poles • Delay in obtaining a mainline telephone connection (days); national level and in three growth poles • Firms using the Web in interaction with clients/suppliers (%);national level and in three growth poles 52. Similar to Doing Business indicators in the poles, data on enterprise survey indicators need to be collected from each of the poles as well, and later the coverage can be extended to other regions. This in turn will help to improve the national investment 18 climate through the improvement of and reduced disparity in regional investment climates. Furthermore, as discussed above a panel of firms needs to be surveyed regularly to rigorously monitor the changes over time and across poles (and other regions in the future) to clearly establish the link between changes in different dimensions of investment climate and productivity and growth. Output Indicators 53. As already mentioned, identification of all the detailed outputs (and the corresponding activities and inputs) are not the subject matter of this report and should be developed as a part of EDBM’s business and work plan. Generically, the output indicators will be the list of reforms recommended by EDBM (coming out of its various projects) on each of the above dimensions of investment climate. The Logical Chain 54. Implementations of EDBM’s outputs (i.e., recommendations on necessary reforms) will result in the corresponding process/intermediate outcomes (i.e., legal/policy and/or institutional reforms to improve the corresponding dimension of the investment climate). This in turn will result in a streamlined regulatory environment measured by reduced time, cost, processes of different regulatory processes, lesser bureaucratic burden and corruption, improved predictability of regulations, and so on. All these outcomes will then contribute to the desired impacts in terms of more investment, jobs, business, trade and growth, and thereby less poverty. Based on this logic and the above mentioned list of indicators a draft logframe for EDBM intervention is presented in Annex 1. 3. Monitoring EDBM’s Contribution to the IC Reform Process 55. So far, EDBM’s monitoring of the investment climate and its resultant impacts have been discussed. This in turn would enable the EDBM to inform the policy makers and the relevant stakeholders on what reforms are working and where more reform efforts are needed to achieve the government’s commitment of a private sector led high growth economy to reduce poverty in Madagascar. 56. In this section, the focus is given on how to monitor EDBM’s contribution to the PSD and IC reforms in Madagascar. As already mentioned above, many of these reforms are actually the reforms of other government agencies and institutions within the government and EDBM’s role is to instigate these reforms. Monitoring EDBM’s contribution to the reform process 57. To routinely monitor EDBM’s contribution to the reform process and the status of the key reforms recommended by EDBM’s projects, a reform tracking matrix (see figure 3) can be developed, among other instruments. Such a matrix is used in the monitoring and evaluation of private sector development in Sierra Leone and is routinely updated by 19 each of the project team to disseminate the status of the current state of the proposed reforms and to flag any key issues/challenges. 58. A reform tracking matrix for each of the EDBM project can be regularly updated (i.e., on a quarterly basis). Information from these matrices can thus inform the EDBM Board, management as well as stakeholders from the public and private sector on the status of the EDBM recommended reforms and where the actions are most needed. 20 # Key Reform Specific Objectives of Target s Key Milestones Current Status Key bottlenecks for Key lessons learned so far Key Potential Com ent m m hy Recom ended the Reform (i.e., w date/tim eline for implementing the as regards im plementing Benefits/Outcom es/Re specifically this reform the Reform Reform the Reform sults of the Reform m has been recom ended) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Figure 2: Reform Tracking Matrix 21 4. Next S teps 59. The key objective of this report is to propose a set of indicators for the EDBM to monitor the investment climate and its resultant development impacts. The next step is to design and operationalize an M&E system for the EDBM to monitor the activities, outputs, and evaluate the outcomes and impacts of EDBM interventions to improve the investment climate and thus to attract more investments to Madagascar. 60. In order for EDBM to design and implement the M&E system, a (draft) detailed terms of reference (ToR) for the M&E system design is developed and attached in Annex 2. As a next step, the EDBM will need to review and finalize this ToR and proceed with the M&E system design work. 61. Some of the key activities/issues involved in the actual M&E system design, as discussed in the ToR, are as follows. First, the logframe, indicators and their corresponding potential sources as presented in this report should be validated. Adequacy of the indicators should be assessed and if required the indicators and/or sources should be modified. Second, a detailed assessment of the data quality of relevant PSD/IC impact and outcome indicators from the national sources (such as INSTAT, Tax Authority, relevant PSD related line ministries/agencies) should be undertaken. The methodology and frequency of data collection and available means of sharing the data with the EDBM (i.e., printed reports, electronic format, etc.) needs to be analyzed. The recommendations to improve the methodology, survey instruments, and data quality should be made as a part of this exercise. These recommendations then should be put forward to the team that is working on the National Statistical Capacity Building initiative. Preliminary meetings with this team during the field mission suggest that the focus on PSD/IC related data and statistics has so far been absent from this initiative. Third, appropriate M&E instruments, need to developed and deployed to regularly collect data based on the Doing Business methodology and enterprise survey, both at the national level and from the poles. As already mentioned above, the basis for a number of investment climate indicators is the World Bank ICA survey. The ICA survey is likely to be conducted once in every four years. But these indicators need to be updated at least on an annual basis. The EDBM proposed to conduct an enterprise survey on an annual basis to monitor the growth and expansion of the enterprises over time and the state of the investment climate. This survey instrument along with the methodology and an appropriate implementation strategy should be developed as a part of the M&E system design exercise. 22 In this connection, due consideration should be given on the issue of “survey fatigue� and on alternative ways of data collection. For instance, during our consultation with the business associations, SIM (business association of manufacturers) suggested that EDBM should coordinate with all the business associations so that the business associations can supply EDBM the required data from their member enterprises that the associations routinely collect from their members. This in turn will not require EDBM to survey these firms again. Moreover, data obtained through this way can be more accurate as the firms feel more comfortable to provide information to their associations rather than to a surveyor or government body. And in fact, during our consultation with the INSTAT (statistic bureau), it highlighted “non� or “false� responses of the firms is a serious problem in Madagascar. If the survey needs to be conducted, then an appropriate implementation mechanism should be also analyzed, i.e., whether EDBM will conduct it, or outsource it to a survey firm or to INSTAT. Similarly, to reduce survey fatigue, whether such a survey module should be tagged into one of the regular surveys of INSTAT. Options like these, on the one hand, can contribute to the capacity development of the INSTAT, and on the other hand, can save resources through exploiting economies of scales. Fourth, once the logframe and list of indicators are validated and finalized, baseline data on these indicators need to be collected against which progress and results of reforms will be benchmarked. Fifth, an appropriate project performance tracking tool needs to be developed to track the activities and outputs of EDBM and how these are contributing to the intended outcomes. Similarly, the appropriate M&E tools should be developed and deployed to measure the organizational effectiveness of EDBM and its contribution to the reform process. Examples of two such tools are given in this report, but by no means are these comprehensive. Validity of these tools needs to be analyzed, and accordingly these tools need to be further modified or new tools need to be put in place. Sixth, efforts should be made to design a live and web-based M&E system for EDBM, which should keep updated the concerned stakeholders (i.e., EDBM Board and Management, relevant stakeholders from the government, private sector and donors) on the state of investment climate in Madagascar and EDBM’s contribution to that. Seventh, an appropriate institutional system needs to be established to operationalize the M&E system so that the system remains neutral and credible. In this connection, due consideration should be given to the issue whether the M&E activities should be carried out by an independent third party managed by a small M&E cell of the EDBM, or it should be carried out in-house (such as the IGPP (PIC) Project’s M&E system). An appropriate coordination mechanism 23 between IGPP’s M&E and EDBM’s M&E system also needs to be established to eliminate duplication of data collection and to maximize the leveraging of one M&E system on the other. 24 Annex 1: EDBM Program Logic Framework ( Logframe) Intervention logic Objectively verifiable indicators Sources of verification Assumptions/Outside Influences/Risks Goal/ Reduced poverty: • Poverty Headcount Ratio (Global), i.e., World Development Impact reduce the population below the percentage of people below $1/day Indicators (WDI) poverty line by 50% by 2015 (MDG) • Poverty Headcount Ratio (National), i.e., percentage of people below the national INSTAT poverty line (preferably disaggregated by gender) • Madagascar Human Development Index INSTAT Reduced Informality • Informal sector as a percentage of GDP Doing Business, World Growth of business, in Bank particular of SMEs and To be confirmed, INSTAT9 reduced informality lead to more employment (both for • Sales amount reported by a typical firm ICA/EDBM enterprise men and women) and for tax purposes (%) survey10 increased investment and Increased employment • Number of manufacturing and service INSTAT, PIC export. Higher levels of sector employment (gender investment (both domestic and disaggregated) national and in the three foreign) in turn lead to higher growth poles level of growth and • Number of employment in SMEs employment and thus reduce (gender disaggregated) ) national and in poverty. Growth and poverty the three growth poles however are influenced by a wide range of factors other than investment. 25 Intervention logic Objectively verifiable indicators Sources of verification Assumptions/Outside Influences/Risks Increased export in priority sectors • Volume and value of export in EDBM PIC, Central Bank, priority sectors11 ( ICT and agri- INSTAT, EDBM business) national and in the three growth poles • Export from ICT and agro-business as a percentage of total export ( national and in the three growth poles ) Growth of business • Number of new firms registered for INSTAT, Tax Authority, In the absence of adequate data Business EDBM on the exit of firms, the • Number of firms filing tax returns with number of new entrants is the tax authority likely to be inflated due to re- Growth of SMEs • Number of new SMEs, national and in INSTAT, EDBM, Tax registration and failure to the three growth poles Authority, PIC, SME record most exits. Data from tax authority on number of • Number of SMEs filing tax returns with Solution Center (SME database) firms submitting tax returns the tax authority, national and in the would probably be the best three growth poles source but may be difficult to obtain. Increased foreign direct investment • Net inflow of FDI, national and in the INSTAT, EDBM, Central (FDI) three growth poles Bank Increased domestic investment • Volume of Gross Fixed Capital INSTAT, Central Bank, Formation (GFCF), private sector; EDBM, PIC national and in the three growth poles • Volume of GFCF in EDBM priority sectors • Volume of investment of SMEs, national and in the three growth poles 9 If the data is not regularly available from Doing Business study, INSTAT may consider to measure the size of informality in Madagascar 10 Wherever in the logframe ICA is mentioned implies that the data is available in the ICA. But the ICA survey is conducted once in every four years. So data on these indicators need to be updated on an annual basis through surveys conducted by EDBM 11 As far as EDBM mandate is concerned the priority sectors are agro-business, tourism and ICT 26 Intervention logic Objectively verifiable indicators Sources of verification Assumptions/Outside Influences/Risks Process/Intermediate Indicators Objective/ Outcome Ease of doing business • Number/List of reforms undertaken Gazetted changes by the Removal of unnecessary based on EDBM recommendations in GoM, Concerned Line administrative and regulatory terms of policies/laws/legislations and/or Ministries /Public burden contributes to improved institutions related to: Organizations investment climate and thus • Business Start-up leads to higher levels of • Licensing investment, employment, • Employment formalization and productivity. • Property Registration • Credit • Investor Protection • Tax • Trade • Contract Enforcement • Business Exit Result/Final Outcome Indicators: • Ease of Doing Business in terms of EDBM reduced time, cost, and processes (national level and in the three growth poles) involved in: • Business Start-up • Licensing • Employing Workers • Property Registration • Getting Credit • Investor Protection • Tax Payment • Trade • Contract Enforcement • Business Exit 27 Intervention logic Objectively verifiable indicators Sources of verification Assumptions/Outside Influences/Risks Better bureaucracy Process/Intermediate Indicator: • Number of relevant government Concerned government Improved access to organizations publishing their respective organizations; EDBM information reduces the risk regulatory policies and procedures in and uncertainty of doing their respective websites based on business and thus positively EDBM recommendation. affects investment flow. • Number of government organizations with e-government options based on Increased transparency of EDBM recommendations/interventions regulatory policies reduces • Measures undertaken by EDBM for policy uncertainty. business facilitation Result/Final Outcome Indicator: ( national level and in the three growth poles) • Percentage of senior management time ICA/EDBM enterprise spent in dealing with requirements of survey, EDBM government regulation; preferably disaggregated by regions • Percentage of firms view that “circulaires� affecting their business are implemented with no further interpretation from relevant authorities Well functioning court system Process/Intermediate Indicator: • Number of measures undertaken by Ministry of Justice GoM to improve the court system Result/Final Outcome Indicator: ( national level and in the three growth poles) • Percentage of firms having confidence in ICA/EDBM enterprise the court system survey • Time spent in solving disputes in the court (weeks) 28 Intervention logic Objectively verifiable indicators Sources of verification Assumptions/Outside Influences/Risks Reduced corruption Process/Intermediate Indicator: • Number of measures undertaken by Concerned government relevant government organizations to organizations; EDBM curb corruption (i.e., on-line tracking system of applications; well-functioning complaint mechanism, e-governance, etc.) based on EDBM recommendation/intervention • Number of measures undertaken by EDBM as a part of its business facilitation role to curb corruption Result/Final Outcome Indicator: ( national level and in the three growth poles) • Unofficial payments for typical firm to ICA/EDBM enterprise get things done (% of sales) survey Better public service delivery to Process/Intermediate Indicator: enterprises • Number of measures undertaken by the Concerned government concerned government organizations to organizations; EDBM improve the service delivery to the businesses Result/Final Outcome Indicator: ( national level and in the three growth poles) • Delay in obtaining an electrical ICA/EDBM enterprise connection (days) survey , EDBM • Delay in obtaining a mainline telephone connection (days) • Firms using the Web in interaction with clients/suppliers (%) 29 Intervention logic Objectively verifiable indicators Sources of verification Assumptions/Outside Influences/Risks Outputs EDBM’s recommendations to improve • Number of recommendations provided in EDBM Recommendations are various dimensions of investment each topic area accepted and implemented by climate as measured by the outcome the Government in a indicators above comprehensive and meaningful way. Activities • Desk research for preparing the • Survey instruments and templates Regular Status Update Activities are duly undertaken diagnostics, solution design and • Reports on desk research and field work Reports from EDBM according to the corresponding implementation plan • Notes from stakeholder consultations and timelines with reasonable • Field work undertaken for public-private dialogues degree of cooperation from all preparing the diagnostics, solution the relevant parties design and implementation plan • Stakeholder consultations • Public-private dialogues on PSD/IC Reform Issues • Development of Action Plans and Reform Strategies • Advising the GoM on undertaking necessary reforms Resource • Different phases of EDBM • USD $$$$ (Number of days of staff and EDBM’s Financial Allocation of resources is Inputs consultants time plus other itemized Reporting System efficient in order to obtain cost) desire outputs, which in turn • + In-kind contribution (i.e., amount of influences intended outcomes staff time and resources spent from the and impacts Government and private sector) 30 31 Terms of Reference Monitoring and Evaluation System Design For the Economic Development Board of Madagascar 1. Background and context of the Economic Development Board of Madagascar (EDBM) In its development strategy - the Madagascar Action Plan (MAP), the Government of Madagascar (GoM) aims at raising the country’s economic growth rate between 7% and 10% by 2012. To achieve this growth rate, which is crucial for poverty alleviation, the government commits to have a diversified and strong private sector driven by local and international investment and trade. In particular, to boost the foreign direct investment, the GoM in the MAP sets the target to increase the foreign direct investment (FDI) from US$84 millions in 2005 to US$500 millions in 2012. As is now widely recognized, an investment climate without unnecessary and excessive administrative and regulatory burdens, with transparent rules and regulations, and accountable and well-functioning public institutions, is a key to increase investment and growth. Accordingly, in the MAP the government commits to generate the fundamental conditions to support the facilitation of business. To improve Madagascar’s investment climate, the GoM in the MAP sets the target to improve the ranking of the ease of doing business indicator from 149th in the Doing Business Report, 2007 to 80 by 2012, assuming all else equal (i.e., measured against the global 2006 ranking). To improve investment climate and thus to increase investment and to achieve the target of private sector led high growth economy by 2012, the GoM created the EDBM, a privately managed structure, under the Office of the Presidency in 2006. The EDBM is a single and autonomous body with a private-public Board and plays a role of investment promotion and business facilitation. The EDBM is funded under the World Bank’s Integrated Growth Poles Project (IGPP) for the first three years. EDBM’s mission is to: • promote Madagascar as a preferred destination for investment; • increase levels of FDI by proactively attracting and securing investments; • provide professional and efficient investment facilitation services to investors; and • create value for investors and actively contribute to the economic development and growth of Madagascar and improvement of its business climate. 2. Monitoring and Evaluation of the EDBM Accomplishment of such an energized and result-oriented mission requires setting up of a neutral and credible monitoring and evaluation (M&E) system in the EDBM from the beginning of its operation. Monitoring reforms tend to generate reforms. This is 32 demonstrated by a number of reform experiences around the world (i.e., the Bulldozer Reform in Bosnia and Herzegovina, the Procurement Law Reform in the Philippines, the Citizen Score Card Initiative in Indian Cities, Inspectorate Reform in Latvia, etc.) and the lessons from the World Bank Group’s Doing Business Studies. Hence, the M&E will be viewed as an integral part of EDBM’s effort to realize its stated mission. This M&E system will monitor the EDBM interventions and assess their impacts. In particular, the M&E system will: • regularly monitor the overall investment climate in Madagascar, • regularly monitor to what extent EDBM is contributing to the improvement of investment climate, and • evaluate to what extent the improvement of the investment climate is contributing to the relevant goals of MAP in terms of expanded private sector of all sizes (i.e., small, medium, and large enterprises), increased domestic and foreign investment, trade, and growth and thereby increased employment and reduced poverty. 2.1.Objectives of the M&E system The objective of the M&E system will not be to serve accounting purposes. Neither will it be in place as a donor requirement to generate reports for the donors. Rather it will serve at least three critical purposes. First, the M&E system will promote the transparency and accountability of the government’s PSD reform process by making the progress and results of the reforms public. EDBM’s mandate is to facilitate the investment climate reforms through policy advocacy. Using appropriate M&E tools, the M&E system will also monitor to what extent EDBM is reaching out to the private sector at large and is contributing to the PSD and investment climate (IC) reforms that are deemed crucial by the private sector. In particular, the M&E system will demonstrate the attribution of EDBM interventions to the IC reforms and the corresponding results (in terms of outcomes and impacts) in Madagascar. This in turn will enhance EDBM’s credibility, transparency and accountability to its stakeholders (i.e., the government, private sector and the civil society at large). Second, the M&E system will serve as a feedback loop and lesson-learning tool to provide constructive feedback to all the concerned stakeholders about where reforms are needed and whether the desired results from the reforms are realized. This in turn will enable the policy makers to better design the reforms. Similarly, monitoring and evaluating how well EDBM is contributing to the reforms and making progress in accomplishing its stated mission should provide appropriate feedback to the EDBM Board and Management on the effectiveness of EDBM’s current strategy and interventions. This in turn will further improve EDBM’s strategy and interventions. Third, through regular dissemination and communication of the key M&E findings, the M&E system will facilitate the reform process by generating public 33 awareness, public support and political will for the reforms and by weakening the illegitimate vested interests against reforms. 2.2.EDBM’s program logic framework A comprehensive program logic framework (logframe) for the EDBM has already been designed (see Annex I), which will serve as the back-bone of the EDBM M&E system. This logframe defines the goals, objectives, outcomes, and impacts of the EDBM interventions. Generic definitions of the activities and outputs of EDBM are also mentioned in the logframe, but the actual indicators for these need to be developed by consulting the EDBM management, WBG team, and EDBM’s business and work plan. The logframe also provides a set of objectively verifiable impact and outcome indicators (OVIs) and their potential sources to track the results of EDBM interventions. 3. Scope of work The EDBM is now seeking consulting services to design the EDBM M&E system to monitor and evaluate the inputs and activities, outputs, outcomes, and impacts of EDBM intervention using appropriate evaluation methodologies. In designing the EDBM M&E system to fulfil the three objectives discussed in Section 2.1 above, the consultant will undertake the following key activities: 3.1.Validate the EDBM logframe, take stock of existing Investment Climate (IC) and Private Sector Development (PSD) related surveys, indicators and data, and develop appropriate M&E instruments and the baseline data The consultant will validate the EDBM M&E logframe. More specifically, the consultant will do the following: • Evaluate the adequacy of the objectively verifiable indicators (OVIs) in the logframe and suggest necessary changes or additions of indicators to appropriately measure the outputs, outcomes and impacts. • Evaluate each of the sources of verification (SoVs), make necessary changes or additions to these data sources based on data availability, evaluate the data quality of the SoVs, and suggest further improvement in terms of methodology, frequency, and data collection processes of the SoVs. • Collect the baseline data for the OVIs and prepare a baseline data matrix against which the results of the EDBM interventions will be benchmarked. For the OVIs for which data is not currently available, the consultant should propose a methodology and strategy for data collection. • In relation to the evaluation of SoVs and collection of baseline data, the consultant will also take stock of all the existing Investment Climate and PSD related surveys, indicators, and data available from the WBG, IGPP, and other International and 34 National Sources and assess whether there is any need for any additional survey to particularly measure the outcomes and impacts of the EDBM interventions. If yes, then the consultant will propose sample questions, survey methodology, and an implementation plan. • The consultant will develop appropriate M&E tools to collect data on Doing Business indicators both at the national level and from the growth poles. Similarly, the basis for a number of proposed investment climate indicators is the World Bank ICA survey. But the ICA survey is likely to be conducted once in every four years, while these indicators need to be updated at least on an annual basis. The EDBM proposes to conduct an enterprise survey on an annual basis to monitor the growth and expansion of the enterprises over time and the state of the investment climate. This survey instrument along with the methodology and an appropriate implementation strategy should be developed as a part of the M&E system design exercise. • In connection with the proposed enterprise survey of the EDBM, the consultant will take into account the issue of “survey fatigue� and will explore alternative ways of data collection. For instance, during the WBG consultation with the business associations, SIM (business association for manufacturers) suggested that EDBM should coordinate with all the business associations so that the business associations can supply EDBM the required data from their member enterprises that the associations routinely collect from their members. This in turn will not require EDBM to survey these firms again. Moreover, data obtained through this way can be more accurate as the firms feel more comfortable to provide information to their associations rather than to a surveyor or government body. In fact, “non� or “false� responses of the firms is highlighted as a serious problem in Madagascar by INSTAT (National statistics office). • If the enterprise survey needs to be conducted, then the consultant should propose an appropriate implementation mechanism, i.e., whether EDBM will conduct it, or outsource it to a survey firm or to INSTAT. Similarly, to reduce survey fatigue, whether such a survey module should be tagged into one of the regular surveys of INSTAT. Options like these, on the one hand, can contribute to the capacity development of the INSTAT, and on the other hand, can save resources through exploiting economies of scales. 3.2. Develop a comprehensive, live and web-based M&E System The consultant will design a comprehensive, live and web-based M&E system. As the first component of the M&E system, the consultant will lay out a detailed M&E plan that will address (but not be limited to) the following questions: • What would be the appropriate evaluation methodologies (i.e., comparison against baseline, case studies, experimental or quasi-experimental designs, etc.) for the 35 EDBM interventions to best demonstrate the outcomes and impacts of the EDBM interventions and to deal with the attribution issues? • How will the monitoring of inputs, activities, outputs, and outcomes, and evaluation of outcomes and impacts be undertaken to ensure transparency, accountability and credibility of EDBM to its stakeholders? • How EDBM’s organizational effectiveness and its contribution to the investment climate reform process in Madagascar are going to be monitored and evaluated? What will be the appropriate M&E tools for this purpose? • Who will be the key implementing bodies of the EDBM M&E system, what will be their reporting structure, roles and responsibilities? • Who are potential local implementers and firms with whom EDBM can collaborate on M&E activities? How will these partners work and what will be the role and responsibilities of all the concerned partners? • How to ensure that the monitoring data is live, web-based and easily available to all the concerned stakeholders through secured web access? Is there any risk associated with the M&E system being live, web-based, and easily accessible to concerned stakeholders, and if yes, what are the mitigation measures? • What will be the reporting options of the M&E results, i.e., quarterly hard copy reports, monthly soft copy reports, and/or web based live reports, and what are the implications of each? • How the EDBM M&E system can serve as an effective lesson-learning tool for EDBM? Based on the M&E plan, the consultant will: • Design and develop appropriate M&E instruments (such as the project performance tracking tool, instruments to obtain data on Doing Business indicators, instruments to obtain enterprise-level data, etc.) and user-friendly templates and systems for data collection on inputs, activities, outputs, outcomes, and impacts of the EDBM interventions • Design and develop a performance management scorecard for EDBM • Design and develop the processes to implement these M&E instruments and performance management scorecards by the concerned M&E implementing bodies • Design and develop the reporting structure of the M&E implementing bodies 36 • Design the concept process of making M&E data live, web-based, and easily accessible to all the concerned stakeholders • Develop baseline data matrix for the EDBM OVIs • Evaluate the possibilities and outline the proposals for conducting experimental/quasi-experimental impact evaluation of EDBM interventions to clearly demonstrate the outcomes and impacts of these interventions • Develop selection criteria, and recommend the roles and responsibilities for all the M&E implementing bodies and MoUs for the potential local partners with which EDBM will engage to implement its M&E system. 4. Key considerations: 4.1. Ensure the neutrality and credibility of the EDBM M&E system The consultant will design an M&E institutional structure to ensure the neutrality and credibility of the EDBM M&E system. Box 1 below gives an overview of a tentative institutional arrangement of the EDBM M&E system. It is anticipated at this stage that to ensure neutrality and credibility of the M&E system, a significant portion of EDBM’s M&E activities, such as routine data collection and analysis, will be undertaken by an independent and credible local M&E contractor that will be managed by EDBM’s internal M&E cell. Moreover, while the EDBM M&E cell will be managed by the EDBM management for its day-to-day performance and activities, the M&E cell will directly report the M&E findings to the EDBM board and its Donor Oversight Committee. The reporting structure along with all other M&E institutional details will be thoroughly assessed in light of best practices to design a neutral and credible M&E system. The consultant will prepare detailed ToRs with roles and responsibilities for all the concerned M&E implementing bodies (such as the local M&E contractor, the EDBM M&E cell, and the EDBM management). 4.2. Assess local capacity and identify key local partners The consultant will evaluate and identify potential local M&E contractors to conduct data collection, collation, storage, analysis and reporting of the EDBM OVIs with credibility, neutrality and competence. The M&E cell in EDBM will have oversight of this local M&E contractor on behalf of the EDBM Board and Donor Committee, but the local M&E contractor will maintain independence in its analysis, evaluations, and reporting. The consultant will prepare detailed ToRs for the local M&E contractor to successfully carry out EDBM’s M&E activities with the oversight of the EDBM M&E cell. 5. Key deliverables 37 Based on the scope of the work, the key deliverables from the consultant are as follows: • Design of a neutral and credible M&E institutional structure to implement the EDBM M&E system together with a list of potential local partners and a set of selection criteria for partnership, • Design a comprehensive, live, and web-based M&E system that involves the issues, items and deliverables discussed above in Section 3 and Section 4. 6. Level of Effort and Timeline The selected consulting firm will have about eight to ten weeks to complete this EDBM M&E system design assignment. After initial desk-research, the assignment will involve a two to three-week field mission in Madagascar around the middle of January 2008. The consultant will be reimbursed for travel cost related to the assignment. The consultant will submit all the draft deliverables within four to five weeks from the completion of the field trip and the final deliverables within two weeks from receiving the comments on the draft deliverables. The consultant will report to EDBM. Draft deliverables are reviewed and validated by EDBM, IGPP and WBG teams. Consultant Qualifications The consultant should have the following qualifications: • a sound understanding of investment climate and private sector development programs in developing countries, including knowledge of World Bank Group investment climate assistance; • a sound understanding of the political economy of investment climate reforms with a high degree of relevance to Madagascar; • a demonstrated understanding of the relationship between IC/PSD Technical assistance intervention and outcomes and impacts; • a strong experience in designing an M&E system for monitoring and evaluating technical assistance related to advisory, public policy, institutional capacity, economic zones, and/or regulatory programs in private sector development; and, • have substantial work experience in Sub-Saharan Africa and Madagascar in particular. 38 39