ECA Field Notes for Practitioners Implementing a business-to- government feedback mechanism in the Kyrgyz Republic to improve public-private dialogue and regulatory service delivery December 2017 In Partnership With ECA Field Notes for Practitioners ©2017 The World Bank Group 1818 H Street NW Washington, DC 20433 Telephone: 202-473-1000 Internet: www.worldbank.org All rights reserved. This volume is a product of the staff of the World Bank Group. The World Bank Group refers to the member institutions of the World Bank Group: The World Bank (International Bank for Reconstruction and Devel- opment); International Finance Corporation (IFC); and Multilateral Investment Guarantee Agency (MIGA), which are separate and distinct legal entities each organized under its respective Articles of Agreement. We encourage use for educational and non-commercial purposes. The findings, interpretations, and conclusions expressed in this volume do not necessarily reflect the views of the Directors or Executive Directors of the respective institutions of the World Bank Group or the govern- ments they represent. The World Bank Group does not guarantee the accuracy of the data included in this work. Rights and Permissions The material in this publication is copyrighted. Copying and/or transmitting portions or all of this work with- out permission may be a violation of applicable law. The World Bank encourages dissemination of its work and will normally grant permission to reproduce portions of the work promptly. For permission to photocopy or reprint any part of this work, please send a request with complete infor- mation to the Copyright Clearance Center Inc., 222 Rosewood Drive, Danvers, MA 01923, USA; telephone: 978-750-8400; fax: 978-750-4470; Internet: www.copyright.com. All other queries on rights and licenses, including subsidiary rights, should be addressed to the Office of the Publisher, The World Bank Group, 1818 H Street NW, Washington, DC 20433, USA; fax: 202-522-2422; e-mail: pubrights@worldbank.org. 2 May 2017 Acknowledgements This document was written by Victoria Tetyora, Private Sector Specialist, Serhiy Osavolyuk, Se- nior Private Sector Specialist, Syinat Arynova, Private Sector Specialist, Trade & Competitiveness Global Practice in Europe and Central Asia, World Bank Group. The authors wish to thank the Swiss Confederation through its State Secretariat for Economic Affairs (SECO), and the Depart- ment for International Development of the United Kingdom of Great Britain (DFID), for their generous support in funding the activities of the World Bank Group’s Investment Climate Project in the Kyrgyz Republic. The authors would like to also thank and acknowledge Syed Akhtar Mahmood, Lead Private Sector Specialist, Trade & Competitiveness Global Practice and Christopher David Miller, Senior Private Sector Specialist, Trade & Competitiveness Global Practice in Europe and Central Asia, for their valuable comments and peer review of this document. 3 ECA Field Notes for Practitioners Background and rationale for engagement Businesses in the Kyrgyz Republic have long suffered from excessive inspections. Inspections coverage was high, often reaching 90 percent annually and inspectorates did not usually apply risk-based approaches when selecting businesses for visits. As a result, inspectors applied the same intensity for chvecking small and large firms. The World Bank Group’s Trade & Competi- tiveness Global Practice, through its IFC Investment Climate Project, has helped the government to introduce risk-based inspections legislation. The Government of the Kyrgyz Republic adopted a 2007 Law on Inspections, with subsequent amendments introduced throughout 2011-16. This has helped to reduce the average inspections burden to about 70 percent, which although lower is still far too high. Figure 1: Coverage of businesses by inspections remains high despite the adoption of legislation 69% Legal Entities 73% 73% Individual Enterprises 58% 2008 2011 Source: IFC Investment Climate Reports, 2008-11. At the same time, the implementation of institutional reforms has begun, with the number of inspectorates being reduced from 21 to 12. However, reduction in number of inspectorates has not led to fewer inspections of businesses. The consolidation of inspectorates occurred largely through the merging of several inspectorates into one. For example, the State Inspectorate of Ecology and Technical Safety merged nine different inspectorates. This merger of nine inspec- torates has not resulted in real functional integration at the organizational level. In addition, a lack of intra-governmental coordination has remained. The large number of violations cited by businesses and the sanctions issued by inspectors in- dicated that compliance promotion activities have not been among the inspectors’ main pri- orities. They devoted little-to-no time to these activities. The gap between businesses’ view of compliance (necessary, possible) and the real situation indicated the importance of intensifying inspectorates’ efforts to provide advice and guidance to businesses so that they effectively know what the requirements are, and how to comply with them. 4 May 2017 Figure 2: Inspections continue to indicate low levels of compliance with regulations Yes, violations were detected but not registered 9% Yes, violations were detected 60% and officially registered 31% No, not a single violation was detected Source: IFC Investment Climate Reports, 2008-11. Previously, there have been no reliable ways of communication between the government and the private sector. For example, ‘hotlines’ for businesses, de-jure existing in every agency, have not been used at all. Instead, businesses continued to rely on friends and contacts for informa- tion about regulations and compliance requirements. Figure 3: The majority of entrepreneurs do not use the ‘hotline’ Not aware of the hotline’s existence 14% Did you No, did not use 73% use the hotline? 13% Yes, used Source: IFC Investment Climate Reports, 2008-11. 5 ECA Field Notes for Practitioners Causes of implementation gaps The key causes of the inefficiencies in implementing inspections legislation reform in the Kyrgyz Republic identified above are as follows: 1) Inability of inspectorates to embrace the reform institutionally and functionally Institutionally the agencies were not sufficiently prepared to perform their functions in accor- dance with the new rules introduced by the reform. This involved failure to coordinate activities between different units of one agency and between several agencies with inspections func- tions. Internal policies, systems, procedures and practices, and even mentality and perceptions towards the goal of inspections (e.g., to prevent vs to find fault and fine) meant that the everyday ‘life’ of the agency has changed little, if at all. Inspectorates’ personnel were not equipped with sufficient knowledge, skills and expertise to perform their duties in accordance with the new rules. This resulted from the failure of the in- stitutional set-up to provide sufficient support for inspectors and other agency staff to prepare for and embrace the changes brought by the reform. A performance management system, competency frameworks, division of functions between departments, continuous professional development and incentives systems were not revised and aligned with reform objectives. 2) Formal versus functional merger of inspectorates Several departments that used to belong to separate inspectorates are now part of one agency operating under the same mandate. The main goal of such merger was to eliminate the dupli- cation of functions, ensure a more efficient use of resources and improve information exchange and risk management. However, the merger was not followed by a functional review of all de- partments of each merged inspectorate. Therefore, duplication remained, with no additional ef- ficiency gained. Failure to perform a proper internal restructuring led to a failure to adopt, among others, a compliance promotion activities strategy and, as part of the rebalancing of inspectors’ work, to allocate more time to advice, outreach, analysis and planning activities rather than to ‘business-as-usual’ visits to businesses. 3) Poor dialogue with the private sector The private sector was neglected in the reform process. The government tried to establish a business ‘hotline’ to hear the views of businesses. However, this was largely a ‘tick-the-box’ ex- ercise, since feedback received through the hotline was rarely considered seriously. As a result, businesses lost trust in the communication channel, leading to an extremely low level of use. Public-private dialogue in reform design and implementation was inconsistent. Even when the government introduced improvements to the legislation, businesses were rarely informed of them. This led to many problems, such as an absence of trust between the public and private sectors, implementation gaps and non-compliance. 6 May 2017 Table 1: Tools that help identify the regulatory implementation gaps Tool & Description Benefits Considerations Focus groups with businesses  Face time/ direct contact with  Time consuming businesses  Requires meticulous Focus group discussions (FGD) are one of the qualitative research methods used to solicit feed-  In-depth and detailed infor- selection of the audience back from a homogenous group of individuals on mation about a specific issue according to specific a certain issue or product. In the context of the Kyr- from the target group criteria gyz IC work, the team used FGD to glean the views  Opportunity to drill down to  Requires trained facilita- and perceptions of private sector representatives the root causes of the prob- tors on how certain inspections regulations were be- lem with implementation or  Has little coverage of the ing implemented, as well as to check whether they non-compliance target population were aware of the inspections legislation, and their rights and obligations. Studies and surveys  Quantifies the problem to  Time consuming strengthen the argument  Usually outsourced to a The Kyrgyz team conducted an inspections imple- mentation and governance study to analyze the  Can cover large segments of research company that way inspections are implemented by the Environ- the target population needs to be managed mental Safety Department of the SIETS in Bishkek  Opportunity to explore a large closely and two districts of Chui region. The study ana- number of issues  Recommended to do lyzed the views of inspectors and entrepreneurs.  Can be flexible in terms of repeatedly to measure the The study was the first (project-led) feedback-seek- methods –telephone/ face- progress ing vehicle that, besides revealing implementation to-face interview, a written  Need to be used in com- gaps, also reinforced the need for the agency to questionnaire; large or small- bination with qualitative systematically solicit feedback from businesses. scale (depending on number research methods, e.g., of issues explored) FDG or case study Meetings with inspectorates  Ability to voice concerns and  Need to be regular to keep present proposals/ solutions the momentum going These range from formal meetings with top lev- directly to the target audi-  Consider having at least el of inspectorates to working and/or fact-find- ence one meeting with top ing meetings with agency’s mid-level personnel; working group meetings.  Ability to obtain information management of the agen- about the ways things are cy to help them become done in the agency, identify more involved inefficiencies  Need to have and follow  Engagement of agency a specific agenda to avoid personnel in problem identi- derailed discussions and fication and solution find- time-wasting ing, thereby strengthening the agency’s buy-in in the process 7 ECA Field Notes for Practitioners Tool & Description Benefits Considerations Public-Private Dialogue events  Both sides of reform at one  More effective when there table discussing, debating is a series of events dedi- PPD is a structured, inclusive and participatory and looking to find solutions cated to a certain subject mechanism of policymaking. In the context of the together  Preparation is key: partic- Kyrgyz IC work, the team has not established a PPD platform per se, but used approaches and mech-  Facilitate dialogue between ipants should know up- anisms from the PPD, for example, organized reg- the public and private sectors front the purpose, agenda, ular events with participation of public and private  Outreach to wider civil their roles and expected sectors to discuss the issues of poor implementa- society outcomes tion of inspections regulations and how it affect-  Put public officials on the  For events open to public, ed businesses. The team also participated in such spot and encourage them to mass media brings a new events organized by other players (either from the take action dimension of capturing public or private sector) and ensured the issue of and publicizing the event’s  Flexibility – can be working the implementation gap was on the agenda. The decisions widely and put- events having representa- idea was to have both parties at the table jointly ting pressure on officials tives of public and private looking at problems and identifying solutions. to ‘walk their talk’ sectors or events open to other stakeholders, includ- ing mass media for greater coverage Desk research  Enriches the findings of  Obtaining access to inter- focus groups or surveys nal documentation is not This involves a review of relevant legislations (laws,  Identifies bottlenecks and always possible bylaws, decrees, orders) and agency’s internal documentation, such as procedural documents, loopholes in legislation  Interpretation of internal operational manuals.  Helps to establish wheth- procedural documents er there are ambiguities in can vary in agency interpretation of legislation regional representative by agencies offices  Identifies gaps in procedural  Should be done in com- documents bination with interviews of agency staff to avoid misinterpretation 8 May 2017 The solution: introducing a web-based feedback mechanism The identified problems are complex and diverse. The Kyrgyz Trade & Competitiveness team provided comprehensive support to the Government of the Kyrgyz Republic in eliminating key inefficiencies that caused implementation gaps of the inspections legislation. This case study covers one aspect of this work, namely the establishment of a business-to-government feed- back mechanism to improve the government’s dialogue with the private sector and regulatory service delivery to contribute to closing implementation gaps. The Kyrgyz government has tried to gather feedback from businesses in the past. However, the ‘hotline’ telephone line, created by the government several years ago, was not popular among businesses. Some of the reasons for this included complicated operations procedures, poor technical implementation and the government’s failure to respond to feedback. Businesses did not see any reaction from the government to their feedback, further undermining the private sector’s trust in the process. The new feedback mechanism developed by the Kyrgyz T&C team together with the State In- spectorate of Ecology and Technical Safety (SIETS) and launched in December 2015 addresses the aforementioned concerns. It is user-friendly and mainly automated. More importantly, the government is committed to taking action on the feedback received and disseminating the re- sults to the business community, i.e. closing the feedback loop. The new feedback mechanism is embedded in the www.proverka.kg portal—a government portal for processing inspections. The mechanism works as follows (Figure 4):  Inspectors collect the email addresses of all businesses they visit as part of planned in- spections. After each inspection an email is sent automatically to the entrepreneur from www.proverka.kg1 portal inviting them to complete a short online questionnaire. Each link 1 An information and inspections management sent to an entrepreneur is unique. The system processes the results of the received feedback portal in the Kyrgyz instantaneously. Data are aggregated in real time and are presented on charts, available on Republic launched in 2012 the portal quarterly. with the help of the World Bank Group. ‘Proverka’  Entrepreneurs can also access the feedback questionnaire any time directly from the stands for ‘inspection’ in www.proverka.kg, the websites of the SIETS, the Sanitary Department, and the Ministry of Russian. Economy. Figure 4: How the feedback mechanism works Inspector-triggered During inspection After inspection Inspector collects entrepre- Inspector files an inspections report on neur's email address and logs www.proverka.kg portal. After feedback it in the inspection report is received An automated email is sent to the entrepre- neur asking to fill in the feedback questionnaire Feedback results are processed instantaneously. Data are agregated in real Entrepreneur-triggered time and are presented on After inspection charts, available on www.proverka.kg portal Entrepreneur visits www.proverka.kg quarterly portal or the websites of SIETS, the Sanitary Department and the Ministry of Economy and fills in a post-inspection questionnaire 9 ECA Field Notes for Practitioners Benefits of having a two-way approach for soliciting feedback from businesses:  It provides flexibility in access points of the questionnaire.  The second access point (through websites) also serves as a back-up for human factor errors, e.g., if inspectors forget to collect emails or make a typo when registering emails either in the book during inspection or when transferring it to the online inspections portal during reporting.  Feedback is not only connected to a recent inspection. It can be provided any time. Those entrepreneurs that have not been inspected can also send their feedback by accessing the questionnaire on websites any time they feel like providing feedback and expressing their views on the service they received from inspectors. Box 1: The role of the inspections portal www.proverka.kg The online feedback mechanism is linked to www.proverka.kg — an official national portal of the Ministry of Economy launched in 2012 (also with support from the Trade & Competitiveness team and other development partners) that com- puterizes the inspections process. The portal operates on specialized software that enables officials to plan, approve, conduct, and monitor inspections based on risk assessment. The system assigns risk profiles for each entrepreneur based on pre-defined risk criteria. Businesses can also use the portal to obtain updates on legislation and to access in- spections plans, approvals and checklists. The portal is accessible by:  Employees of inspectorates  Employees of the Ministry of Economy of the Kyrgyz Republic responsible for monitoring inspections  Businesses through their personal virtual rooms  Other stakeholders (access to non-classified data only)  System administrators (full technical access) The portal offers an automated approval process of inspections plans (the Management Information System stores in- formation about document development and movement, its author, date and time of drafting, its status (submitted for consideration, approved, rejected). Similarly, the reporting system is automated—a detailed report on each inspection or by all types of inspections is available in the MIS, a general report is generated, information on document movement is stored, as well as the author of the draft, the time of drafting and its status (submitted for consideration, approved). The portal also has one of the access points to the feedback mechanism. It hosts a link to the questionnaire on its front page and is accessible to any entrepreneur willing to leave feedback at any time. Interim reports indicate 70 percent of feedback received so far has come through www.proverka.kg portal making it the most popular access point. 10 May 2017 Considerations when deciding what feedback tool to choose There are many ways of soliciting feedback from businesses, from using simple online ques- tionnaires on websites of agencies to telephone-enabled hotlines to SMS services to interactive Apps on mobile devices. The choice of the tool depends on several factors. One of the key as- pects in choosing the tool is that the process of choosing it should be inclusive and consultative. The Kyrgyz T&C team involved the following key stakeholders in the process: The public sector These were agencies directly involved in the process, e.g., high and mid-level employees of SIETS. The team consulted with them about the internal operations of the agency and its de- partments, the systems (including IT), any peculiarities related to the areas departments cover, and inspections-related subject matter information. It was important to involve the agencies in the selection process because it would be their staff who would be operating the feedback mechanism. Their buy-in, support and understanding of how the process works were crucial to the future effective operation of the mechanism. The team also involved representatives from the broader government community—the top hi- erarchy in the process, such as the Ministry of Economy. Their support of the process was key as SIETS and other inspection agencies report to the Ministry of Economy. The key purpose of involving the ministry was to ensure sustainability of the tool from several perspectives:  Strategic: ensuring that senior government officials understand the benefits of the tool for them, support it and ensure the reporting agencies support it too.  Financial: ensuring the government has sufficient resources (envisioned in the budget) to support the tool when the external support from the project ends.  Operational: feedback is not just about listening to businesses but, more importantly, about taking action. Its future functioning depends on the agency’s post-feedback collection ac- tions. Support and oversight of senior officials often help to stimulate such action. The private sector The team consulted with and involved business associations and their members from the very start of the process. The purpose was to understand their needs and expectations from the feedback mechanisms. Businesses are the key players in the feedback process, as they are at the ‘sending’ end of the process. Consulting with them and involving them in selecting the ways they can send their feedback were important in securing their buy-in and support. Learning from the private sector was another important and beneficial aspect in the process. Businesses often gather feedback from their customers, so learning of the ways and tools they use and lessons were helpful in determining which feedback mechanisms could work best. Learning from the banking sector was especially helpful. This sector seemed to be more ad- vanced in customer relationship management and shared some valuable knowledge and les- sons with the team. The Kyrgyz T&C team identified several areas that need to be taken into consideration when selecting a feedback tool. These are summarized in Figure 5. 11 ECA Field Notes for Practitioners Figure 5: Key considerations when choosing which feedback tool to use Private sector Public sector Country Project Tool context context contest context context  Readiness of the  Readiness of the  Overall agenda and  Project capacity  Accessibility private sector to public sector (in pace of reforms, to support the  Usability (how engage in giving this case agencies at least in the area client easy is it to use by feedback (both involved) where the feed-  Does the project businesses and by attitudinal and prac-  Understanding back mechanisms is have enough inspectors?) tical, e.g. computer of what feedback planned expertise to help  Ability to build literacy, mobile mechanisms are  Economic develop- implement the add-ons over time phone/email cov- and how agencies ment – priority areas selected tool?  Time to develop erage) should apply them for economic devel-  Can this exper-  Cost (development  History of the pri-  What mechanisms opment tise be mobi- and support) vate sector provid- have been used be-  Business environment lized on time? ing feedback (how,  Fit-for-pu rpose fore, why they have most burdensome  Does the project how often, with not been success- areas according to  Durability have enough what results). What ful (understanding busrness  Possibility to con- resources (fi- in their view did not of the causes of nect with existing  Business demograph- nancial, human, work failures) similar tools ics (to help decide on time) to dedicate  Private sector role  Agency’s capacity to pilots geography) to implement- in reform process service the select- ing the selected  Public and private – do they take part? ed tool once it is tool? sector computer How active are they launched (finances, literacy, intemet cov-  Can this ex- in defending their people, institutional) erage, mobile phone perience be rights?  Support from other coverage,preferred replicated else-  Private sector involved public means of communi- where? Is this contribution – what sector institutions cation something that resources they are whose decisions will can improve  General government- ready to contribute be crucial for imple- institutional business relations in to the tool develop- menting the tool knowledge on the country ment the subject? 12 May 2017 Considerations for building and operating the feedback tool Selection of the tool is followed by the building and operationalization of the tool. For an IT- based option, irrespective of the degree of complication, an IT contractor is hired to perform the technical work. It is recommended to involve the IT company at this stage of the tool’s selection to understand the technical peculiarities of each considered option. Key considerations when building the tool:  There are two main tasks the selected IT company needs to perform: • Technical – the actual building/ programming of the tool, testing/ piloting it in demo mode, launching and supporting its operation throughout its ‘life’ and potentially upgrade it on an as needed basis • Educational – it is important that IT specialists teach inspectorate staff how to operate the tool and how to analyze the results, as well as share other important knowledge to enable staff to work with the mechanism on their own or with little support from IT specialists.  Before starting to do the programming, there are many questions that need to be clarified and clearly communicated to the IT company. Some of these include: • what the feedback tool is supposed to be doing; • what type of data it should collect, in what way, how often; • how the data will be collated, by whom, who will have access to it; • how the data will be presented; • how the data will be analyzed, how often; • how the users access the tool to do both filling the questionnaire and accessing the re- sults; • how to balance open accessibility and confidentiality; and • what data should automatically be transformed into a publicly available graph what should not.  Questionnaire development. The Kyrgyz team wanted the questionnaire to be short and at the same time meaningful, i.e., to ensure that it measured key aspects of the inspection process. The agreement was to have 16 questions with pre-defined answers for entrepre- neurs to choose from and two open questions seeking any comments or suggestions about the inspections process and the questionnaire. The questionnaire measures the following aspects: • General information about an inspection (e.g., planned vs unplanned); • Inspector’s compliance with legal requirements of conducting inspections; • Performance of inspectors during the inspection (e.g., if an inspector provided consulta- tion in addition to checking); and • Instances of non-compliance of businesses with legislation and their consequences. 13 ECA Field Notes for Practitioners The agency will use the findings and analysis to “name and shame” those inspectorates or re- gional branches that fall short of implementation, thus allowing their bosses to show that they are taking action against the specific district or agency that is shown to be underperforming. Likewise, they will use the data to commend the work of those branches that are performing well and where businesses report satisfaction with their performance. The feedback will also help them to understand what legal requirements businesses do not know/understand, or need more clarity on in order to comply. It enables the agency to design appropriate responses to ad- dress the identified gaps in knowledge and understanding, e.g., develop explanatory brochures or organize training, or simplify the requirements, where appropriate.  It is recommended to assign some people from the agency (in addition to the agency’s IT staff) to work side-by-side with the IT developers. This will reduce total dependency on IT specialists when operating the tool.  A significant portion of time should be allocated for agency staff learning. Agency staff who will be involved in operating the system, as well as all inspectors, should go through relevant training programs to understand how to operate the feedback system, and how to process and analyze the results.  Piloting the feedback mechanism before the launch. When the Kyrgyz T&C team launched the feedback mechanism, the piloting stage lasted for about five months. The team wanted to identify how the tool worked, if there were any technical glitches or issues in accessing and using it by businesses and inspectors. For instance, the team identified that the majority of received responses came from the inspections portal www.proverka.kg. It is too early to say why this happened. The team, together with the inspectorate, are considering a few possible reasons: (i) there is a gap in time from the moment inspections take place to the mo- ment the inspectors report on them in the system after which the inspection is considered ‘closed’ and an email is automatically sent to entrepreneur; (ii) technical glitches in the system when sending an email to entrepreneur, (iii) the inspector made a typo in the email, (iv) the inspector did not ask for an email during inspection, (v) the entrepreneur did not respond to the email sent to him after the inspection. 14 May 2017 Figure 6: Example of how the results are aggregated and displayed – the use of checklists 57 questionnaire were received by the end of May 2017 Received through Received through Received through websites email proverk proverka.kg Total # 57 88% 12% 65% 65 of questionnaires estionnaires Did the inspector use a checklist during inspection? Used Don’t know/ checklists don’t remember 10% 48% Don’t know/ 71% 4% don’t remember 19% Unplanned inspection 48% Did not use checklists Planned inspection Source: www.proverka.kg Promoting the tool once it has been launched. For the tool to work and serve its purpose there is a need to invest in countrywide promotion. The aim is that businesses hear about the tool and know how to use it. It is also important to ensure that the topic of business-to-government feedback remains “live” in business talks and in the media—to maintain pressure on the govern- ment to take action. The promotion strategy can include a mix of direct contact events with businesses and inspectors, outreach through the distribution of promotional materials (leaflets, brochures, posters and videos), mass media and social media—none of which can work alone but can be most effective when used together. This cannot be done by project alone. Col- laborating with business associations, mass media and other development organizations that work in similar areas is crucial. Likewise, government support and involvement, especially of the agency that seeks feedback, are crucial. Their involvement also ensures that they recognize their ownership of the promotion program and do not view it as the project’s program. Thus the presence of senior and middle-level officials at events gives credibility to the efforts and builds the trust among businesses for the tool. Key to promotion is that it should be continuous and it does not stop when the tool is launched. 15 ECA Field Notes for Practitioners Key observations and outcomes One of the key outcomes is that, inspired by the work of SIETS, two more inspectorates have joined the feedback mechanism since its launch. This was one of the team’s intentions and hopes—to use one agency as a test-bed and make a case for others to join. So, the team’s over- arching goal of the exercise was achieved. Gradually, other agencies have started to take an interest and join. At the agency level, first results have also emerged. The quarterly analysis of the operation of the feedback mechanism revealed the following:  Low number of filled in questionnaires received that could be due to lack of knowledge about the tool.  The necessity for agencies to develop and provide specialized training programs for inspec- tors and businesses on legislation requirements related to inspections.  Poor quality of interaction between businesses and inspectors, i.e., low level of service deliv- ery during the inspection process. The first quarterly report demonstrating feedback results was placed on websites of relevant agencies involved in the feedback gathering exercise. Despite low uptake of the feedback mech- anism by businesses registered so far, the agencies involved took the results seriously and de- signed appropriate measures. The Ministry of Economy, the SIETS and the Sanitary Department organized meetings with representatives of business associations in major cities of the country. The purpose of those meetings was to inform businesses about the feedback mechanism and how the process works, as well as to present the first findings. Businesses were invited to com- ment and propose improvements. Based on the above consultations with businesses, the SIETS introduced new performance management criteria of inspectors, in particular, the necessity to provide consolations and train- ing to businesses. The SIETS is investing in the continuous professional development program for its inspectors to improve their service delivery to businesses. The State Metrology Inspec- torate regularly meets with businesses and publishes weekly operational reports. Box 2: How the agency reacts on the received feedback The Sanitary Inspectorate received feedback from entrepreneurs working in district A 2 of the Kyrgyz Republic reporting that head of the Sanitary Inspectorate in this district asked to pay bribes. The Director of the Sanitary Inspectorate request- ed the project team to conduct a follow-up survey with entrepreneurs to verify if bribery cases really occurred in this particular district, and to check if similar cases were registered in nearby districts. A phone survey confirmed incidents in which inspectors had been seeking bribes in district A There were positive surprises too. Some of the survey questions sought to assess cooperation levels among the local sanitary inspectors and entrepreneurs, and the willingness of inspectors to help entrepreneurs and answer their ques- tions. The survey revealed that one inspector in district B regularly consulted entrepreneurs about changes in the legis- lation and helped entrepreneurs to comply with legal requirements via one-on-one meetings. Based on the findings of the survey, the Sanitary Inspectorate decided to conduct an internal analysis of the performance of inspectors working in district A, and to organize a peer-to-peer learning event for the inspectorates at which the in- spector from the district B could share experience about raising the awareness of entrepreneurs and helping them with day-to-day operations. 2 For confidentiality reasons actual names of the districts are not mentioned. 16 May 2017 In November 2016, the Kyrgyz T&C team, together with the Ministry of Economy, the Technical Inspectorate, the Sanitary Inspectorate and the National Alliance of Business Associations orga- nized a conference to discuss the feedback mechanism, its results and future steps. The confer- ence was an opportunity to assess the first 18 months of the feedback mechanism’s operation, discuss the results and the government’s actions in addressing the results, and identify next steps in developing the tool further and spreading the practice to other agencies. The conference was an open forum gathering over 60 representatives of government agencies, the private sector and mass media. One of the important outcomes from the event was the proposal by business- es associations to introduce such feedback mechanisms in all inspectorates in the country, and the subsequent recommendation of the Ministry of Economy to do so by the end of 2017. The Kyrgyz T&C team and the SIETS focused on understanding the reasons for the low up- take of the mechanism by businesses. They conducted the first series of focus group discus- sions with entrepreneurs in three cities, collectively chose the most appropriate communication channels for promoting the feedback mechanism, and discussed potential adjustments to the existing questionnaire. The FDGs revealed that businesses were largely unaware of the feedback mechanism, but supported the initiative, hence the team’s continuous focus on promoting the tool. The project hired a business association to conduct an outreach and communication cam- paign promoting the feedback mechanism throughout the country. The above examples demonstrate that, despite the low level of feedback questionnaires received so far, the SIETS is committed to taking the results into account and making the necessary changes based on the feedback. It is also committed to analyze more deeply any complaints received, as well as commend the good practice of inspectors and instill this good practice and these principles into all branches across the country. These efforts help to build the trust of businesses in the agency and in the feedback mechanism, as they can see that their voices have been heard and their opinion taken into account. The Ministry of Economy, seeing these positive changes, has recommended making business-to-government feedback mechanisms standard practice for all inspectorates across the country. As next steps the Kyrgyz T&C team is planning to accomplish the following:  Assist the Ministry of Economy in introducing the feedback mechanism in other inspection agencies;  Assist the Ministry of Economy in designing and delivering trainings on implementation of the feedback mechanism for these agencies;  Conduct a second series of FDGs with entrepreneurs to measure the impact of the outreach and promotion campaign; and  Assist the Ministry of Economy and the SIETS in analyzing the results of the FDGs and de- signing appropriate responses to address the feedback received. 17 ECA Field Notes for Practitioners Lessons learned The Kyrgyz team identified the two critical dimensions that determine success or failure of im- plementing feedback mechanisms: securing client commitment and ensuring the mechanism is used to close the implementation gap. Securing client commitment Securing client commitment is indispensable for collaboration, especially when disrupting the status quo of agencies and proposing new ways of conducting their work. The following can be helpful in obtaining understanding and commitment from the client. Pick the champion and focus area. Picking a pilot agency and a specific focus (functional area of the agency) to test the new ground helps to focus efforts and minimize resources. This is especially relevant for countries with multiple inspections agencies. Even when selecting one agency, it is recommended to pilot work on one functional area. The idea is to test the mech- anism with the goal of rolling it out on other areas and with the involvement of other agencies. Use both quantitative and qualitative techniques to press your case. Words can be highly persuasive, but when convincing is required then unquestionable facts and evidence trump words every time. Facts and figures strengthen the argument and leave little room for push back. That is why using quantitative research techniques, such as surveys, in addition to qualitative FGDs, is always a winning strategy. Build pressure from the other end of the process. This is about collaborating with private sector allies. Similar to agencies, businesses also need to understand the benefits of the feedback exercise. They need to know their role in the process. Business participation in the process is equally important and businesses should recognize this. Often feedback mechanisms fail due to business inertia and an unwillingness to engage, so engaging them from the very beginning is crucial. The tool’s success depends on country context and environment. It is important to un- derstand and remember that private sector is also a client and securing their commitment and buy-in is crucial. This is especially true at the initial stage of the tool’s development – the private sector should know about it, be involved in the process as much as possible, trust in its effec- tiveness and ultimately use it without fear. This, however, is not always possible to secure, especially in less democratic/ liberal countries. Therefore, it is critical to assess the country context and environment and identify how ‘open’ the private sector is in voicing their opinions in a particular country before offering to launch a feedback mechanism of any sort. Getting the client’s commitment is often like putting a jigsaw puzzle together. The challenges to the team are (i) to have a vision of the final picture while the puzzle is still in pieces on the floor and (ii) to have the patience and determination to achieve the ultimate goal when putting the pieces together. 18 May 2017 Ensuring the mechanism is used to close implementation gaps The ultimate goal of introducing feedback mechanisms is to improve the way something is done, in our case, improve inspections process and ensure that there are no gaps between what the law says and how it is applied in reality. The exercise is thus useless when nothing follows the collection of feedback, when the results are not used to change the situation. The Kyrgyz team learned that the following is helpful in ensuring ‘the measured’ gets done. Simplicity by design. The simpler the mechanism is to operate, the greater the probability that it will be used by businesses and inspectors. Making it easy to analyze the re- sults is also important. The Kyrgyz team wanted to make aggregation and analysis of results as simple as possible for the inspectorate, lead- ing to the decision to have the sys- tem instantaneously add results of each completed questionnaire in real time and present the results in an aggregate form. Learn from the private sector. The private sector can be a valu- able resource and knowledge point, especially, in issues of feed- back gathering and performance management. Private sector busi- nesses are more in the organiza- tional management domain and historically, organizational man- agement is the area most explored and developed by the private sec- tor. Showing that we are open to learn from businesses and, even more so, open to accepting their ideas, will also strengthen their trust in the tool, thus increasing the probability of business making use of it. Promote the tool heavily. One of the key findings from piloting the feedback mechanism was low up- take of businesses, most likely due to their low awareness about the tool. Promotion is another import- ant part of the process: it is crucial that businesses hear about the tool and know how to use it. 19 ECA Field Notes for Practitioners Get the government to publicly declare its commitment to go beyond just the introduc- tion of the tool. The Kyrgyz team organized (or participated in third party) public-private dia- logue-type of events where the business-to-government feedback was always on the agenda, backed by both the project and the private sector participants. It is important to have the agency commit publicly to the use of feedback to change the situation. Once this has become public, it is very difficult for the agency to renege on its declared commitments. Publicize the feedback results. In addition to the need for promoting the tool, the same vigor should be applied to communicating the results of the feedback exercise—from the problems identified through the feedback to the actions taken by the government and results they brought, if any. This information greatly reinforces trust in the tool and demonstrates the government’s commitment to implementing change. 20 May 2017 Conclusion, sustainability and replicability The case study presents just one aspect that contributes to the process of improving regulatory delivery and closing the regulatory implementation gap. Establishment of the business-to-gov- ernment feedback mechanism has not resolved all the implementation failures of inspections legislation in the Kyrgyz Republic. However, it has helped make the inspections process more transparent and clearer. It has made more prominent the role of the private sector in reform design and implementation. It has contributed to shifting mentality and behaviors of the inspec- torate personnel toward businesses, from one of ‘command and control’ to ‘listen, communi- cate and advise’. It has also stimulated the agency to rethink its accountability framework and to focus more resources on competency-based performance management and the continuous professional development systems to improve the quality of service delivery. One of the objectives of this exercise was to start from one agency and to make a case for rep- lication in other agencies with similar mandates. This happened during the piloting stage: in ad- dition to the SIETS the Sanitary Service expressed its interest in joining the feedback mechanism. More agencies joined later. Now the Ministry of Economy has recommended that all inspections agencies adopt the same feedback mechanism as standard practice. The private sector has wel- comed this initiative. This indicates the level of interest for this exercise from both sides—private and public sectors. More importantly, it demonstrates the government’s commitment to contin- ue the work that started with help from development partners, including the World Bank Group. One of the strategies that the team used when developing the feedback mechanism to ensure sustainability was to involve the public sector (the agency’s personnel) in the design and imple- mentation. Capacity-building and training of personnel on how to use the feedback mechanism was central to this work. Therefore, the agency is now equipped with necessary knowledge and skills needed to operate the mechanism. The Kyrgyz experience was the first pilot of implementing an ICT-enabled feedback mechanism in an IDA country in the Europe and Central Asia (ECA) region. The team applied best practice, adapting it to the local environment and learning ‘on the go’. Now this experience can serve a blueprint for other IDA countries with similar environments and contexts. For example, neigh- boring Tajikistan has already expressed its interest in learning from its Kyrgyz peers and plans to introduce a similar feedback mechanism and other ICT tools to improve implementation of inspections legislation in Tajikistan. 21