IDSC TEAM'S WORLD BANK SUPPORTED STUDY TOUR TO PAF BANGALORE ON IMPLEMENTATION OF SOCIAL ACCOUNTABILITY TOOLS IN MONITORING & EVALUATION March 7-11, 2017 Conducted by PUBLIC AFFAIRS FOUNDATION For STUDY TOUR REPORT SUBMITTED TO BY 27th March 2017 i IDSC TEAM'S WORLD BANK SUPPORTED STUDY TOUR TO PAF BANGALORE ON IMPLEMENTATION OF SOCIAL ACCOUNTABILITY TOOLS IN MONITORING & EVALUATION March 7-11, 2017 - STUDY TOUR REPORT - 27th March 2017 IDSC TEAM'S WORLD BANK SUPPORTED STUDY TOUR TO PAF BANGALORE ON IMPLEMENTATION OF SOCIAL ACCOUNTABILITY TOOLS IN MONITORING & EVALUATION STUDY TOUR REPORT CONTENTS Abbreviations ............................................................................................................................... iii Executive Summary....................................................................................................................... 1 1. Background................................................................................................................................ 5 2. Role of Public Affairs Foundation [PAF] ...................................................................................... 6 3. The Study Tour .......................................................................................................................... 7 3.1 PAF’s Objectives....................................................................................................................... 7 3.2 Pre-Study Tour Preparations and Post-Study Tour Deliverables by Participants ........................ 8 3.3 Scope of work .......................................................................................................................... 9 3.4 Suggested Profile of Participants .............................................................................................. 9 4. Summary of Sessions ............................................................................................................... 10 Day 1- Session 1: Inauguration & Opening Remarks ..................................................................... 10 Day 1- Session 2: IDSC - GoE's Think Tank .................................................................................... 10 Day 1- Session 3: Keynote Address - Structure of Governments and Governance in India ............. 13 Day 1- Session 4: Measuring Governance: Public Affairs Index (PAI) ............................................. 14 Day 1- Session 5: Hands-on Simulation Exercises ......................................................................... 19 Day 2 - Session 1: Moraji Desai Residential School (MDRS)........................................................... 20 Day 2 - Session 2: Bangalore Electricity Supply Company [BESCOM] Call Center ........................... 22 Day 2 - Session 3: A Government Official’s perspective on using Social Accountability Tools, and data driven decision making ........................................................................................................ 23 Day 3 - Session 1: M&E from a Government Perspective- Central Government, State Government and Local Government (Decentralisation, Public Expenditure and Budget Tracking)..................... 24 Day 3 - Session 2: Deconstructing Accountability ......................................................................... 24 Day 3 - Session 3: Fiscal Decentralization in India ......................................................................... 25 Day 3 - Session 4: PAISA (Planning, Allocations and Expenditures, Institutions in Accountability) for Panchayat.................................................................................................................................... 26 Day 3 - Session 5: Hands on exercises .......................................................................................... 27 Day 3 - Session 6: Wrap Up .......................................................................................................... 27 Day 4 - Session 1: Welcome, Overview of PAC, and Setting the Context ....................................... 28 Day 4 - Session 2: Citizens Involvement in Monitoring of PMGSY Roads ....................................... 29 (Prime Minister's Rural Roads Scheme) ........................................................................................ 29 Day 4 - Session 3: Climate Change Score Card (CCSC) ................................................................... 30 Day 4 - Session 4: ......................................................................................................................... 31 a) Measuring SDGs progress with a focus on Health, Nutrition and Education .............................. 31 b) From MDGs to SDGs - Implications for India’s Health, Nutrition and Education Sectors ............ 32 Day 4 - Session 5: Improving Consumer Voices and Accountability in Swachh Bharat Mission ...... 34 Day 5 - Session 1: Discussions and Presentations ......................................................................... 35 Day 5 - Session 2: ‘Way Forward’ ................................................................................................. 36 Day 5 - Session 3: Closing Ceremony, Certificates, Group Photos ................................................ 38 Annexure 1: PUBLIC AFFAIRS FOUNDATION [PAF] - A SNAPSHOT - March 2017 ........................... 39 Annexure 2: Pre-Study Tour Preparations and Post-Study Tour Deliverable by Participants ......... 40 Annexure 3: Agenda .................................................................................................................... 42 Annexure 4: Profiles of Invited Speakers and PAF / PAC Resource Persons ................................... 47 Annexure 5: List of Participants ................................................................................................... 54 Annexure 6: Evaluation Forms for Day-wise sessions, and for the Study Tour .............................. 55 Annexure 7: Summary of Participants' Evaluation Scores ............................................................. 57 ii IDSC TEAM'S WORLD BANK SUPPORTED STUDY TOUR TO PAF BANGALORE ON IMPLEMENTATION OF SOCIAL ACCOUNTABILITY TOOLS IN MONITORING & EVALUATION March 7-11, 2017 - STUDY TOUR REPORT - 27th March 2017 Abbreviations AI Accountability Initiative ANSA-AW Affiliated Network for Social Accountability - Arab World APU Azim Premji University BESCOM Bangalore Electricity Supply Company CAPI Computer aided personal Interviews CRC Citizen Report Card CSC Community Score Card CSO Civil Society Organisation EDO Equitable Development Observatory EIA Environmental Impact Assessment GoE Government of Egypt GoI Government of India GoK Government of Karnataka GRM Grievance Redress Mechanism IA Impact Assessment IAS Indian Administrative Service (Government of India) IDSC Information and Decision Support Centre KREIS Karnataka Residential Educational Institutional Society (SWD/GoK) MASAF Malawi Social Action Fund M&E Monitoring and Evaluation MDG Millennium Development Goal MOIIC Ministry of Investment & International Cooperation NGO Non-Government Organisation NHFS National Family Health Survey (GoI) NITI Aayog National Institution for Transforming India, GoI's premier policy Think Tank NRRDA National Rural Roads Development Agency (GoI) ODK Open Data Kit (an open source software for data collection) PAC Public Affairs Centre, Bangalore PAF Public Affairs Foundation PAPI Paper and pencil interviews PET Public Expenditure Tracking RCT Randomised Control Trial PM&E Participatory Monitoring and Evaluation SA Social Accountability SBM Swatchh Bharat Mission (Government of India's national sanitation programme) SDG Sustainable Development Goal SIA Social Impact Assessment SWD Social Welfare Department (GoK) TA Technical Assistance (World Bank) TASAF Tanzania Social Action Fund TTL Technical Team Lead (World Bank) WB World Bank Group iii IDSC TEAM'S WORLD BANK SUPPORTED STUDY TOUR TO PAF BANGALORE ON IMPLEMENTATION OF SOCIAL ACCOUNTABILITY TOOLS IN MONITORING & EVALUATION March 7-11, 2017 - STUDY TOUR REPORT - 27th March 2017 IDSC TEAM'S WORLD BANK SUPPORTED STUDY TOUR TO PAF BANGALORE ON IMPLEMENTATION OF SOCIAL ACCOUNTABILITY TOOLS IN MONITORING & EVALUATION STUDY TOUR REPORT Executive Summary The Study Tour Public Affairs Foundation (PAF), Bangalore hosted a delegation from the Government of Egypt’s Information and Decision Support Center (IDSC) for a study tour from 7th to 11th March 2017 focusing on the implementation of Social Accountability (SA) Tools. The World Bank’s Technical Assistance (WB TA) component supported this “South-South” exchange. The Tour was a follow up to the successful workshop on Participatory Monitoring and Evaluation (PM&E) using Social Accountability Tools, conducted by the World Bank with PAF as the lead trainer at Al Sokhna in Egypt in May 2016. The design of the Study Tour and its primary purpose revolved around supplementing the learning at the May 2016 Workshop, gain firsthand knowledge through field visits, and build capacity at IDSC on institutionalising evidence based research and monitoring of government services and programmes using a range of PM&E and SA tools including the Citizen Report Card (CRC). Study Tour Design PAF designed a customised and practical programme that enabled the visiting team to: 1. Interact with the teams of PAF and its associate think tank / research organisation Public Affairs Centre [PAC] to learn about a) PAF/PAC's experience in the development and implementation of evidence based participatory SA tools and applied social research; and b) understand how these tools have enabled improvements in governance and delivery of public services. 2. Visit Government Departments, meet their senior officials, and get first hand feedback from them on how they have used citizen feedback to effect reforms and service delivery improvements. 3. Learn from some thought leaders and practitioners in the social development sector (independent think tanks and academia) on their perspectives on central, state, and local level planning, implementation, and monitoring of public services and social development programmes using a range of SA tools. All sessions were as interactive as possible and included several hands on exercises that focused on potential specific implementation by participants on return to Egypt. Participants The team of 13 participants who attended represented eight departments of IDSC and three levels. Eight had attended the Workshop in May 2016, eight are proficient in English; six had attended the 2016 Workshop and are proficient in English. 1 IDSC TEAM'S WORLD BANK SUPPORTED STUDY TOUR TO PAF BANGALORE ON IMPLEMENTATION OF SOCIAL ACCOUNTABILITY TOOLS IN MONITORING & EVALUATION March 7-11, 2017 - STUDY TOUR REPORT - 27th March 2017 Summary of the Sessions On Day 1, PAF's Executive Director Mr Ravishankar Rao welcomed the participants and gave a brief orientation to the study tour; the WB TTL Ms Amal Faltas thanked PAF and emphasised that the outcomes expected from the Tour are specific PM&E proposals that could be presented to IDSC; Ms Hebatallah Hamid, International Cooperation Specialist, IDSC gave a brief overview of IDSC’s mission and scope of work. A Keynote Address on the Structure of Governments and Governance in India delivered by PAC's Director Mr G Gurucharan gave an overview of the federal structure of government in India, the distribution of powers between its legislative, administrative and executive arms, and the devolution of legislative powers. He briefly spoke about the (earlier) Planning Commission and the (current, since 2015) National Institution for Transforming India - NITI Aayog (www.niti.gov.in), Government of India's premier policy Think Tank, headed by the Prime Minister, and consisting of various Union Cabinet Ministers and Chief Ministers of all States. India, he said, has made much progress but much more needs to be done, for example, to eradicate poverty, reduce vulnerability, and improve education and health. He spoke about PAC/PAF's Founder (late) Dr Samuel Paul and his immense contributions to the process of measuring and improving governance across the world. Dr C K Mathew, Senior Fellow & Head, Public Policy and Research Group, PAC, gave an introduction to the origins and concept of Public Affairs Index 2016 [PAI] - its need to measure governance, methodology, results, and dissemination, and about the forthcoming PAI 2017. He said that it is perhaps the first time that a non-government institution such as PAC has produced a comprehensive index to measure and compare the performance of all 30 States of India on a wide range of 10 major themes, 25 focus subjects, and 68 indicators. PAI's uniqueness, and its strength, credibility, and reliability, stem from the use of only government published data accessible to citizens. He gave some interesting comparative insights about Egypt and India in terms of Population; GDP; Poverty Head Count; Life expectancy at birth; Geographical area; Urban vs rural, and Administrative structure. Day 2 consisted of field visits planned around providing opportunities for participants to meet and observe various Government institutions and officials that had used Social Accountability Tools to assess their work. First, the team visited a government run residential school (50 km east of Bangalore) with 250 underprivileged boys and girls in classes 6 to 10. The purpose of the visit was to understand the academic, administrative, and infrastructure facilities available in the school through observation and interactions with different stake holders, use this "scoping" information to prepare (on Day 3) an M&E plan to assess the school's facilities, and get hands-on experience of collecting data. The team recorded their observations in hand held Android Tablets using PAF's specifically designed and pre- loaded observation checklist, and had interesting interactions with some boys and girls. In the afternoon, the team visited the Bangalore Electricity Supply Company [BESCOM] Call Centre. Here they witnessed grievance redress in action at the 24/7, 30-seat call centre called Public Grievance Redressal System (PGRS) where citizens’ complaints are received by trained customer care executives, promptly responded to, and resolved. 2 IDSC TEAM'S WORLD BANK SUPPORTED STUDY TOUR TO PAF BANGALORE ON IMPLEMENTATION OF SOCIAL ACCOUNTABILITY TOOLS IN MONITORING & EVALUATION March 7-11, 2017 - STUDY TOUR REPORT - 27th March 2017 The team then met Mr P Manivannan, Secretary, Social Welfare Department (SWD), Government of Karnataka (GoK), to get a government official’s perspective on using SA Tools, and data driven decision making. Mr Manivannan said bureaucrats are both policy makers and implementers, and work under immense pressure. In order to change/improve any policy or process, there needs to be evidence that the existing one is not working as well as it should. This is where agencies like PAF help the government to understand ground realities. He said the study conducted by PAC led to many changes in the functioning of BESCOM. He hoped the government machinery would be more open to listening to citizens' voices through CRCs and other accountability mechanisms that could help them to serve citizens better. In the first half of Day 3, Mr T R Raghunandan, Advisor, Accountability Initiative (Non-Profit Think Tank/Civil Society Organization), explained the concept of citizenship - a legal and political status - conferred by the state by which citizens secure their identity, exercise their rights, and participate in society. He gave an insight into fiscal decentralization in India and how citizens could be empowered to hold their governments responsible for compliance with financial norms by using social accountability tools such as participatory budgeting, public expenditure tracking surveys, social audits, and use of transparency portals. He explained the process and modes of assigning expenditure responsibilities to different levels of government, the constitutional framework for inter-governmental fiscal transfers in India, grants in aid provided to states and their further sharing between the State and Local Governments. In the second half, the team went through hands-on exercises using case study and field visit materials from Day 2 to discuss, plan and prepare M&E studies of SWD run schools and hostels. This covered the scope of the study, its objectives, tools to be used, sampling plan, stakeholders to be interviewed, timelines, and plans for reporting and dissemination. Both groups briefly presented their plans. The PAF team provided its inputs and suggestions. Ms Amal Faltas conducted a debrief session of the three days’ sessions, and hoped that participants would make full use of the learning from the Study Tour to make specific proposals for monitoring and evaluation of development programmes that require attention in Egypt, and that they would present these to IDSC on their return. On Day 4, participants visited the PAC/PAF Campus and spent the day interacting with the staff and guest speakers through seminars on various topics of interest that showcased the portfolio of PAC's current work in different areas such as:  Citizens Involvement in Monitoring of PMGSY Roads (Prime Minister's Rural Roads Scheme) - Mr Basil Liongs, Head - Citizen Action and Support Group, PAC, presented this project for GoI's National Rural Roads Development Authority that enabled citizens to monitor (using simple tools and training), the quality of construction and maintenance of rural roads.  Climate Change Score Card: Mr Arvind Sha highlighted PAC's focus on constructive engagement “to move environment research away from a top -down, extractive and information gathering process to a participatory, bottom-up and inclusive process to ensure sustainable development”. He mentioned the SA tools developed and used by PAC in monitoring and evaluation of environment related issues - Green Manifesto, PAC Waste Tracker (for Solid Waste Management), and Climate Change Scorecard (CCSC); he conducted a hands-on exercise for using the CCSC in Egypt. 3 IDSC TEAM'S WORLD BANK SUPPORTED STUDY TOUR TO PAF BANGALORE ON IMPLEMENTATION OF SOCIAL ACCOUNTABILITY TOOLS IN MONITORING & EVALUATION March 7-11, 2017 - STUDY TOUR REPORT - 27th March 2017  Dr Shreelata Seshadri, Azim Premji University, Bangalore spoke on “From MDGs to SDGs - Implications for India’s Health, Nutrition and Education Sectors”. Using data on poverty, malnutrition and stunting among children in India, from the National Family Health Surveys (NFHS), she highlighted the importance of good, reliable data in measuring progress of a nation, whether in the MDGs phase, or the current SDGs.  Improving Consumer Voices and Accountability in Swachh Bharat Mission [SBM(G)]: Dr Meena Nair, Head - Participatory Governance Research Group, PAC, gave a brief overview of this GoI programme; the study conducted by PAC and PAF across two large Indian states (Odisha + Tamil Nadu) over a three year period (2014 -17) on behalf of the Bill & Melinda Gates Foundation; the SA tools used (CRC, CRC+, CSC); the study objectives; and the steps taken to achieve them. She highlighted (with examples) some of the key themes and learning from the study, which she said could never have been collected, analysed or interpreted from routinely compiled statistics, data or opinions. Dr Meena spoke about the need for advocacy and briefly covered some of the advocacy initiatives undertaken. On Day 5, based on the visit to a school on Day 2, the two IDSC groups continued discussions and presentation of their proposals for an M&E study of schools/hostels. PAF team provided inputs on how they could make their studies more comprehensive and robust. PAF then briefly presented the actual questionnaires used in their ongoing study; these helped the IDSC team to understand the nuances of questionnaire design and sampling. Using PAF's template, participants presented a basic structure of the Study Tour Note that they would place before the IDSC management for consideration. These covered the potential sectors and programmes for which monitoring and evaluation could be piloted, and the SA tools that could be used. PAF team provided inputs to the teams on how they could refine their Notes, after the teams return to Egypt, to make their proposals more comprehensive and robust. Way Forward: Mr Ravishankar Rao summarised the learning from the Study Tour for both IDSC and PAF and presented his views on the way forward. He encouraged participants to share what they had learnt with IDSC's management, and through them, with their colleagues. All members from the IDSC team were awarded "Certificates of Participation". The IDSC team appreciated and thanked PAF and WB for organising and supporting this very useful, productive, and enjoyable Study Tour. Participants evaluated each session on all five days, and the overall Study Tour. All thirteen gave a rating of 4 or 5 (out of a maximum of 5) to the Study Tour on design, content and facilitation by the trainers / PAF. 4 IDSC TEAM'S WORLD BANK SUPPORTED STUDY TOUR TO PAF BANGALORE ON IMPLEMENTATION OF SOCIAL ACCOUNTABILITY TOOLS IN MONITORING & EVALUATION March 7-11, 2017 - STUDY TOUR REPORT - 27th March 2017 IDSC TEAM'S WORLD BANK SUPPORTED STUDY TOUR TO PAF BANGALORE ON IMPLEMENTATION OF SOCIAL ACCOUNTABILITY TOOLS IN MONITORING & EVALUATION STUDY TOUR REPORT 1. Background1 The Information and Decision Support Center (IDSC) is Egypt’s premier policy and strategy think tank that reports directly to the Egyptian Cabinet of Ministers and is critical in conducting evidence- based research, providing policy advice, and influencing the focus and implementation of Government of Egypt (GoE) programs. At the request of the GoE, the World Bank (WB), through a grant2 from the MENA Multi-Donor Trust Fund (MDTF), is providing Technical Assistance (TA) to IDSC to promote a new social contract for the Egyptian people that empowers and engages citizens and civil society actors to become equal partners in the policy-making process and furthers Egypt’s democratic transition. The two specific elements of the IDSC’s work that will be supported are: (i) The Equitable Development Observatory (EDO): This focuses on poverty monitoring and targeting in relationship to the GoE's priority social protection programs for the poor. (ii) Participatory Monitoring & Evaluation (M&E): This component will seek to establish structured mechanisms to engage citizens in the process of assessing different government policies, programs and activities. This will build on existing IDSC work including the systems of observatories (eg: the community observatory), opinion polls and field surveys and leverage IDSC’s mandate to promote active citizenship and institutionalized channels for participation and social dialogue. The combination of beneficiary feedback and evidence-based decision making for achieving stronger results will strengthen the GoE’s efforts to reduce poverty and boost shared prosperity. Citizens’ engagement in the assessment of policies and programs of the government will work to strengthen the sense of social accountability and will help in creating channels of trust between the government and citizen. For decision makers, this component will serve in following up the progress achieved in such programs and policies, encourage learning, help in studying reactions and extract results and lessons learned to inform the decision making process for improving the programs further. The combination of beneficiary feedback and evidence-based decision making for achieving stronger results will strengthen the GoE’s efforts to reduce poverty and boost shared prosperity. The TA is a WB executed activity covering two main components as follows:  Component 1: Diagnostic Analysis and Capacity Building: In supporting the two elements of the IDSC (as above), WB will undertake a series of diagnostic exercises to assess the functioning of the EDO and current M&E systems and identify its strengths and weaknesses. Based on this, WB will undertake a series of training and capacity building activities around 1 From a Concept Note provided by J. Singh, The World Bank Group, via email, August 17, 2015. 2 MDTF Grant P155297 - “Enhancing Participatory Monitoring and Community Engagement in Egypt”. 5 IDSC TEAM'S WORLD BANK SUPPORTED STUDY TOUR TO PAF BANGALORE ON IMPLEMENTATION OF SOCIAL ACCOUNTABILITY TOOLS IN MONITORING & EVALUATION March 7-11, 2017 - STUDY TOUR REPORT - 27th March 2017 these themes based on international best practice with monitoring of social protection programs and participatory M&E methodologies, which can be adapted by the IDSC for the Egyptian context. Specifically, WB will introduce IDSC to internationally recognized participatory M&E tools such as the Citizen Report Card (CRC) that can be applied for assessing the quality of service delivery in social sectors. Outputs from the component will include research studies, guidance materials and policy notes, as well as a series of training workshops and seminars (both face-to-face and via VC). These efforts would in turn assist in developing a revamped participatory M&E system for the IDSC.  Component 2: South-South exchange and knowledge support: The second component will finance a range of knowledge and technical support activities. This will include connecting the IDSC to other international think-tanks engaging in social accountability related work (such as the Public Affairs Foundation in Bangalore, India) through a range of virtual as well as physical south-south exchanges. As part of the knowledge sharing under this program, a series of experience sharing workshops covering various tools of participatory M&E and Social Accountability such as CRCs (among others) and study tours to gain firsthand knowledge will be undertaken. 2. Role of Public Affairs Foundation [PAF] PAF is a knowledge‐based service organisation committed to improving the quality of governanc e by providing advisory support and customised knowledge products to a wide range of clients in the public arena. PAF’s strengths lie in its rich experience and ability to conceptualise, plan, and design fact-finding projects; collect, analyse, interpret and disseminate authentic and comprehensive user feedback; and monitor processes and evaluate interventions. Since it was established in 2003, PAF has completed numerous social research projects in India and overseas using social accountability tools such as Citizen Report Cards (CRCs) and Community Score Cards (CSCs). With sound and well tested methods, and emphasis on quality, PAF has been able to deliver reliable assessments, inputs and insights into service delivery reform based on citizen feedback while establishing the Foundation's credibility in the development sector. PAF has been constructively collaborating and partnering with Municipal Corporations, State and Central Governments in India, International Donors (for projects in India and many other countries), and Civil Society Organisations to "close the citizen feedback loop" by hearing the voices of citizens and then helping these organisations to listen and act upon the feedback they receive. PAF has been involved intensively and directly in the CRC process with varied contexts in several countries, covering almost all major public services. Based on this rich experience, PAF has been implementing CRCs (see PAF's Snapshot in Annexure 1), conducting training workshops on CRCs in India and many other countries (for WB/IDSC in Egypt in 2016; ANSA-AW in Jordan in 2015 for participants from 7 Arab Countries - Egypt, Jordan, Lebanon, Morocco, Palestine, Tunisia, Yemen; MASAF in Malawi in 2015-16; for R4D in Uganda in 2013; and TASAF in Tanzania in 2012). PAF has also been facilitating study tours for team of Government officials (TASAF & MASAF to Bangalore in 2012) to build capacities on the use of CRC methodologies within client organisations - governments and civil society - to help develop internal capabilities for better governance and delivery of services. These workshops and study tours are custom designed for clients and aim at giving participants a conceptual as well as hands on knowledge on the Citizen Report Card approach. Participants at the workshops and study tours have been from government and civil society, from one or several countries, and from one or several services. 6 IDSC TEAM'S WORLD BANK SUPPORTED STUDY TOUR TO PAF BANGALORE ON IMPLEMENTATION OF SOCIAL ACCOUNTABILITY TOOLS IN MONITORING & EVALUATION March 7-11, 2017 - STUDY TOUR REPORT - 27th March 2017 3. The Study Tour The Technical Team Lead (TTL) at World Bank's Cairo Office is in charge of executing the TA (mentioned in the "Background" above). The TTL and her team closely coordinated with IDSC and PAF to support the Study Tour to Bangalore, and enabled PAF to execute it. A team of 13 IDSC Staff attended. 3.1 PAF’s Objectives This Study Tour was a part of the South-South exchange of information and knowledge sharing in the practice of participatory M&E and Social Accountability tools. PAF designed a customised and practical programme that enabled the visiting team to: 4. Interact with the teams of Public Affairs Foundation [PAF] and its associate think tank / research organisation Public Affairs Centre [PAC]. PAC and PAF shared with participants their experience (of 36+ years combined) in the development and implementation of evidence based participatory Social Accountability (SA) tools and applied social research. They demonstrated how these tools have facilitated better understanding and enabled improvements in governance and delivery of public services to citizens across a wide spectrum of sectors and geographies in India and other developing countries. 5. Visit Government Departments [Social Welfare (SWD), Bangalore Electricity Supply Company (BESCOM)], meet their senior Officials, and get first hand feedback on how they have used citizen feedback to effect reforms and service delivery improvements. 6. Learn from some thought leaders and practitioners in the social development sector such as independent think tanks [Accountability Initiative (AI)] and academia [Azim Premji University (APU)] on their perspectives on central, state, and local level planning, implementation, and monitoring of public services and social development programmes using a range of SA tools such as CRCs, CSCs, PET, IA, GRM, and others. All sessions were as interactive as possible and included several hands on exercises that focused on potential specific implementation by participants on return to Egypt. The primary purpose of the Study Tour was to supplement the learning at the May 2016 Workshop & Clinic, gain firsthand knowledge, and build capacity at IDSC on institutionalising evidence based research and monitoring of government services and programmes using a range of participatory M&E and Social Accountability tools including the Citizen Report Card. The overarching objectives of the Study Tour were to create a deeper interest among the participants to learn more about each of the SA tools; enable them understand how these tools can complement ones that they are already familiar with; have an understanding of how to use the more comprehensive set of tools to enable them provide IDSC's stakeholders - the Cabinet and various Ministries - with more robust, evidence (users' voice) based research that will help inform policy, enable improvements in service delivery, and strengthen governance, in their own contexts; and build capacities to train others in IDSC (and in the client Ministries, if required) to institutionalise evidence based research using the entire range of tools. 7 IDSC TEAM'S WORLD BANK SUPPORTED STUDY TOUR TO PAF BANGALORE ON IMPLEMENTATION OF SOCIAL ACCOUNTABILITY TOOLS IN MONITORING & EVALUATION March 7-11, 2017 - STUDY TOUR REPORT - 27th March 2017 3.2 Pre-Study Tour Preparations and Post-Study Tour Deliverables by Participants Based on PAF's learning at the Workshop & Clinic in Egypt in May 2016, and their interactions with a very knowledgeable and enthusiastic group of participants from IDSC and MOIIC, the Study Tour Agenda covered specific sectors that would be of interest to the visiting team. To make the Study Tour as focused and effective as possible, PAF had requested IDSC and all participants to make preparations in three stages:  prior to their arrival in Bangalore - recap learning at the May 2016 Workshop in Egypt and commence planning the SDGs, sector/s, and program/s in which to implement PM&E studies  while in Bangalore - each participant should (a) interactively discuss, in each session, issues that can aid implementation of PM&E study/s on their return, for the specific sectors / programs chosen by them; and (b) create and discuss a suitable structure / implementation plan prior to departure from India  on return to Egypt - refine the structure / plans prepared as above and present their specific proposal/s to the concerned stakeholders The "Pre-Study Tour preparations ('homework') for Participants" and "Study Tour Note" are given in Annexure 2. 8 IDSC TEAM'S WORLD BANK SUPPORTED STUDY TOUR TO PAF BANGALORE ON IMPLEMENTATION OF SOCIAL ACCOUNTABILITY TOOLS IN MONITORING & EVALUATION March 7-11, 2017 - STUDY TOUR REPORT - 27th March 2017 3.3 Scope of work PAF conducted and facilitated the Study Tour covering the following scope of work: 1. Planned the agenda and schedule in consultation with WB TA team, and organised, managed, and facilitated all activities. 2. Put together an internal (PAF / PAC) and external panel of speakers who shared their experience in using SA tools to assess implementation and monitoring of government programmes; most sessions had hands on exercises and interactive discussions that were linked to possible practical applications in the Egyptian context; hence most speakers and presenters were also "trainers". 3. Organized field visits to government offices / agencies, interacted with officials, and discussed the outcomes of using participatory monitor mechanisms and citizens feedback. 4. Materials used in all sessions were compiled and provided in pen drives to participants. 5. Hosted the visiting team and took care of them from arrival (on 7th March till departure on 12th March 2017 [ie: airport transfers, hotel accommodation, breakfast and lunch, conference proceedings, local transport for field visits]. 6. Overall: PAF provided guidance and support to the IDSC team to develop  notes that reflect their learning in Egypt (2016) and India (2017)  specific action plans for implementing their learning to benefit IDSC in improving public service delivery and strengthening governance in their own contexts The day-wise Agenda is Annexure 3; brief profiles of the external (invited) speakers and PAF / PAC resource persons are in Annexure 4. PAF designed the content so as to make it as focused as possible to meet the objectives of the study tour. 3.4 Suggested Profile of Participants For optimum effectiveness and utility to participants, IDSC, and WB, PAF had suggested (Annexure 5) that the group size should be about 10 to 15 persons; that the team should consist of a good mix of personnel from different levels and relevant inter-related functions of IDSC; that they should be involved in monitoring and evaluation of Government of Egypt's social development programmes and delivery of public services; and have proficiency in English. The team of 13 participants who attended represented eight departments: Policies Monitoring & Evaluation, Information Analysis, Information Resources, International Cooperation, Quality, Public Opinion Poll Center, E-Complaints System, and Population & Sustainable Development; and three levels: Managers, Researchers, and Specialists. In this team  8 (62%) had attended the Workshop in Al Sokhna in May 2016  8 (62%) are proficient in English  6 (46%) had attended the 2016 Workshop and are proficient in English IDSC TEAM'S WORLD BANK SUPPORTED STUDY TOUR TO PAF BANGALORE ON IMPLEMENTATION OF SOCIAL ACCOUNTABILITY 9 TOOLS IN MONITORING & EVALUATION March 7-11, 2017 - STUDY TOUR REPORT - 27th March 2017 4. Summary of Sessions Day 1: Tuesday, March 7, 2017 Day 1- Session 1: Inauguration & Opening Remarks Mr Ravishankar Rao, Executive Director, PAF warmly welcomed all participants, thanked IDSC and WB for supporting and facilitating the Study Tour, wished the visiting team a productive week of learning and sharing about research in action, and hoped they would have an enjoyable stay in Bangalore. After a brief background of PAF, he gave them an overview of the Tour programme and Agenda and outlined the objectives of the Tour. Ms Amal Faltas, Senior Social Safeguards Specialist, World Bank thanked Mr Ravishankar Rao and the PAF team for organising the event at short notice. She welcomed participants, highlighted the importance of social accountability as an emerging aspect of governance in Egypt, emphasised the commitment of IDSC (and WB) to build internal capacities to provide evidence based research to the stakeholders they serve, and emphasised that the outcome expected from participants from the Tour are specific / practical study proposals that they could present to IDSC on return. Day 1- Session 2: IDSC - GoE's Think Tank Ms Hebatallah Abdelhamid International Co-operation Specialist, IDSC Ms Heba made a well-structured presentation to give PAF (and PAC) an overview of the GoE's think tank. She outlined IDSC's mission to incubate leading consultants, experts, advisors and researchers in government and academia in Egypt leading to independent, high quality research, policy recommendations and analysis on a full range of public policy issues. The main aim is to bridge the advocacy gap that results from integrating expert knowledge into the evidenced based decision making process. IDSC TEAM'S WORLD BANK SUPPORTED STUDY TOUR TO PAF BANGALORE ON IMPLEMENTATION OF SOCIAL ACCOUNTABILITY 10 TOOLS IN MONITORING & EVALUATION March 7-11, 2017 - STUDY TOUR REPORT - 27th March 2017 Ms Heba summarised IDSC's strategic objectives - to bring about innovative and practical solutions to Egypt's reform and developmental challenges; disseminate findings and views through a regular flow of publications and public events to create awareness; engage societal partners on policy issues of relevance to the Egyptian Citizen; and develop regional and international partnerships that result in the adoption of international best practices in the government planning process and operations. She summarised IDSC's quantitative and qualitative research tools and policies, and monitoring and evaluation research projects that include GIS mapping to graphically display data, to provide decision makers with evidence based policy recommendations. IDSC TEAM'S WORLD BANK SUPPORTED STUDY TOUR TO PAF BANGALORE ON IMPLEMENTATION OF SOCIAL ACCOUNTABILITY 11 TOOLS IN MONITORING & EVALUATION March 7-11, 2017 - STUDY TOUR REPORT - 27th March 2017 She briefly covered the roles and objectives of Energy Efficiency Unit, Management of Crises and Disasters, Central Operational Room, and Presiding National Committees of IDSC. Ms Heba highlighted how the E-Complain System facilitates communication between the citizens and decision-makers about the citizens’ concerns. She said that IDSC is engaged in monitoring and evaluating a wide range of policies, programs and activities. The M & E process covers three steps - monitoring and evaluation of various policies, programs, and projects to assess the impact of specific governmental policies; periodic follow-up of the government statements to identify, prioritize and raise public debates on critical issues; and tracing Egypt’s development in the international reports’ context. Regional Integration, to share experiences in decision support with neighbouring countries, and Regional and International Partners for social development, were also mentioned. For PAF and PAC, this presentation about IDSC's wide range and large scale of work was a very useful learning. IDSC TEAM'S WORLD BANK SUPPORTED STUDY TOUR TO PAF BANGALORE ON IMPLEMENTATION OF SOCIAL ACCOUNTABILITY 12 TOOLS IN MONITORING & EVALUATION March 7-11, 2017 - STUDY TOUR REPORT - 27th March 2017 Day 1- Session 3: Keynote Address - Structure of Governments and Governance in India Mr G Gurucharan, IAS (Retired) Director, Public Affairs Centre Governments and Governance Mr Gurucharan welcomed everyone to India and to PAF/PAC, and hoped that their learning from the Study Tour would benefit them, IDSC, WB, and the citizens of Egypt. He gave an overview of the positive social changes being witnessed in India and Egypt. He said that such Study Tours enable exchange of useful ideas and relevant information and hoped that PAF/PAC in India and IDSC in Egypt, along with the World Bank, can learn from each other and build a long term partnership. The aim, he said, should be to treat social accountability as the big picture, and not merely at a programme or department level. He emphasized the need to share relevant tools from both countries and use them for better governance. He said India and Egypt are somewhat similar in terms of the democratic political structure, development, poverty ratio, human development index, the young populations with hope and aspirations of jobs and a better life. Mr Gurucharan gave an overview of the federal structure of government in India as mandated by the Constitution, the distribution of powers between its three main arms - legislative, administrative and executive, and the devolution of legislative powers to the states through Union, State, and Concurrent lists. Planning He briefly outlined the role of the (earlier, since 1950) Planning Commission and the (current, since 2015) NITI Aayog, the National Institution for Transforming India. NITI Aayog (www.niti.gov.in), he explained, is a premier policy Think Tank of the Government of India which focuses on national development priorities, sectors and strategies with the active involvement of States. It is headed by the Prime Minister, and consists of various Cabinet Ministers and Chief Ministers of all States. Development in India and Egypt India, he said, has made much progress but it is still work-in-progress. Much more needs to be done to eradicate poverty, reduce vulnerability, and improve education and health. Mr Gurucharan spoke about PAC/PAF's Founder (late) Dr Samuel Paul and his immense contributions to the process of measuring and improving governance across the world. He introduced PAC's group leaders and outlined their work. He highlighted the importance of partnering with government for effecting improvements in governance and achieving scale to bring about the desired impact. Challenges and Possible Solutions Mr Gurucharan said that large parts of the economy are run by the state, sometimes less effectively than desired or possible, while many aspects are managed more efficiently by the market. These and other imbalances create many barriers to India's growth. He said the need is to strike a meaningful balance between the activities of the market and the state. He made a historical reference to the Indian Civil Service (ICS), the administrative bureaucracy that was created to solely serve and further the political and economic interests of (the then) British rulers. After India's independence in 1947, even though the ICS transformed into the Indian Administrative Service IDSC TEAM'S WORLD BANK SUPPORTED STUDY TOUR TO PAF BANGALORE ON IMPLEMENTATION OF SOCIAL ACCOUNTABILITY 13 TOOLS IN MONITORING & EVALUATION March 7-11, 2017 - STUDY TOUR REPORT - 27th March 2017 (IAS), the difference was unfortunately only in form, not in content. There was no role for participatory governance in a very large country with an extremely diverse, multi-ethnic, multi-lingual population, that urgently required unification and a common market. Mr Gurucharan suggested that we should view development from the perspective of Human Development. India's path to prosperity remains fragile. Many pockets of India still do not have access to education, health and sustainable income, and along with inequality and poverty, are hindering growth. Due to the extreme diversity of our people, there is a need to have local, regional and population-specific solutions to the many problems being faced, along with improving skills and capacity building at all levels in order to improve governance. He spoke of some of the direct protection / safety nets offered by the Government for the poor and vulnerable such as pensions, subsidies, direct transfer of funds, and subsidized food, to improve social security. He briefly described PAC's collaborative project with Governments and NGOs in seven Indian states to train local communities and involve them in monitoring the construction / maintenance of rural roads) to help citizens participate in the creation of assets that serve them better. Mr Gurucharan concluded by saying that evidence based policy decisions and are the best way forward for all stakeholders. Combined with this, it is essential to develop metrics that regularly measure performance, review outcomes, and evaluate impact of policies and programmes. He wished everyone a pleasant and enjoyable stay in Bangalore, and welcomed them to visit us again. Participants' assessment of this session (Scale 1 = lowest to 5 = highest):  Content: 100% rated it 4 or 5 (useful to very useful)  Usefulness: 91% rated it 4 or 5 Day 1- Session 4: Measuring Governance: Public Affairs Index (PAI) Dr. C.K. Mathew IAS (Retd) Senior Fellow & Head, Public Policy and Research Group, PAC Dr Mathew gave an introduction to the origins and concept of Public Affairs Index 2016 - its need to measure governance, methodology, results, and dissemination, and about the forthcoming PAI 2017. He said that it is perhaps the first time that a non-government institution such as PAC has produced a comprehensive index to measure and compare the performance of all 30 States of India on a wide range of themes, subjects, and indicators. PAI's uniqueness, and its strength, credibility, and reliability, stem from the use of only government published data accessible to citizens. IDSC TEAM'S WORLD BANK SUPPORTED STUDY TOUR TO PAF BANGALORE ON IMPLEMENTATION OF SOCIAL ACCOUNTABILITY 14 TOOLS IN MONITORING & EVALUATION March 7-11, 2017 - STUDY TOUR REPORT - 27th March 2017 He gave some comparative insights about Egypt and India in terms of Population; GDP; Poverty Head Count; Life expectancy at birth; Geographical area; Urban vs rural, and Administrative structure. He also explained how India is geographically, regionally, ethnically diversified. He quoted noted economist J M Keynes, and highlighted how his thoughts are echoed by the World Bank's views on good governance. Dr Mathew spoke of the twelve principals of good governance. As examples, he mentioned Transparency - free flow of information and its access to citizens; and Accountability - a two way process by which citizens should be aware of their right to information; and decision-makers in government, the private sector and civil society organizations should be accountable to citizens. IDSC TEAM'S WORLD BANK SUPPORTED STUDY TOUR TO PAF BANGALORE ON IMPLEMENTATION OF SOCIAL ACCOUNTABILITY 15 TOOLS IN MONITORING & EVALUATION March 7-11, 2017 - STUDY TOUR REPORT - 27th March 2017 He highlighted the complexity of governance in India and quoted the Prime Minister’s call and commitment in 2014 to "maximise governance and minimise government". He provided an overview of distribution of power, as per the Constitution, between the Union and the States through three lists - List I: Union list, List II: State list, and List III: Concurrent list. Dr Mathew explained the objectives and methodology adopted in the construction of PAI using the following three tier matrix of major themes, focus subjects, and key indicators: Sl 1# Theme Sl2# Focus Subjects Sl3# Indicators I Essential 1 T & D Losses (Transmission and Distribution) Infrastructure A Power 2 Per Capita Consumption of Power 3 Households electrified as a % of total 4 % of Households with access to safe drinking water 5 Total Irrigated Area vs Total Agricultural Area B Water 6 Existence of Water Regulatory Commission 7 Existence of Ground Water Regulation Act 8 Surface Roads as a % total Roads Roads & C 9 Road Density per 1000 sq. KM Communication 10 % of households with access to Cell phone 11 No. of Pakka Houses as a % of total Housing D 12 Slum Population as a % of total Urban population 13 % of households with toilets inside premises II Education & 14 Educational Development Index Health 15 ASER Learning Levels A Education 16 No. of Higher Education Colleges per 1 Lakh Population 17 Educational Expenditure as a % of SGDP 18 IMR 19 Average Population served per hospital bed B Health 20 Full Immunization 21 Health Exp as a % of GSDP III Social Public Distribution 22 Allocation and off-take of grain under food security A Protection System act 23 % of Pension beneficiaries of the total above 60 Social Justice and population B Empowerment 24 % of Households with no land 25 Incidence of crime against SC/ST IDSC TEAM'S WORLD BANK SUPPORTED STUDY TOUR TO PAF BANGALORE ON IMPLEMENTATION OF SOCIAL ACCOUNTABILITY 16 TOOLS IN MONITORING & EVALUATION March 7-11, 2017 - STUDY TOUR REPORT - 27th March 2017 C Minority welfare 26 No. of Minority Children given pre metric scholarship Employment 27 Unemployment Rate D 28 % of Manual Casual Labour IV Child & 29 Crime against Children Women 30 Percentage of Child Labour A Child 31 % of Beneficiaries under ICDS 32 Child Sex Ratio 33 % of Malnourished children 34 Women Working Population ratio 35 Utilization of Janani Suraksha Yojna Funds B Women 36 Male Female Literacy Gap 37 Institutional Delivery V Control of 38 Rapes per million population Crime, Law & A Violent Crimes 39 Murders per million population Order 40 Dowry Deaths per million population 41 Custodial Deaths per million population B Atrocities 42 No. of police firings C Policing 43 No. of police personnel per million VI Delivery of A Pendency of Cases 44 Pendency in High Court Justice 45 Pendency in District Court B Vacancies of 46 Vacancy in High Court Presiding Officers 47 Vacancy in District Court VII Environment A Pollution & 48 SPM Emissions Environmental 49 SO 2 Emissions Violations 50 NO 2 Emissions 51 No. of Environmental Violations in the State (Per capita) B Forest Cover 52 Increase/Decrease in Forest Cover C Renewable Energy 53 Renewable Energy as a % of total energy generated VIII Transparency A Transparency 54 Adherence to Section 4 RTI & Public 55 RTPS act legislated or not Accountability 56 No. of Services provided under E Governance plan 57 Lokayukt: Constituted/Bill Passes, Existence of Individual Sites and Chairperson appointed Public B 58 No. of ACB cases Disposed as a % of total Accountability 59 Social Audit under NREGA: % of GP's covered 60 Panchayat Devolution Index Score IX Fiscal 61 Revenue Surplus (% of GSDP) Management A FRBM Indicators 62 Fiscal Surplus (% of GSDP) 63 Debt Burden (% of GSDP) Resource Generation & 64 Per Capita Development Expenditure B Development Expenditure 65 States own tax revenue growth X Economic 66 No. of Industrial Entrepreneurs Memorandum filed Economic Freedom A 67 Ease of Doing Business Freedom 68 Value of MSME's assets (% of GSDP) IDSC TEAM'S WORLD BANK SUPPORTED STUDY TOUR TO PAF BANGALORE ON IMPLEMENTATION OF SOCIAL ACCOUNTABILITY 17 TOOLS IN MONITORING & EVALUATION March 7-11, 2017 - STUDY TOUR REPORT - 27th March 2017 He explained why, apart from the breadth and depth of analysis through the three tier matrix, PAI is unique and different from others indices. To complement the quantitative aspects of the analysis and findings, he said rich, qualitative data through a "sentiment analysis" was also gathered (through news articles and tweets of various stakeholders) to provide feedback on the emotional aspects of development. Dr Mathew presented some key findings of the project. Kerala, among the large states, scored the highest and ranked first; amongst the small states Mizoram ranked the highest. He spoke about the dissemination process whereby the findings were shared with the Union Government and all State Governments. Participants' assessment of this session (Scale 1 = lowest to 5 = highest):  Content: 100% rated it 4 or 5 (useful to very useful)  Usefulness: 100% rated it 4 or 5 IDSC TEAM'S WORLD BANK SUPPORTED STUDY TOUR TO PAF BANGALORE ON IMPLEMENTATION OF SOCIAL ACCOUNTABILITY 18 TOOLS IN MONITORING & EVALUATION March 7-11, 2017 - STUDY TOUR REPORT - 27th March 2017 Day 1- Session 5: Hands-on Simulation Exercises a) "Egyptian Development Index (EDI)" Mr Vivek Divekar, Project Consultant, and Ms Udita Dutta, Programme Officer, PAC held an interactive session to guide participants to simulate creation of an "Egyptian Development Index (EDI)" for IDSC . They outlined a few potential themes relevant to the Egyptian context; each theme had some focus subjects; and each focus subject had selected indicators. Participants (in two teams) were asked to retain theme/s they felt are more important, and cross out theme/s they thought are less important; then they had to assign weights to each focus subject (out of a maximum of 100). Example: For the theme Essential Infrastructure, Team 1 gave a weight of25% for each of the four focus subjects, while Team 2 thought Housing was not important and weighed power (50%), water (30%) and road (20%). Team 1 Sl1# Theme Sl2# Focus Subjects A√ Power – 25% Essential B√ Water – 25% I Infrastructure C√ Roads & Communication – 25% √ D√ Housing – 25% Team 2 A√ Power – 50% Essential B√ Water – 30% I Infrastructure C√ Roads & Communication – 20% √ D× Housing - Nil Participants' assessment of this session (Scale 1 = lowest to 5 = highest):  Content: 91% rated it 4 or 5 (useful to very useful)  Usefulness: 91% rated it a 4 or 5 b) Preparation for Day 2 - Field Visits: (i) Recap of 2016 Al Sokhna Workshop Dr Lalita Pulavarti, Senior Manager - Projects, PAF recalled and summarised the various tools and processes discussed during the May 2016 Al Sokhna Workshop:  CRCs, CSCs, IAs, RCTs, SIAs  Selection of tools, questionnaire design, sampling She asked the group to recall the difference between an "opinion poll" and a "CRC". One of the participants correctly said that opinion poll is about the views or perception about a service even if one has had no experience of the programme or service, while the CRC approach includes feedback only if one has experienced these. She explained that the idea of CRC was the outcome of structured, evidence based information received by service providers (typically government) from independent sources outside the government. Gradually, as government saw the merit of this data driven approach and its several uses, the government itself began asking for independent CRCs to be conducted in various sectors by neutral external agencies. Participants' assessment of this session (Scale 1 = lowest to 5 = highest):  Content: 91% rated it 4 or 5 (useful to very useful)  Usefulness: 91% rated it a 4 or 5 IDSC TEAM'S WORLD BANK SUPPORTED STUDY TOUR TO PAF BANGALORE ON IMPLEMENTATION OF SOCIAL ACCOUNTABILITY 19 TOOLS IN MONITORING & EVALUATION March 7-11, 2017 - STUDY TOUR REPORT - 27th March 2017 (ii) Hand Held Tablets for Data Collection Each participant was then given a hand held tablet which they would need to use during the field visit on Day 2 to gather data from a government school. Mr Narendra, Manager - Projects, PAF walked the participants through different steps of using the tablet for computer aided personal interviews (CAPI). He helped the participants practice a few questions in the questionnaire that had already been loaded on the tablet. Participants' assessment of this session (Scale 1 = lowest to 5 = highest):  Content: 90% rated it 4 or 5 (useful to very useful)  Usefulness: 100% rated it a 4 or 5 Day 2: Wednesday, March 8, 2017 The day consisted of field visits covering three organisations / locations. Day 2 - Session 1: Moraji Desai Residential School (MDRS) This boarding school is in Ramanagara District, 50 km east of Bangalore; has 250 underprivileged children (boys and girls) in classes 6 to 10; is run by KREIS/SWD/GoK. Before proceeding to the school, the team was provided with a brief note to acquaint them with the structure and functioning of KRIES Residential Schools. The purpose of the visit was to: 1. understand the academic, administrative, and infrastructure facilities available in the school through observation and interactions with different stake holders 2. use this "scoping" information for smaller groups from the team to prepare (on Day 3) an M&E plan for assessing SWD run residential schools 3. get hands on experience of collecting data via ODK using hand held Android Tablets On arrival the IDSC team was warmly welcomed by the staff and students, provided a brief introduction to the establishment of the school, student and staff strength, various facilities available to students in the school and the hostel. The team was taken on a short tour of all the facilities of the school - classrooms, science labs, library, physical education and sports, hostels, dining halls, kitchen, bathrooms and toilets, playgrounds, security / surveillance, and shown the monthly hygiene kits given to boys and girls. The highlight of the school tour was a ‘smart class’ in which students who were learning with audio visual aids said that they particularly liked this class because it made learning more fun! IDSC TEAM'S WORLD BANK SUPPORTED STUDY TOUR TO PAF BANGALORE ON IMPLEMENTATION OF SOCIAL ACCOUNTABILITY 20 TOOLS IN MONITORING & EVALUATION March 7-11, 2017 - STUDY TOUR REPORT - 27th March 2017 A 'Smart Class' During the tour the team recorded their observations in the Android tablets using PAF's specifically designed and pre-loaded observation checklist. The team keenly interacted with boys and girls (Class 9) and asked them what they liked about their school/hostel; whether they are happy staying there; do they use all the facilities provided; how often do they meet their parents; what problems they face; and what changes would they like to see in their school. IDSC team interacting with students Most students said they are happy being in the school; that they do not miss their families; that they get to meet them at home during summer vacations (April, May) and festive holidays (October), and when parents visit them at school (once a month); they are satisfied with the facilities provided, and use all of them. Students were not satisfied with the quality of drinking water available in the hostels; they suggested that purification units be placed here so that they need not fetch purified water from the dining halls. Participants' assessment of this session (Scale 1 = lowest to 5 = highest):  Content: 100% rated it 4 or 5 (useful to very useful)  Usefulness: 100% rated it a 4 or 5 IDSC TEAM'S WORLD BANK SUPPORTED STUDY TOUR TO PAF BANGALORE ON IMPLEMENTATION OF SOCIAL ACCOUNTABILITY 21 TOOLS IN MONITORING & EVALUATION March 7-11, 2017 - STUDY TOUR REPORT - 27th March 2017 Day 2 - Session 2: Bangalore Electricity Supply Company [BESCOM] Call Center - Example of a Grievance Redress Mechanism (GRM) in operation Mr Bhanuprakash, Assistant General Manager - Customer Relations, BESCOM Mr Bhanuprakash made a brief presentation about BESCOM (see http://bescom.org/en, and http://bescom.org/en/citizens-charter/) and the Company's operations for distribution of power in Bangalore city and eight surrounding districts of Karnataka State across all sectors (domestic, industrial, and public), serving a 20.7 million people. The team witnessed the grievance redress in action at the 24/7, 30-seat call centre called Public Grievance Redressal System (PGRS) where citizens complaints are received by trained customer care executives, and responded / resolved. Citizens have a wide choice of how they can easily communicate - toll free number, sms, BESCOM's web portal, WhatsApp, Twitter, Facebook, and more. The Call Centre receives about 15,000 calls every day, responds to and resolves about 8,000 calls, and disposes about 2000 calls for issues not related to them. To enhance capacity to respond to about 5000 calls that they are currently unable to handle, they intend to increase the seats from 30 to 45. On registering a complaint, a docket number is instantly generated and sent by sms to the consumer and simultaneously to the concerned area executive engineer who then forwards it to the specific person in field maintenance for action. Subsequently, the customer care executive calls the complainant to check if their problem is satisfactorily resolved. BESCOM's Public Grievance Redressal System (PGRS) Call Centre Participants' assessment of this session (Scale 1 = lowest to 5 = highest):  Content: 91% rated it 4 or 5 (useful to very useful)  Usefulness: 91% rated it a 4 or 5 IDSC TEAM'S WORLD BANK SUPPORTED STUDY TOUR TO PAF BANGALORE ON IMPLEMENTATION OF SOCIAL ACCOUNTABILITY 22 TOOLS IN MONITORING & EVALUATION March 7-11, 2017 - STUDY TOUR REPORT - 27th March 2017 Day 2 - Session 3: A Government Official’s perspective on using Social Accountability Tools, and data driven decision making Mr P Manivannan (IAS) Secretary, Social Welfare Department (SWD) & Chief Project Officer, Karnataka State Highways Improvement Project (KSHIP) [Former Managing Director, BESCOM], Government of Karnataka Background: In 2012-13, Public Affairs Centre had conducted a CRC and provided citizen's feedback to BESCOM; this highlighted not only the many problems faced by citizens but also the poor image of the organisation. With the intention of improving their services and their image, BESCOM initiated several reforms. They began "listening" to citizens through the grievance redress system; the call centre is one of the outcomes. [Also see their Citizen's Charter: http://bescom.org/en/citizens-charter/] Mr Manivannan met the team in the Board Room of BESCOM, and warmly welcomed them. He said bureaucrats are both policy makers and implementers, and work under immense pressure. In order to change any policy or process, there needs to be evidence that the existing one is not working as well as it should. This is where agencies like PAF help the government to understand what is actually happening on the ground in a more nuanced manner. Stakeholders have the choice of viewing evidence on the functioning of the government either positively and negatively. Taking a positive view (as he did) could lead to public policies / processes being suitably modified to improve the delivery of services. As an example he said that the BESCOM study conducted by PAC led to many changes in the functioning of BESCOM. On the other hand government machinery may view CRC findings negatively and choose to ignore them. He however hoped the government machinery would be more open to citizen voices through CRCs and other accountability mechanisms that could help them to serve citizens better. Mr Manivannan shares his thoughts on the practical uses of Social Accountability Tools Participants' assessment of this session (Scale 1 = lowest to 5 = highest):  Content: 100% rated it 4 or 5 (useful to very useful)  Usefulness: 100% rated it a 4 or 5 IDSC TEAM'S WORLD BANK SUPPORTED STUDY TOUR TO PAF BANGALORE ON IMPLEMENTATION OF SOCIAL ACCOUNTABILITY 23 TOOLS IN MONITORING & EVALUATION March 7-11, 2017 - STUDY TOUR REPORT - 27th March 2017 Day 3: Thursday, March 9, 2017 Day 3 - Session 1: M&E from a Government Perspective- Central Government, State Government and Local Government (Decentralisation, Public Expenditure and Budget Tracking) Mr T R Raghunandan (IAS, Retired) Advisor Accountability Initiative (Non-Profit Think Tank/Civil Society Organization) Mr Raghunandan explained the concept of citizenship, which is a legal and political status, conferred by the state by which citizens have certain rights to secure their identity, exercise their rights, and participate in society. Citizenship, he said, can have three approaches that are: State cantered / Citizen cantered / Market cantered He explained all the approaches, and their positive and negative aspects. He elaborated on the Citizen cantered approach which lays emphasis on enabling citizens to participate in the decision making process. He talked about some of the criticisms levelled against the citizen centred approach; the circumstances under which participation is most effective; and how citizen participation contributes to democracy. He said effective participation enables citizens to seek better functioning of the government through transparency and increased accountability. Participants' assessment of this session (Scale 1 = lowest to 5 = highest):  Content: 92% rated it 4 or 5 (useful to very useful)  Usefulness: 83% rated it a 4 or 5 Day 3 - Session 2: Deconstructing Accountability Mr Raghunandan said that in a democracy, accountability (ie: being answerable for efficiently performing ones duties and effectively delivering expected outcomes / results) is possible only when there is commensurate authority. Accountability is a process within the principal - agent relationship where the citizen is the principal on whose behalf the state acts as the agent. The agent (state) can be held accountable by the principal (citizen) for non-performance and seek improvements. There are two routes by which citizens can hold their governments accountable for providing services and for their actions. Conducting CRCs is the short, direct route; however, this route does not necessarily ensure that the government takes corrective actions or becomes more responsible. In the long, indirect route such as elections, there is a considerable time lag and citizens have to wait (five years in India) to hold the government accountable for its actions. He said different forms of accountability - demand and supply side, and horizontal and vertical - should be considered. He presented (see matrix below) the range of government functions, the social accountability processes, and the mechanisms and tools available for monitoring and evaluation. IDSC TEAM'S WORLD BANK SUPPORTED STUDY TOUR TO PAF BANGALORE ON IMPLEMENTATION OF SOCIAL ACCOUNTABILITY 24 TOOLS IN MONITORING & EVALUATION March 7-11, 2017 - STUDY TOUR REPORT - 27th March 2017 Social Accountability Tools Government Function Social Accountability Process Social Accountability Mechanisms & Tools Policies & Plans Participatory Policy Making and  local issue forums Planning  study circles  deliberative polling  consensus conferences  public hearings and citizens’ juries Budgets & Expenditures Budget-Related Social  participatory budget formulation Accountability Work  alternative budgets  independent budget analysis  performance-based budgeting  public education to improve budget literacy  public expenditure tracking surveys social audits  transparency portals (budget websit Delivery of Goods & Social Accountability in the  public hearings Services Monitoring and Evaluation of  citizens’ report cards Public Services and Goods  community scorecards  public opinion polls and citizen’s charters Public Oversight Social Accountability and Public  CSO oversight committees Oversight  local oversight committees Day 3 - Session 3: Fiscal Decentralization in India Mr Raghunandan gave an insight into fiscal decentralization in India and how citizens could be empowered to hold their governments responsible for compliance with financial norms by using social accountability tools such as participatory budgeting, public expenditure tracking surveys, social audits, and use of transparency portals. He explained the process of assigning expenditure responsibilities to different levels of government by modes of delegation, de-concentration, and devolution (see matrix below). He spoke of the various ways in which fiscal transfers occurred, the constitutional framework for inter- governmental fiscal transfers in India, grants in aid provided to states, and how this is further shared between the State and Local Governments. IDSC TEAM'S WORLD BANK SUPPORTED STUDY TOUR TO PAF BANGALORE ON IMPLEMENTATION OF SOCIAL ACCOUNTABILITY 25 TOOLS IN MONITORING & EVALUATION March 7-11, 2017 - STUDY TOUR REPORT - 27th March 2017 Modes of Delegation, De-concentration, and Devolution Passing down of authority and responsibility from a superior to an Specific purpose Delegation agency payments agent to carry out specific tasks. Levels of territorial and functional Unified budget with Deconcentration administration of Centre at lower operational levels carrying out central delegation functions. Powers and authority endowed by higher government level of Inter government government to lower government fiscal transfers; level through statute (including revenue Devolution Constitution) backed by financial assignments resources, & considerable autonomy; accountability largely to citizens. Participants' assessment of this session (Scale 1 = lowest to 5 = highest):  Content: 82% rated it 4 or 5 (useful to very useful)  Usefulness: 82% rated it a 4 or 5 Day 3 - Session 4: PAISA (Planning, Allocations and Expenditures, Institutions in Accountability) for Panchayat Mr Raghunandan explained the initiative of Centre for Public Research (CPR, cprindia.org) called "PAISA for Panchayat" which tries to track the flow of funds from the Centre to the State, and onward to the local bodies. The objective is to understand whether decentralisation translates to actual availability of funds, and thereby services, on the ground. A study in Karnataka found that the maximum flow of funds was used for government staff salaries. Local governments had little knowledge of budget allocations for development; hence there is no real true devolution. One way to resolve these issues, he said, is to disseminate information to local governments. To attempt to bring about changes and improvements in the Egyptian context, a beginning can be made in sectors / areas that are in need of urgent reform, touch the lives of those in most need, are non- controversial, and which require little investment. Suitably communicating plans and actions - internally, within the government, and externally - with citizens, is important. Participants' assessment of this session (Scale 1 = lowest to 5 = highest):  Content: 75% rated it 4 or 5 (useful to very useful)  Usefulness: 67% rated it a 4 or 5 IDSC TEAM'S WORLD BANK SUPPORTED STUDY TOUR TO PAF BANGALORE ON IMPLEMENTATION OF SOCIAL ACCOUNTABILITY 26 TOOLS IN MONITORING & EVALUATION March 7-11, 2017 - STUDY TOUR REPORT - 27th March 2017 Day 3 - Session 5: Hands on exercises a) Using case study and field visit materials from Days 2 & 3 to discuss and prepare an M&E plan Two groups from the IDSC team discussed, planned and prepared a monitoring and evaluation plan to study SWD run schools and hostels based on the field visit to a KRIES school on Day 2. This covered the scope of the study, its objectives, tools to be used, sampling plan, stakeholders to be interviewed, timelines, and plans for reporting and dissemination. Participants' assessment of this session (Scale 1 = lowest to 5 = highest):  Content: 100% rated it 4 or 5 (useful to very useful)  Usefulness: 100% rated it a 4 or 5 b) Presenting M&E Plans for the SWD School Programme Both groups presented their plans. The PAF team provided its inputs and suggestions to fine tune, strengthen, and finalize their proposals for presenting PPT s on Day 5. Participants' assessment of this session (Scale 1 = lowest to 5 = highest):  Content: 90% rated it 4 or 5 (useful to very useful)  Usefulness: 90% rated it a 4 or 5 Group Activity Day 3 - Session 6: Wrap Up Ms Amal Faltas Senior Social Safeguards Specialist, World Bank, Cairo Office Ms Amal said that she had to return to Egypt the following day and would have to (sadly) miss Days 4 and 5. She hoped participants would make full use of the learning from the Study Tour to make specific proposals for monitoring and evaluation of development programmes that require attention in Egypt, and that they would present these to IDSC on their return. The PAF team thanked her and WB for supporting this project in 2016 and 2017, and wished her a safe trip home. IDSC TEAM'S WORLD BANK SUPPORTED STUDY TOUR TO PAF BANGALORE ON IMPLEMENTATION OF SOCIAL ACCOUNTABILITY 27 TOOLS IN MONITORING & EVALUATION March 7-11, 2017 - STUDY TOUR REPORT - 27th March 2017 Day 4: Friday, March 10, 2017 Day 4 - Session 1: Welcome, Overview of PAC, and Setting the Context Mr G Gurucharan, IAS (Retired) Director, PAC Welcome Mr Gurucharan welcomed participants and said that he was honoured by the visit to PAC and PAF by such an important delegation from IDSC and the Government of Egypt. He said that the entire team at PAC and PAF are keen to engage and interact with the IDSC team. He said that the pace of change in national and international development, and the associated problems and challenges, are so rapid that we need to quickly, and continuously, upgrade our capabilities and skills in order to make meaningful contributions to governance and Government. Overview of PAC He briefly outlined PAC's beginnings, the journey so far, and their current work. He said that PAC is a result of the Founder (late) Dr Samuel Paul's clear vision, from a simple idea in 1994 to an established and well recognised institution. Instead of merely complaining about the poor civic services in Bangalore, Dr Paul decided to find workable solutions. He focused on service delivery and what can be done better to improve services to citizens. To do so, he pioneered an important Social Accountability tool - the Citizen Report Card (CRC) - which has been widely accepted and used by many organizations around the world (including the World Bank). A CRC on electricity supply in Bangalore city was conducted by PAC independent of the Government. Dr Paul presented the study to the service provider. The results caused much surprise and shock. The city's citizens had a poor impression about electricity supply; many had to pay bribes to get an electricity connection; complaints made were either not heard or there was no response. The service provider's initial reaction was to dismiss the report as faulty. However, Dr Paul persisted and demonstrated to them that the strength of the report was in its study design and methodology by which robust feedback had been gathered from a scientifically picked and representative sample of citizens. The agency was (finally) convinced, and used the findings to initiate reforms. PAC is a non-partisan, independent, non-profit think tank and is not aligned with any political party or government or business interests. PAC's research is based on evidence and primary data collected from actual users of services, especially the under-served and marginalized sections of the population whose voices are not usually heard. PAC studies and presents its findings in a non-judgmental and neutral manner, covering all aspects of policies and programmes - the good, the average, and the not so good - as experienced by users. These learning helps course corrections and improvements. Mr Gurucharan highlighted that the three key players - the state, the market and civil society - all need to play their respective roles to achieve good governance. Setting the Context (for IDSC) He suggested that, from amongst the many agencies they serve, IDSC can prioritise sectors that need urgent attention (say, health); choose the targeted population (marginalised); pick the geographies (most backward); clearly define the problem (high infant mortality rate); have targeted and time bound strategic objectives (reduction of IMR); specify how and what to measure in the policy / programme IDSC TEAM'S WORLD BANK SUPPORTED STUDY TOUR TO PAF BANGALORE ON IMPLEMENTATION OF SOCIAL ACCOUNTABILITY 28 TOOLS IN MONITORING & EVALUATION March 7-11, 2017 - STUDY TOUR REPORT - 27th March 2017 (reduce infant mortality in the 0 - 3 age group by x% vs the national average of y%); define the qualitative and quantitative indicators and matrices; and stay focused on the key parameters. It is extremely important, he said, to make a distinction between – outputs, outcomes, and impact. Quite often, outputs are measured as indicators of performance, rather than outcomes and impact. As an example, he mentioned that the outcomes of constructing rural roads are access to schools, primary health care centres, cities, while the impact is increased and sustainable livelihoods and income for people who can easily and quickly transport and sell their agricultural produce. To a query from a participant on an example of impact based research by PAC, Mr Gurucharan quoted PAC's collaborative efforts with the Government, local NGOs and citizens of villages / communities to monitor the quality of roads being constructed (and maintained, if already built) under the Prime Minister's National Programme for Rural Roads. The problems faced were poor quality and corruption. Through an innovative approach, citizens were trained (and empowered) to check the quality of the roads being constructed or maintained, caution the contractor, and seek his compliance with required parameters. Successful implementation of this approach in seven states of the country was appreciated by Government's National Roads Authority; PAC is being asked to replicate the community led monitoring of roads in more states across India. In conclusion, Mr Gurucharan wished the visiting team well. He expressed that PAC and PAF look forward to collaborating with IDSC in the future, in any manner that is appropriate, and to having more dialogues and exchanges of ideas and actions that lead to better governance, in India and in Egypt. Day 4 - Session 2: Citizens Involvement in Monitoring of PMGSY Roads (Prime Minister's Rural Roads Scheme) Mr Basil Liongs, Head - Citizen Action and Support Group, PAC Mr Pankil Goswami, Program Officer, PAC Mr Basil explained the need for, and importance of, citizen participation by which an empowered citizen displays greater involvement and responsiveness to public policies, seeks improved services, and these in turn influence outcomes. Positive changes can be brought about in framing and improving policies by collaborating with government and their agencies, constructively engaging citizens, partnering with grassroots organizations, permitting the flow of information, and through regular communication. He said that citizens' general dissatisfaction with the roads in their communities led to the concept of involving them in monitoring the quality of construction (and maintenance thereafter) of rural roads that are meant to serve the citizens, and improve their lives. He presented PAC's project that involved - collaboration with all stakeholders - GoI / NRRDA / SRRDAs, rural communities, local NGOs and contractors, domain experts - design and development of simple tools that ordinary citizens could understand, and use - these had inputs from domain experts and were based on quality parameters set by the government - training of citizens to enable them monitor the quality of roads constructed (and maintained) under the national rural roads scheme, and provide regular, structured reports - presentation of findings to NRRDA and SRRDAs - corrective actions taken by the Government and its agencies based on the findings / learning - establishment of grievance redress mechanisms to address issues identified by citizens IDSC TEAM'S WORLD BANK SUPPORTED STUDY TOUR TO PAF BANGALORE ON IMPLEMENTATION OF SOCIAL ACCOUNTABILITY 29 TOOLS IN MONITORING & EVALUATION March 7-11, 2017 - STUDY TOUR REPORT - 27th March 2017 He said the project was successfully piloted to monitor 70 roads in three regions / states of India. GoI and NRRDA are satisfied with the results and have expanded project implementation to monitor 140 roads in seven states. Participants' assessment of this session (Scale 1 = lowest to 5 = highest):  Content: 92% rated it 4 or 5 (useful to very useful)  Usefulness: 92 % rated it a 4 or 5 Day 4 - Session 3: Climate Change Score Card (CCSC) Mr Arvind L Sha, Environmental Governance Group, PAC Ms Priyanka Agarwal, Environmental Governance Group, PAC Mr Arvind highlighted PAC's focus on constructive engagement “to move environment research away from a top-down, extractive and information gathering process to a participatory, bottom-up and inclusive process to ensure sustainable development”. Some of the social accountability tools developed and used by PAC in monitoring and evaluation of environment related issues are: Climate Change Scorecard (CCSC), Citizen Led EIA, Green Manifesto, and PAC Waste Tracker He focused on the approach and processes involved in applying CCSCs, an innovative SA tool that supports communities with evidence and rationale to prioritize issues on planning adaptation to climate and environment degradation. CCSC considers three important dimensions - Society, Climate and Governance, provides a platform for dialogue with relevant stakeholders based on the knowledge generated, and helps in evidence-based policy decisions. He shared the findings from some of the PAC’s projects. He explained how cognitive science is used to capture uncertain/hazy knowledge, develop qualitative impact models of vulnerability, and link casual relationships between variables. For a better understanding of how CCSCs can be helpful in the Egyptian context, participants were given a hands on exercise to draw Fuzzy Cognitive Maps (FCM), a practical and potentially powerful tool used for anticipatory action research by incorporating multiple stressors for planning. Participants discussed and prioritised food security as the sector in Egypt that is most vulnerable to climate change. They then formed two groups, representing the "farming community" and "government". Each group developed IDSC TEAM'S WORLD BANK SUPPORTED STUDY TOUR TO PAF BANGALORE ON IMPLEMENTATION OF SOCIAL ACCOUNTABILITY 30 TOOLS IN MONITORING & EVALUATION March 7-11, 2017 - STUDY TOUR REPORT - 27th March 2017 an FCMs; they listed all variables that affect food security, identified relationships (negative and positive, if any) between the variables, and assigned a relationship strength value for each, ranging from 0 (no impact) to 1 (very high impact). The maps were then analysed by the PAC team and findings were shared. "Rules/regulations" and "water pollution" were considered as the most important factors hampering food security by the "government" group; the "farming community" group considered crop production, food availability and well-being of the people to be critical. The exercise demonstrated how FCMs can capture the similarities and differences between the priorities and perceptions of various stakeholders, and based on their analyses, how policy options under different scenarios can be generated. Participants' assessment of this session (Scale 1 = lowest to 5 = highest):  Content: 100% rated it 4 or 5 (useful to very useful)  Usefulness: 100% rated it a 4 or 5 Day 4 - Session 4: a) Measuring SDGs progress with a focus on Health, Nutrition and Education Dr C K Mathew, Senior Fellow, PAC Dr K Sanjeev, PAC Dr Mathew outlined India's progress in achieving SDGs in Education (which he and Dr Sanjeev covered), and Health and Nutrition (that Dr Shreelata Seshadri focused on). He talked about India's independence in 1947, the fragmented and diverse education "systems" that prevailed then, the major reforms that led to standardisation across the country, the shifting of education from the state list to the concurrent list by which GoI took over the power to make changes, and the extremely important introduction of the Right to Education (RTE) Act. Despite such landmark changes, he said India’s education system is still in a dismal state. Recruitment and deployment of teachers and school management is marred with political interference; there is great imbalance with rural areas still in dire need of teachers; numbers of children far outweighs the number of schools available; corrective steps have been ad hoc and sporadic. IDSC TEAM'S WORLD BANK SUPPORTED STUDY TOUR TO PAF BANGALORE ON IMPLEMENTATION OF SOCIAL ACCOUNTABILITY 31 TOOLS IN MONITORING & EVALUATION March 7-11, 2017 - STUDY TOUR REPORT - 27th March 2017 Dr Mathew said India’s attempt to achieve the MDGs in education by 2015 have not succeeded. While the MDGs called for universalisation of primary education, India overstepped and included elementary education too, resulting in 2000 in a huge influx of children to a school system that was grossly unprepared; it took India 10 years to build the required resources. He however said that such a move turned out to be a blessing in disguise as it created easy access to education by which there is one primary school within every populated square kilometre, thereby laying a foundation for the future. He said some the problems that adversely impact India’s present education system are poor quality (in both government and private schools), absence of teacher training, exclusion of students in the most impoverished areas, lack of monitoring and evaluation of the education system, and political interference. Dr Mathew pointed out that states in India which are doing well in the social sector are also doing well in the economic sector, he gave the examples of Himachal Pradesh and Sikkim. He wondered if the same was true for the governorates in Egypt, whereby some governorates are more developed than others due to a well developed and robust social sector. Dr Sanjeev presented PAC's ongoing study on ‘Development Transformation in Himachal Pradesh', one of India's states, to examine the reasons behind the exceptional development by the state when compared to other Indian states. The preliminary findings and learning reveal that this could be due to a combinations of the following factors: women’s empowerment, better road connectivity, 100% electrification, committed and transparent Government with accountability and good governance, social cohesiveness and communal harmony, and high public awareness. He said the study is also probing the impact of social movements, civil activism, and good governance on gender development, and the role of women's empowerment in the State's progress. b) From MDGs to SDGs - Implications for India’s Health, Nutrition and Education Sectors Dr Shreelata Seshadri, Azim Premji University, Bangalore Dr Shreelata gave an introduction about health, nutrition and education, and the linkages between them. She mentioned GoI's major initiatives for health data collection through four rounds of the National Family Health Survey (NFHS), with round 4 having been conducted in 2014. She spoke of the high prevalence among children in India of malnutrition, stunting and wasting across rural and urban settings; the effects of poverty on stunting, child development, and school achievement; the prevalence of high maternal mortality rates (MMR); the presence of about 7.5 million low birth weight (LBW) infants in India alone compared to 9.5 million in the rest of the world; and how the vicious circle of all these factors results in malnutrition. Dr Shreelata briefly covered the millennium development goals (MDGs, 2000-15) and whether, or to what extent, these goals have been achieved. She mentioned the transition to the Sustainable Development Goals (SDGs, 2015-30), discussed how these are different (or similar) to the MDGs, and whether, and how, the objectives of the SDGs are likely to be achieved. IDSC TEAM'S WORLD BANK SUPPORTED STUDY TOUR TO PAF BANGALORE ON IMPLEMENTATION OF SOCIAL ACCOUNTABILITY 32 TOOLS IN MONITORING & EVALUATION March 7-11, 2017 - STUDY TOUR REPORT - 27th March 2017 IDSC TEAM'S WORLD BANK SUPPORTED STUDY TOUR TO PAF BANGALORE ON IMPLEMENTATION OF SOCIAL ACCOUNTABILITY 33 TOOLS IN MONITORING & EVALUATION March 7-11, 2017 - STUDY TOUR REPORT - 27th March 2017 Participants' assessment of this session (Scale 1 = lowest to 5 = highest):  Content: 85% rated it 4 or 5 (useful to very useful)  Usefulness: 85% rated it a 4 or 5 Day 4 - Session 5: Improving Consumer Voices and Accountability in Swachh Bharat Mission Dr Meena Nair Head - Participatory Governance Research Group, PAC Dr Meena gave a brief overview of the SBM(G) programme of the GoI; the study conducted by PAC and PAF across two large Indian states (Odisha + Tamil Nadu) over a three year period (2014 -17) on behalf of the Bill & Melinda Gates Foundation; the SA tools used (CRC, CRC+, CSC); the study objectives; and the steps taken to achieve them. She highlighted some of the key themes and learning from the study, which she said could never have been collected, analysed or interpreted from routinely compiled statistics, data or opinions. As examples, she explained how data demonstrates that - beneficiaries who build their own toilets (or have them built under their supervision) have a greater sense of ownership, and tend to use their toilets, as compared to toilets built by contractors which provided little or no sense of ownership, and tend to be less used or not used at all - beneficiary built toilets are of better quality and last longer than contractor built ones - Swachhata Doots (messengers of cleanliness / local information providers) and Rural Sanitary Marts (material providers) are effective, key players whose numbers need to be increased and who need to be better supported in order facilitate and speed up building of toilets - repairs and maintenance of toilets (chart below) are important aspects that also need to be communicated to enable sustainability of the programme Dr Meena spoke about the need for advocacy and briefly covered some of the advocacy initiatives. IDSC TEAM'S WORLD BANK SUPPORTED STUDY TOUR TO PAF BANGALORE ON IMPLEMENTATION OF SOCIAL ACCOUNTABILITY 34 TOOLS IN MONITORING & EVALUATION March 7-11, 2017 - STUDY TOUR REPORT - 27th March 2017 Participants' assessment of this session (Scale 1 = lowest to 5 = highest):  Content: 92% rated it 4 or 5 (useful to very useful)  Usefulness: 92% rated it a 4 or 5 Day 5: Saturday March 11, 2017 Day 5 - Session 1: Discussions and Presentations a) Case study discussion and presentation Based on the field visit to the KRIES school on Day 2, the two IDSC groups discussed amongst themselves and prepared their proposals and plans for monitoring and evaluation of SWD run schools and hostels. These included the scope of their study, objectives, tools to be used for conducting the study, sampling plan, various stakeholders to be interviewed, timelines, reporting and dissemination. The two groups presented their proposals and plans. PAF team provided inputs to the teams on how they could make their studies more comprehensive and robust. PAF then briefly presented the actual questionnaires that had been used to conduct a study of SWD run schools and hostels. These helped the IDSC team to understand the nuances of questionnaire design and sampling. Participants' assessment of this session (Scale 1 = lowest to 5 = highest):  Content: 100% rated it 4 or 5 (useful to very useful)  Usefulness: 100% rated it a 4 or 5 IDSC TEAM'S WORLD BANK SUPPORTED STUDY TOUR TO PAF BANGALORE ON IMPLEMENTATION OF SOCIAL ACCOUNTABILITY 35 TOOLS IN MONITORING & EVALUATION March 7-11, 2017 - STUDY TOUR REPORT - 27th March 2017 Teams presenting their case studies b) Presentation of Study Tour Note - from learning to action Based on a template provided by PAF (see Annexure 2), participants presented a basic structure of the Study Tour Note that they would place before the IDSC management for consideration. These covered the potential sectors and programmes for which monitoring and evaluation could be piloted, and the SA tools that could be used. PAF team provided inputs to the teams on how they could refine their Notes, after the teams return to Egypt, to make their proposals more comprehensive and robust. Participants' assessment of this session (Scale 1 = lowest to 5 = highest):  Content: 100% rated it 4 or 5 (useful to very useful)  Usefulness: 100% rated it a 4 or 5 Day 5 - Session 2: ‘Way Forward’ Ravishankar Rao Executive Director, PAF Mr Ravishankar Rao summarised the learning from the Study Tour for both IDSC and PAF and presented his views on the way forward. He encouraged participants to share what they had learnt on the Study Tour with IDSC's management, and through them, with their colleagues / teams / departments, especially those that had attended the 2016 Workshop in Al Sokhna. Problem solving in the Egyptian context, he said, is a gradual process which requires careful planning. These involve selecting one or two sector/s and programmes which need urgent reforms but are not controversial; understanding the needs and concerns of all stakeholders, and arriving at a common IDSC TEAM'S WORLD BANK SUPPORTED STUDY TOUR TO PAF BANGALORE ON IMPLEMENTATION OF SOCIAL ACCOUNTABILITY 36 TOOLS IN MONITORING & EVALUATION March 7-11, 2017 - STUDY TOUR REPORT - 27th March 2017 purpose; and clearly defining the problems, objectives, and scope of work. Building trust with all stakeholders and obtaining their "buy-in", he said, are extremely important. This is possible when they are aware (and convinced) that the use of Social Accountability tools by unbiased / neutral organisations to independently assess the functioning of government's social development programs and delivery of public services, including gathering feedback from scientifically and randomly sampled citizens (ie: beneficiaries of programmes and users of services), is not aimed at finding fault with or auditing or investigating anyone. Studies must be true mirrors that reflect ground realities - the good, the average, and the not so good aspects of governance and delivery, so that they lead to reforms and improvements. Mr Ravishankar said that governments often do a lot of good work but this is either not publicised, or not "marketed" as effectively as they should be. As a result, citizens do not always get to know the many positive measures that governments take to serve them. Regular, effective, two way, evidence based communication is therefore key to building trust between citizens and governments, and within government's own organisations, in order to get 'buy-in" from citizens and for motivating the government machinery to do better. Mr Ravishankar thanked all participants for visiting India / Bangalore / PAF; IDSC for facilitating the Study Tour; and the World Bank for supporting it. He wished participants and IDSC success in fully benefiting from the learning of the 2016 Workshop and the 2017 Study Tour. PAF, he said, will be happy and willing to collaborate in future too with IDSC and WB in any manner that is appropriate to the Egyptian context and helps the Government and citizens of Egypt. 'The Way Forward' - Mr Ravishankar Rao, Executive Director, PAF Participants' assessment of this session (Scale 1 = lowest to 5 = highest):  Content: 100% rated it 4 or 5 (useful to very useful)  Usefulness: 100% rated it a 4 or 5 IDSC TEAM'S WORLD BANK SUPPORTED STUDY TOUR TO PAF BANGALORE ON IMPLEMENTATION OF SOCIAL ACCOUNTABILITY 37 TOOLS IN MONITORING & EVALUATION March 7-11, 2017 - STUDY TOUR REPORT - 27th March 2017 Day 5 - Session 3: Closing Ceremony, Certificates, Group Photos All members from the IDSC team were awarded "Certificates of Participation". The PAF team wished each of them well, a safe journey home, and success in their efforts with M&E using the full range of SA tools that they have been exposed to, in theory and practice, in the Workshop / Clinic in 2016 and this Study Tour in 2017. The IDSC team appreciated and thanked PAF and WB for organising and supporting this very useful , productive, and enjoyable Study Tour. Participants' assessment of the overall Study Tour (Scale 1 = lowest to 5 = highest): In their overall evaluation of the Study Tour, all thirteen participants gave a rating of 4 or 5 (out of a maximum of 5) to the Study Tour on design, content and facilitation by the trainers / PAF. IDSC TEAM'S WORLD BANK SUPPORTED STUDY TOUR TO PAF BANGALORE ON IMPLEMENTATION OF SOCIAL ACCOUNTABILITY 38 TOOLS IN MONITORING & EVALUATION March 7-11, 2017 - STUDY TOUR REPORT - 27th March 2017 Annexure 1: PUBLIC AFFAIRS FOUNDATION [PAF] - A SNAPSHOT - March 2017 15 KIADB Industrial Area, Jigani Link Road, Bangalore 560105, India T: +91 80 27839918/19/20 E: director@pafglobal.org W: www.pafglobal.org Mission - Partnership For Better Governance The Public Affairs Foundation - a knowledge‐based service organisation - is committed to improving the quality of governance by providing advisory support and customised knowledge products to a wide range of clients in the public arena. PAF’s strengths lie in its rich experience and ability to conceptualise, plan, and design fact-finding projects; collect, analyse, interpret and disseminate authentic and comprehensive user feedback; and monitor processes and evaluate interventions. Social Accountability Established in 2003, PAF has completed numerous social research projects in India and overseas using social accountability tools such as Citizen Report Cards (CRCs) and Community Score Cards (CSCs). With sound and well tested methods, and emphasis on quality, PAF has been able to deliver reliable assessments, inputs and insights into service delivery reforms based on citizen feedback and establish the Foundation's credibility in the development sector. PAF has been constructively collaborating and partnering with Municipal Corporations, State & Central Governments in India, International Donors, and Civil Society Organisations to "close the citizen feedback loop" by listening to citizens voices in order to enable them effect evidence based reforms and improvements. Structure and Management  PAF is registered as a Section 8 Not-for-Profit Company under the Indian Companies Act 2013 (Sec 8, Act of 1956). Its policies and services are guided by an eminent Board of Directors consisting of well-known professionals with a wide range of expertise and domestic/international experience in operational and advisory work.  Padma Shri Dr Samuel Paul [1930-2015], Founder of Public Affairs Foundation, was its Chairman till his demise in October 2015. He was also the Founder of Public Affairs Centre, a Not-for-Profit think tank, known internationally for its work on governance and public accountability, and a former Director of the Indian Institute of Management, Ahmedabad and Advisor to the World Bank, UNDP, and ILO.  Mr Sudhakar Rao, IAS (Retd), former Chief Secretary, Government of Karnataka, is PAF's Chairman since November 2015. He has held several senior positions in Government of India and is an Independent Director on the boards of several well-known companies.  The Foundation’s services are delivered by a highly skilled team of staff and consultants with considerable national and international experience. It is headed by Executive Director Ravishankar Rao, a senior management executive with extensive experience in leading commercial organisations in India and overseas and close association with the non-profit sector. Sectoral Areas of Work Expertise  Social Accountability (SA) Tools  Public Service Delivery Assessment, Impact Assessment, and  Monitoring and Evaluation; Impact Assessment Evaluation of schemes and programmes using Social  Poverty Alleviation, Pro-poor Services, Electricity, Water, Accountability Tools Sanitation, Health, Nutrition, Public Transport, Roads, Food  Workshops on SA Tools to train and build capacities within Security & Public Distribution, Immigration, Social Security, client organisations to help develop internal capabilities for Education, Direct Benefit/Cash Transfers, Social & Physical assessment, monitoring, and evaluation of public service Infrastructure, ... delivery and programmes International Projects  Africa: Kenya, Zanzibar & Mainland Tanzania, Ethiopia, Uganda,  Eurasia: Ukraine, Kyrgyzstan, Tajikistan Senegal, Malawi, Egypt  Middle-East: Jordan  Asia Pacific: Philippines, Vietnam, Laos, Pakistan, Bhutan Projects in India - State Governments & Union Territories  Karnataka  Delhi / NCR  Maharashtra  Madhya Pradesh  Nagaland  Punjab  West Bengal  Bihar  Sikkim  Dadra & Nagar Haveli International Donor Agencies / Clients Served  World Bank  Asian Development Bank  UNDP  Japan Bank For International Cooperation  DFID  Adam Smith International  UNICEF  Transparency International  R4D  Open Society Institute  UN Habitat  TEPAV, Turkey  TIRI  CARE International / ANSA-AW  Google  University of California at Berkeley  Govt of Tanzania, TASAF  Govt of Malawi/MASAF/LDF-TST  Action Aid  Govt of Egypt/MOIIC/IDSC Civil Society Organisations / Non-Government Organisations / Academic Institutions Served  Arghyam  VSO  Public Affairs Centre  Jan Sahas  Impact India Foundation / Dasra  Navajeevana Mahileyara Okkuta IDSC TEAM'S WORLD BANK SUPPORTED STUDY TOUR TO PAF BANGALORE ON IMPLEMENTATION OF SOCIAL ACCOUNTABILITY 39 TOOLS IN MONITORING & EVALUATION March 7-11, 2017 - STUDY TOUR REPORT - 27th March 2017 Annexure 2: Pre-Study Tour Preparations and Post-Study Tour Deliverable by Participants I Pre-Study Tour preparation ("homework") for participants: The following preparations will enable participants to have a more effective study tour: 1) Review the notes / presentations on sessions from the May 2016 workshop on Social Accountability tools and their applicability in participatory M&E. 2) Review the discussions and notes from Day 5 (the last day) of the above workshop (May 26th, 2016) when specific SDGs were chosen and groups worked on applying the SA tools to measurement of the chosen SDGs. 3) Discuss with the World Bank TA team as to which of these SDG/s they would like to choose; and, for the selected SDG/s, which sector(s) will be chosen for implementation of PM&E using the tools discussed (if relevant, in combination with any other tools). 4) Finalise the shortlisted programs chosen for PM&E implementation (2 to 3 programs). II Study Tour Note: A format / structure for the Note to be prepared by participants at the end of the tour is given below. The Note can be refined after the team returns home; however, we encourage the team(s) to prepare the basic structure of the note prior to departure from India. Sessions will be set aside on Day 5 or 6 of the tour for discussing and working on this exercise. Part I: Sectors and Programs Team3 experience: Sectors involved in / experienced in 1) Sectors chosen (in consultation with WB TA?) 2) Whether Sectors chosen fall under specific SDGs; if so, which SDGs 3) Government programs / WB + Government programs associated with each chosen sector 4) [Planned: ____________ ; Existing / Ongoing________________ ] 5) Time period of Implementation of the above program(s): beginning date_____________ , end date_____________________. 6) Geographic area(s) of implementation 7) Ministry / Ministries / Agencies responsible for implementation 8) Other stakeholders (such as WB) associated with the implementation 9) Any other details such as: a. Sub-populations targeted (eg: school going children, adolescents, pregnant women, rural population, etc) b. Program budget c. PM&E Budget d. Interdependencies (such as: project cycle dates, budget approvals, performance-tied grants, etc) An example of a table that can be prepared to aid easy review is as follows: 3 “Team” refers to a) IDSC team; b) MOIIC team; and / or, c) sub groups formed out of the IDSC participants based on sectors / specialties / team affiliation within IDSC. IDSC TEAM'S WORLD BANK SUPPORTED STUDY TOUR TO PAF BANGALORE ON IMPLEMENTATION OF SOCIAL ACCOUNTABILITY 40 TOOLS IN MONITORING & EVALUATION March 7-11, 2017 - STUDY TOUR REPORT - 27th March 2017 Geographic Implementation Status Ministries / areas (eg: Schedule Related (Planned; Agencies / Sector Program(s) country-wide; (estimated dates) SDG Ongoing / Others specific Existing) Start End involved Governorates) 1. i. ii. 2. i. ii. Columns and rows may be added as required Part II: Plan for PM&E Implementation using the SA Tool(s) The following steps in Part II can be discussed and completed during the study tour (Days 5/6), and refined after return to Egypt: 1) Each group (IDSC / MOIIC / IDSC Sub-Teams) to pick a Sector + program from the above list 2) Present justifications for picking the sector + program 3) Present the objectives / goals of the PM&E component of the above program 4) Choose the tools that will be used [this can be done by Day 5 or 6 of the study tour] and a tentative study design (eg: quantitative, qualitative, mixed methods and, type if known) 5) Present justifications / reasoning for the tools selected 6) Mention geographic scope of the PM&E exercise (eg: whether all Governorates, or some selected / sampled). 7) Based on prior experience and / or exercises from the May 2016 Workshop, present a sampling plan and sample sizes for the PM&E exercise. 8) Present an implementation plan for the PM&E component, along with a timeline 9) Discuss in the Note all resources required for the implementation of the PM&E study, including technical, financial and personnel resources; and, how they will be mobilized 10) Discuss whether more than one program can be combined into one PM&E exercise (eg: same sample populations can be involved in measuring two different programs). Part I and Part II together will form the Note for each group / sub group. The multiple notes can be combined into one note if required or kept as separate notes. WB TA to decide on the format of the final note. Any other inputs / additions can be added to the outline above. IDSC TEAM'S WORLD BANK SUPPORTED STUDY TOUR TO PAF BANGALORE ON IMPLEMENTATION OF SOCIAL ACCOUNTABILITY 41 TOOLS IN MONITORING & EVALUATION March 7-11, 2017 - STUDY TOUR REPORT - 27th March 2017 WORLD BANK / IDSC TEAM'S STUDY TOUR TO PAF BANGALORE ON IMPLEMENTATION OF SOCIAL ACCOUNTABILITY TOOLS IN MONITORING & EVALUATION March 7-11, 2017 Annexure 3: Agenda Day 1: Tuesday, March 7, 2017 / Venue: Esquire I - The Chancery Pavilion Hotel Session # Session / Topics Presenter Time - indicative Registration and Seating of Participants 10.30 – 11 am Welcome Ravishankar Rao 11 - 11:10 am Overview of IDSC, WB, PAF and the Study Tour 1 All Participants Introduction of Participants and Speakers 11.10 - 11.30 am + invited Speakers Keynote address: Mr G Gurucharan, IAS 3 Structure of Governments and Governance in Director, Public Affairs 11:30 am - 12:15 pm India Centre, Bangalore Public Affairs Index [PAI]: 4 Dr C K Mathew, IAS 12:15 - 1:00 pm Measuring Governance Senior Fellow, PAC & Preview of Exercise / Discussion on Indicators and 5 Visiting Professor APU 1:00 – 1.15 pm data sources for a hypothetical PAI in Egypt Lunch 1.15 pm – 2.15 pm Dr C K Mathew Exercise / Discussion on indicators and data and the PAC Team - 6 sources for a hypothetical PAI in Egypt 2.15 - 3.30 pm Ms Udita Dutta Q & A and discussion on PAI Mr Vivek Divekar Coffee 3.30 pm – 3.45 pm Expectations and Outcomes from the Study Tour: 7 - Participant expectations from tour All participants / PAF 3.45 pm – 4 pm - Structure, Format and content of Note Preparation for Day 2 (Field Visits): 1) Quick run through of tools discussed during the May 2016 Ain Al Sokhna Workshop: - CRCs, CSCs, RCTs, SIAs 8 All participants / PAF 4 pm – 5.30 pm - Selection of tools, questionnaire design, sampling 2) Case Study Review 3) Introduction to GRM and BESCOM visit 9 Day 1 Evaluation Forms, Wrap up, Day 2 Plans 5.30 pm – 5.45 pm IDSC TEAM'S WORLD BANK SUPPORTED STUDY TOUR TO PAF BANGALORE ON IMPLEMENTATION OF SOCIAL ACCOUNTABILITY 42 TOOLS IN MONITORING & EVALUATION March 7-11, 2017 - STUDY TOUR REPORT - 27th March 2017 Day 2: Wednesday, March 8, 2017 / Venue: Field Visits Session # Session / Topics - indicative Presenter Time - indicative 1 Registration and Assembly 8.00 am Proceed to Field trip locations All Participants + PAF 8.15 am Visit to a Social Welfare Department School/Hostel Field hands-on training [case study material 10 am - 11.30 am 2 discussed on Day 1]. Visit will be to get first hand Led by PAF view of the object of study / evaluation in preparation for hands on study design on Day 3 Proceed to Chancery Pavilion Hotel for Lunch All Participants + PAF 11.30 am – 1 pm Lunch 1 pm – 2 pm Proceed to Vikasa Soudha 2 pm Field Trip: Macro level planning and Micro level Mr P Manivannan, IAS implementation: Central budgets, State budgets - Secretary, Social Welfare 3 implementation and monitoring - GoK State Department (SWD) 2.45 pm – 4 pm Highways Improvement Project (KSHIP), GoK Social and MD, Karnataka State Welfare Department (SWD) Highways Improvement Q & A on above Project (KSHIP) Ms Jayanthi, General Manager / Bangalore Electricity Supply Company [BESCOM] Mr Bhanuprakash, 4 Call Center - Example of a Grievance Redress 4.15 - 5 pm Assistant General Mechanism (GRM) in operation Manager - Customer Relations Distribute Day 2 Evaluation Forms PAF 5 5 pm Proceed to Hotel All Participants + PAF IDSC TEAM'S WORLD BANK SUPPORTED STUDY TOUR TO PAF BANGALORE ON IMPLEMENTATION OF SOCIAL ACCOUNTABILITY 43 TOOLS IN MONITORING & EVALUATION March 7-11, 2017 - STUDY TOUR REPORT - 27th March 2017 Day 3: Thursday, March 9, 2017 / Venue: Esquire I - The Chancery Pavilion Hotel Session # Session / Topics - indicative Presenter Time - indicative 1 Registration + collection of Day2 Evaluation Forms 9 - 9.15 am Mr T R Raghunandan, IAS M & E from a Government perspective – Central Advisor - Accountability Government, State Government and Local 2 Initiative (Non profit 9.15 am – 10.30 am Government (Decentralization, Public Expenditure Think Tank / Civil Society and Budget Tracking) Organization) Coffee 10.30 – 10.45 am PAISA 4 for Panchayats 5 - Tracking Fiscal Devolution to Local Governments - a case study from Kolar district, Karnataka on Decentralization, Mr T R Raghunandan 3 Panchayat Budgets. Continued Discussion on M&E of Panchayat level budgets 10.45 am – 1.00 pm and performance Group Activity – Applicability: Discussion on how 4 today’s concepts can be applied to a department TRR / WB / PAF or agency in Egypt Lunch 1 pm - 2 pm Hands-On Exercise: Using case study and field visit materials from Days 2 & 3, prepare a monitoring / 5 Led by PAF 2 pm - 3.30 pm evaluation plan [team activity] Review of tools, if required; continued from Day 1 Coffee 3.30 pm - 4 pm Team Presentations: on M/E Plan for the Social 4 pm - 4.45 pm 6 All Participants Welfare Department Programs / School visited Led by PAF 7 Discussion and Q&A on the proposals presented 4.45 pm - 5.15 pm 8 Wrap up, Day 3 Evaluation Forms, Day 4 Plans PAF 5.15 pm – 5.30 pm 4 Planning, Allocations and Expenditures, Institutions Studies in Accountability [PAISA] See http://www.accountabilityindia.in/paisa-studies 5 Panchayats = Local Government at the Village(s) level IDSC TEAM'S WORLD BANK SUPPORTED STUDY TOUR TO PAF BANGALORE ON IMPLEMENTATION OF SOCIAL ACCOUNTABILITY 44 TOOLS IN MONITORING & EVALUATION March 7-11, 2017 - STUDY TOUR REPORT - 27th March 2017 Day 4: Friday, March 10, 2017 / Venue: PAC/PAF Campus Session # Session / Topics - indicative Presenter Time - indicative Assembly and Departure from Hotel Visiting Team 8.30 am One Day Workshop with PAF/PAC Mr G Gurucharan, IAS Director Public Affairs Centre Welcome, overview of PAC, and Setting the 1 + 10 - 10.15 am Context Mr Ravishankar Rao Executive Director Public Affairs foundation Coffee (at the Tables) 10.15 - 10.30 am Mr Basil Liongs Citizen Involvement in Monitoring – PMGSY Head - Citizen Action & 2 Road Monitoring by training Citizen groups Support Group 10.30 - 11.15 am (PM's Rural Roads Scheme) & Mr Pankil Goswami, PAC Climate Change Score Card (CCSC) – NGOs / PAC : Think Tanks / Research institutions utilizing the Mr Arvind L Sha Score Card to measure Climate Change and its Ms Priyanka Agarwal 11.15 am – 3 impact; Policy Recommendations to the Sacred:6 12.30 pm Government Ms Mary Philomena, + Exercise Director Lunch 12.30 – 1.30 pm 4 CCSC Exercise Results Same as above 1.30 pm – 2 pm Dr Shreelata Seshadri Azim Premji University Measuring SDG progress with a focus on Bangalore 5 2 - 3.30 pm Health, Nutrition and Education + Dr C K Mathew & Dr K Sanjeev, PAC Coffee 3.30 – 3.45 pm Using SA tools to drive Policy and Improvements in Service Delivery: Some examples: Dr Meena Nair, Head - a. SBM (multiple tools – CRC, CRC+, Case studies Participatory Governance 6 and CSCs), 3.45 pm – 5 pm Research Group, PAC & b. BWSSB (Bangalore Water Supply and Dr Satyajeet Nanda Sewerage Board) Q & A on Tools, Methods and Outcomes Mr G Gurucharan 7 Closing Remarks 5 pm – 5.10 pm Mr Ravishankar Rao 8 Day 4 Evaluation Forms; Day 5 Plans PAF 5.10 - 5.30 pm Departure to hotel Visiting Team 5.30 pm 6 Sacred is an NGO; PAC is a Think Tank IDSC TEAM'S WORLD BANK SUPPORTED STUDY TOUR TO PAF BANGALORE ON IMPLEMENTATION OF SOCIAL ACCOUNTABILITY 45 TOOLS IN MONITORING & EVALUATION March 7-11, 2017 - STUDY TOUR REPORT - 27th March 2017 Day 5: Saturday, March 11, 2017 / Venue: Esquire I - The Chancery Pavilion Hotel Session # Session / Topics - indicative Presenter Time - indicative Registration 9.30 - 9.45 am Team Activity – Discussion and preparation of All Participants 9.45 - 11 am Note outline by each team 1 + Coffee 11 – 11.30 am PAF Presentation of Note outline and Discussions 11.30 am - 12.15 pm Ms Amal Faltas Senior Social Safeguards Wrap up 12.15 pm – 12.30 pm Specialist World Bank Egypt 2 The Way Forward – how to integrate the Tools Mr Rafaat Mahmoud into M&E of government programs and SDG 12.30 - 12.45 pm Director General IDSC goals in Egypt Closing remarks Mr Ravishankar Rao, PAF 12.45 - 1 pm Day 5 + Study Tour Evaluation Forms All Participants 1 - 1.20 pm Ms Amal Faltas 3 Closing Ceremony, Certificates, Group Photos 1.20 - 2 pm Mr Ravishankar Rao Lunch 2 - 3 pm Dates / Venues: March 7th Tuesday, 9th Thursday, 11th Saturday: Esquire I Banquet Hall of Hotel Chancery Pavilion March 8th Wednesday: Field March 10th Friday: Campus of PAF/PAC Note: 1) Timings of sessions / speakers / topics may change due to external speakers' tight schedules and unforeseen events. 2) All participants are requested to attend all scheduled sessions and participate actively to enable effective group learning and hands on exercises Annexure 4: Profiles of Invited Speakers and PAF / PAC Resource Persons [Source for all profiles: websites / Wikipedia / internet / Speakers] Mr P Manivannan, IAS Secretary, Social Welfare Department & Chief Project Officer, Karnataka State Highways Improvement Project Government of Karnataka Mr P Manivannan is an Officer of the Indian Administrative Services. He is currently the Chief Project Officer of the Karnataka State Highway Improvement Project (KSHIP) and holds joint charge as Secretary, Social Welfare Department, Government of Karnataka. He is also a Lieutenant with the 106 Infantry Battalion (Territorial Army), Parachute Regiment, Bangalore. He has earlier served in various sectors / departments, and in different districts of Karnataka. These include: Joint Secretary, Water Supply & Drainage Board; Managing Director, Bangalore Electricity Supply Co Ltd (BESCOM); Commissioner / Deputy Commissioner, Urban Development in various Municipal Corporations / Cities (Hubli/Dharwad, Mysore, Tumkur). Mr Manivannan has been (and continues to be) a champion of Social Accountability tools, most notably Citizen Report Cards. In his various positions, he has used CRCs as a means to using citizen feedback in a data driven approach to decision making, and innovation towards improvement of service delivery to citizens of Karnataka. One of the outcomes of this approach is the Grievance Redress Mechanism (GRM) he set up during his tenure at BESCOM. The Call center under this GRM runs 24/7, takes citizen complaints on any lapses in service delivery, and arranges to address and resolve them. IDSC TEAM'S WORLD BANK SUPPORTED STUDY TOUR TO PAF BANGALORE ON IMPLEMENTATION OF SOCIAL 47 ACCOUNTABILITY TOOLS IN MONITORING & EVALUATION March 7-11, 2017 - STUDY TOUR REPORT - 27th March 2017 Mr T R Raghunandan, IAS (Retired) Advisor - Accountability Initiative T R Raghunandan is Founder and Mentor at Accountability Initiative [AI] a non-profit initiative that promotes accountability for public service delivery by developing innovative models for tracking government programmes, disseminating these to policy makers and citizens, and researching how to strengthen accountability for improved service delivery in India. PAISA - Planning, Allocations and Expenditures, Institutions Studies in Accountability - is AI’s flagship project. PAISA is an effort to understand implications of the inter-governmental transfer systems on the delivery of social sector programmes at the front line. It does this by tracking plans, budgets, fund flows, and decision making systems across the administrative hierarchy. As a former Officer of the Indian Administrative Service (IAS), he served as Joint Secretary, Government of India, Ministry of Panchayati Raj (Rural Local Governments, 2004-2009) and as Secretary, Rural Development and Panchayat Raj, Government of Karnataka State (2001-2004). He is a public finance expert, a consultant, and an adviser who is, or has been, associated with Government of India; State Governments of Karnataka, Kerala, and West Bengal; UNDP; the World Bank; and several national and international development agencies involved with local government policy, fiscal decentralisation and capacity development in local governance. IDSC TEAM'S WORLD BANK SUPPORTED STUDY TOUR TO PAF BANGALORE ON IMPLEMENTATION OF SOCIAL 48 ACCOUNTABILITY TOOLS IN MONITORING & EVALUATION March 7-11, 2017 - STUDY TOUR REPORT - 27th March 2017 Dr Shreelata Seshadri Azim Premji University (APU), Bangalore Dr Shreelata is Professor in the MA Development program at the Azim Premji University. Her interests are in the areas of health systems organization and research, governance and accountability, decentralisation, and child nutrition, health and education. She has over two decades of experience in project development, monitoring and evaluation, and health systems research. She has been working extensively with community-based and non-governmental organizations, self-help groups, and local self-government institutions with special focus on public health and quality health services for the poor. She has been associated with several national and international agencies such as the World Bank, WHO, Center for Global Health Research and several NGOs in the development, supervision and evaluation of large scale health projects across many states in India. As Anchor of the Public Health Initiative at the Azim Premji University, she has established a specialization in Public Health within the MA Development program; the program brings a social science lens to the understanding of public health. She has published in both peer-reviewed journals and the popular press on issues relating to accessibility, accountability and quality of public health systems; and has participated in high level consultations that aim to bring evidence to public policy. She is currently engaged in a number of research projects, including in developing models of capacity building of frontline health workers and people’s representatives to enhance health outcomes at the community level. IDSC TEAM'S WORLD BANK SUPPORTED STUDY TOUR TO PAF BANGALORE ON IMPLEMENTATION OF SOCIAL 49 ACCOUNTABILITY TOOLS IN MONITORING & EVALUATION March 7-11, 2017 - STUDY TOUR REPORT - 27th March 2017 Mr G Gurucharan, IAS (Retired) Director, Public Affairs Centre Mr G Gurucharan is an Officer of the Indian Administrative Service. He retired recently as Secretary, Performance Management, in Government of India's Cabinet Secretariat. He now heads the Public Affairs Centre as its Director. He was earlier Special Secretary, Department of Consumer Affairs; Director General, Bureau of Indian Standards; Additional Secretary, Food & Public Distribution; and Joint Secretary, Ministry of Overseas Indian Affairs. Among the many positions he has held, both in Karnataka State and at the Union Government, he has served as Managing Director at Hutti Gold Mines and Karnataka State Financial Corporation. At the Ministry of Overseas Indian Affairs, Mr Gurucharan initiated and led two projects that focused on using CRC based evidence from emigrants and recruitment agents to develop appropriate guidelines to protect the rights of migrating Indian workers, and to gather feedback from emigrants on their awareness of migration procedures, the Ministry and its various schemes, media exposure, and their motivation to emigrate. He holds a Degree in Commerce and Masters Degree in Economics. IDSC TEAM'S WORLD BANK SUPPORTED STUDY TOUR TO PAF BANGALORE ON IMPLEMENTATION OF SOCIAL 50 ACCOUNTABILITY TOOLS IN MONITORING & EVALUATION March 7-11, 2017 - STUDY TOUR REPORT - 27th March 2017 Dr C K Mathew, IAS (Retired) Senior Fellow, Public Affairs Centre Dr C K Mathew is an Officer of the Indian Administrative Service. He served the Government of Rajasthan for 37 years before retiring recently as its Chief Secretary. He is currently a Senior Fellow and Head, Public Policy & Research Group at Public Affairs Centre and a Visiting Professor at Azim Premji University . He has wide experience in governance and public policy, having held several important assignments such as District Collector; Commissioner, Commercial Taxes; Secretary / Principal Secretary, to the Chief Minister and the Departments of Mining, Energy, Irrigation, Education, Information Technology; and association with the Finance Department in various capacities. At PAC, Dr Mathew conceptualised and led the development of PUBLIC AFFAIRS INDEX [PAI]. Using statistical data available in the public domain from government sources, this index attempts to objectively measure a range of critical aspects of governance and facilitate comparisons between all of India's 30 states. He has also compiled and edited a volume of essays by former Chief Secretaries of various states of the country entitled “In the Rear-View Mirror: Reflections of Chief Secretaries.” Mr Mathew has authored two books, is an avid blogger. He holds a Bachelors Degree in Arts and a PhD, both in English Literature. He is on the Board of Directors of "Educate Girls Globally", an international education trust. Dr Meena Nair Head - Participatory Governance Research Group, Public Affairs Centre Dr Meena Nair has been with PAC since 2005. She holds an MPhil and a PhD (Urban Geography) from JNU Delhi. She has rich, multi-sector experience and specialisation in conceptualizing, designing and implementing social research studies and assessments especially those related to the implementation of Citizen Report Cards and Community Score Cards in various spheres of governance and levels of public service delivery. Prior to joining PAC, Meena worked for 6 years as a Researcher at various levels in the Social and Environment Research Centre (SERC) for Synovate on projects related mainly to health. IDSC TEAM'S WORLD BANK SUPPORTED STUDY TOUR TO PAF BANGALORE ON IMPLEMENTATION OF SOCIAL 51 ACCOUNTABILITY TOOLS IN MONITORING & EVALUATION March 7-11, 2017 - STUDY TOUR REPORT - 27th March 2017 Mr Basil Liongs Head - Citizen Action & Support Group, Public Affairs Centre Mr Basil Liongs has been with PAC since 2015. He has a Bachelors Degree in Agriculture from Tamil Nadu Agricultural University and a Masters in Business Administration from the Indian Institute of Management, Bangalore. At PAC he Heads the Citizens Action & Support Group. He is a social scientist and project management specialist with 30+ years experience in management consultancy, demographic and behavioural surveys, social research, impact assessment, and monitoring & evaluation of programmes. His work covers a wide range of sectors such as Microfinance, Social Inclusion, Rural Livelihood Promotion, Institutional Development Services, Sustainable Agriculture & Business Development Services, Water Supply and Sanitation, Tribal Development, Enterprise Development, Capacity Building and Mentoring. Mr Arvind L Sha Head - Environmental Governance Group, Public Affairs Centre Arvind L Sha has been with PAC since 2013 and now Heads the Environmental Governance Group. He has over 3 years experience in environment and climate change governance. Arvind holds a Masters in Public Policy and Human Development from UNU-MERIT and Maastricht University, the Netherlands; and a Postgraduate Degree in Climate Science and Policy from TERI University, India. At PAC, his work revolves around policy evaluation and climate impact assessment through nexus approach. He has significantly contributed to the design and implementation of social accountability tools for modelling risk and vulnerability associated with climate change. IDSC TEAM'S WORLD BANK SUPPORTED STUDY TOUR TO PAF BANGALORE ON IMPLEMENTATION OF SOCIAL 52 ACCOUNTABILITY TOOLS IN MONITORING & EVALUATION March 7-11, 2017 - STUDY TOUR REPORT - 27th March 2017 Dr Lalita Pulavarti Senior Manager - Projects, Public Affairs Foundation Dr Lalita Pulavarti has extensive and varied professional experience in social research covering quantitative, qualitative, quasi-experimental, and mixed methodologies; data management, analysis, interpretation, and dissemination. For 12 years prior to joining PAF, Lalita worked in the United States at the Division of Health Care Finance & Policy, Commonwealth of Massachusetts and at the Boston Police Department, City of Boston. In these roles, she developed and implemented efficient data collection and dissemination procedures, and facilitated the use of data by internal and external end-users. At the Foundation since 2011, Dr Lalita has used her expertise in research methodologies to successfully lead and complete a large number of projects covering a wide spectrum of social development sectors across several geographies in India; and lead training programmes to enable client organisations to build internal capacities. Lalita has a Ph D in Sociology from Northeastern University, Boston, USA and a Master of Public Health (MPH – Biostatistics and Epidemiology) from Boston University, USA. Lalita's doctoral dissertation dealt with the contribution of local NGOs to the empowerment of women at the grassroots in India. Mr Ravishankar Rao Executive Director, Public Affairs Foundation Mr Ravishankar Rao has a Bachelors Degree in Statistics & Economics and a Masters in Business Administration. For over 43 years, he has been leading and managing, across India and overseas in the Gulf Co- operation Council (GCC) Countries, strategic business units and corporate organizations involved in manufacturing, marketing, and distributing a wide range of consumer and industrial products; and non-profit institutions providing various public services. At PAF, he leads the Foundation and manages various projects related to governance and social accountability. He actively engages with stakeholders at all levels of government and donor agencies, in the development and execution of strategies to achieve project objectives; in managing resources, timelines, budgets, personnel, vendors; delivery of outputs and dissemination; ensuring high standards of quality, client relations, reliability and independence; building and developing motivated teams; monitoring compliance with all financial, legal and regulatory requirements; and establishing partnerships based on credibility and goodwill. IDSC TEAM'S WORLD BANK SUPPORTED STUDY TOUR TO PAF BANGALORE ON IMPLEMENTATION OF SOCIAL 53 ACCOUNTABILITY TOOLS IN MONITORING & EVALUATION March 7-11, 2017 - STUDY TOUR REPORT - 27th March 2017 Annexure 5: List of Participants IDSC / WB Team's Study Tour to Bangalore th th Mon 7 - Sat 11 March 2017 List of Participants Sl # Name Designation Department Email id IDSC 1 Rafaat Mahmoud M & E Manager raaftm@idsc.net.eg Policies Monitoring & Evaluation 2 Lobna Mounir Economic Researcher lobnafeps@idsc.net.eg 3 Bothaina Farag Economic Researcher Information Analysis bfarag@idsc.net.eg 4 Sameh M Selim Executive Manager saselim@idsc.net.eg Information Resources 5 Ahmed Shawky Data Manager ahshawky@idsc.net.eg 6 Hebatallah A Hamid Int Cooperation Specialist International Cooperation hebah@idsc.net.eg 7 Ghada Zaki Senior Economic Researcher gmzaki@idsc.net.eg Quality 8 Neveen Elhelw Senior Statistician nelhelw@idsc.net.eg 9 Sara Abdel Aziz Salem Political Researcher Strategic Planning saaziz@idsc.net.eg 10 Eman El Gewily Economic Researcher Public Opinion Polling Center eesam@idsc.net.eg 11 Dalia EL Sheikh Planning & Follow-up Specialist Chairman’s Office dfekry@idsc.net.eg 12 Dalia Hosni Researcher E-Complaints System dhosni@idsc.net.eg Mahmoud Emam Population & Sustainable 13 Statistical Researcher maemam@idsc.net.eg Hassan Development World Bank Senior Social 1 Amal Faltas WB Cairo Office afaltas@worldbank.org Safeguards Specialist Public Affairs Foundation 1 Ravishankar Rao Executive Director director@pafglobal.org 2 Lalita Pulavarti Senior Manager - Projects lalita@pafglobal.org 3 Narendra L V Manager - Projects narendra@pafglobal.org PAF Bangalore 4 Pooja Ravi Research Associate pooja@pafglobal.org 5 Shreya Suman Research Analyst shreya@pafglobal.org 6 Sowmya Rajan Executive-HR, Admin, Accounts info@pafglobal.org IDSC TEAM'S WORLD BANK SUPPORTED STUDY TOUR TO PAF BANGALORE ON IMPLEMENTATION OF SOCIAL 54 ACCOUNTABILITY TOOLS IN MONITORING & EVALUATION March 7-11, 2017 - STUDY TOUR REPORT - 27th March 2017 Annexure 6: Evaluation Forms for Day-wise sessions, and for the Study Tour [The evaluation form below was used for overall assessment of the Study Tour. Separate forms were used for each of the five days, covering the specific day-wise sessions.] I. Instructions Please fill in this evaluation form completely. While your feedback would help us make future Study Tours more useful for participants, it will also help you crystallize your thoughts on what you expected from the Tour, what you gained from it and how you will make use of the information and training you received here. Please mark a tick on the suitable response where options are given and write in detail where there is space provided. II. Overall Feedback 1. Overall how useful did you find the 5-day Study Tour? Not Valuable Somewhat Valuable Valuable Very Valuable     2. How would you rate the Design of the Study Tour? Give your feedback in the table below Rating : 1 - 5 1 = Lowest Attribute Comments if any 5 = Highest N/A=Not Applicable Time allocated to sessions Balance between theory and practical exercises Timing of Breaks Use of aids like video, PowerPoint, whiteboard and flip charts Contextualization Inclusion of field visit Usefulness of Hands on Exercises 3. Please rate the group-work sessions in terms of the attributes given in the table below? Rating : 1 - 5 1 = Lowest Attribute 5 = Highest Comments if any N/A=Not Applicable Concept and Design Adequacy of time given Number of group work sessions Content Material Guidance provided Usefulness IDSC TEAM'S WORLD BANK SUPPORTED STUDY TOUR TO PAF BANGALORE ON IMPLEMENTATION OF SOCIAL 55 ACCOUNTABILITY TOOLS IN MONITORING & EVALUATION March 7-11, 2017 - STUDY TOUR REPORT - 27th March 2017 4. How would you rate the quality of facilitation by the Trainers? Rating : 1 - 5 1 = Lowest Attribute 5 = Highest Comments if any N/A=Not Applicable Communication Delivery of presentations Holding interest of participants Knowledge of subject Liveliness Attention to each participant Contextualization Management of group work 5. Did the Study Tour meet your expectations in general? Yes - 1 / No - 2 6. Which of your expectations were not fulfilled and why? a) _____________________________________________________________________ b) _____________________________________________________________________ c) _____________________________________________________________________ 7. How do you intend to use the learning from this Study Tour in your work? Action Yes 1 No 2 Give specifics Conduct a training workshop for others in your country Use tools for monitoring and evaluation Others (Specify) IDSC TEAM'S WORLD BANK SUPPORTED STUDY TOUR TO PAF BANGALORE ON IMPLEMENTATION OF SOCIAL 56 ACCOUNTABILITY TOOLS IN MONITORING & EVALUATION March 7-11, 2017 - STUDY TOUR REPORT - 27th March 2017 Annexure 7: Summary of Participants' Evaluation Scores Rating Scale (1 Low; 4 High) Rating of 4 as % Day 1: Session / Parameter/s Total 4 3 2 1 of Total How would you rate today’s sessions in terms of usefulness? 11 1 1 13 85% Rating Scale (1 Lowest; 5 Highest) Total Rating of 4 or 5 How would you rate the quality of facilitation by the Trainers? 5 4 3 2 1 as % of Total Communication 10 3 13 100% Delivery of presentations 8 4 1 13 92% Holding interest of participants 10 3 13 100% Knowledge of subject 8 5 13 100% Liveliness 7 6 13 100% Attention to each participant 9 3 1 13 92% Contextualization 10 2 1 13 92% Management of group work 10 3 13 100% Session-wise feedback Keynote address: Structure of Governments and Governance in India Content 7 4 11 100% Time Allocated 6 3 2 11 82% Material 4 5 1 10 90% Facilitation 4 4 1 1 10 80% Usefulness 6 4 1 11 91% Public Affairs Index [PAI]: Measuring Governance Content 9 2 11 100% Time Allocated 7 4 11 100% Material 8 3 11 100% Facilitation 8 3 11 100% Usefulness 7 4 11 100% Exercise / Discussion on indicators and data sources for a hypothetical PAI in Egypt Content 8 2 1 11 91% Time Allocated 7 2 2 11 82% Material 8 2 1 11 91% Facilitation 6 4 1 11 91% Usefulness 7 3 1 11 91% Quick run through of tools discussed during the May 2016 Ain Al Sokhna Workshop: CRCs, CSCs, RCTs, SIAs Selection of tools, questionnaire design, sampling Content 8 2 1 11 91% Time Allocated 6 3 2 11 82% Material 7 2 2 11 82% Facilitation 8 2 1 11 91% Usefulness 8 2 1 11 91% Preparation for Day 2 (Field Visits 1)Case Study Review 2) Introduction to GRM and BESCOM visit Content 5 4 1 10 90% Time Allocated 6 3 1 10 90% Material 6 4 10 100% Facilitation 7 3 10 100% Usefulness 6 4 10 100% IDSC TEAM'S WORLD BANK SUPPORTED STUDY TOUR TO PAF BANGALORE ON IMPLEMENTATION OF SOCIAL 57 ACCOUNTABILITY TOOLS IN MONITORING & EVALUATION March 7-11, 2017 - STUDY TOUR REPORT - 27th March 2017 Rating Scale (1 Low; 4 High) Rating of 4 as Day 2: Session / Parameter/s 4 3 2 1 Total % of Total How would you rate today’s sessions in terms of usefulness? 9 2 11 82% Rating Scale (1 Lowest; 5 Highest) Total Rating of 4 or 5 How would you rate the quality of facilitation by the Trainers? 5 4 3 2 1 as % of Total Communication 7 4 11 100% Delivery of presentations 5 4 1 10 90% Holding interest of participants 6 4 10 100% Knowledge of subject 6 5 11 100% Liveliness 7 4 11 100% Attention to each participant 5 5 10 100% Contextualization 5 4 9 100% Management of group work 6 3 9 100% Session-wise feedback Visit to a Social Welfare Department School/Hostel Field hands- Rating of 4 or 5 5 4 3 2 1 Total on training as % of Total Content 10 10 100% Time Allocated 8 2 10 100% Material 7 2 1 10 90% Facilitation 8 1 1 10 90% Usefulness 9 1 10 100% Field Trip: Macro level planning and Micro level implementation: Central budgets, State budgets - implementation and monitoring - GoK State Highways Improvement Project (KSHIP), GoK Social Welfare Department (SWD) Content 1 4 5 100% Time Allocated 3 2 5 100% Material 3 2 5 100% Facilitation 3 2 5 100% Usefulness 3 2 5 100% Bangalore Electricity Supply Company [BESCOM] Call Center - Example of a Grievance Redress Mechanism (GRM) in operation Content 7 3 1 11 91% Time Allocated 5 5 1 11 91% Material 6 4 1 11 91% Facilitation 5 4 1 1 11 82% Usefulness 7 3 1 11 91% IDSC TEAM'S WORLD BANK SUPPORTED STUDY TOUR TO PAF BANGALORE ON IMPLEMENTATION OF SOCIAL 58 ACCOUNTABILITY TOOLS IN MONITORING & EVALUATION March 7-11, 2017 - STUDY TOUR REPORT - 27th March 2017 Rating Scale (1 Low; 4 High) Rating of 4 as % Day3: Session / Parameter/s Total of Total 4 3 2 1 How would you rate today’s sessions in terms of usefulness? 3 7 2 12 25% Rating Scale (1 Lowest; 5 Highest) Rating of 4 or 5 as How would you rate the quality of facilitation by the Trainers? Total 5 4 3 2 1 % of Total Communication 7 2 3 12 75% Delivery of presentations 5 3 3 11 73% Holding interest of participants 2 9 1 12 92% Knowledge of subject 4 5 2 11 82% Liveliness 3 6 3 12 75% Attention to each participant 5 3 3 11 73% Contextualization 5 4 3 12 75% Management of group work 7 2 2 11 82% Session-wise feedback M & E from a Government perspective – Central Government, State Government and Local Government (Decentralization, Public Expenditure and Budget Tracking) Content 6 5 1 12 92% Time Allocated 8 3 1 12 92% Material 8 3 1 12 92% Facilitation 7 3 2 12 83% Usefulness 6 4 2 12 83% PAISA for Panchayats - Tracking Fiscal Devolution to Local Governments - a case study from Kolar district, Karnataka on Decentralization, Panchayat Budgets Content 5 4 3 12 75% Time Allocated 3 7 2 12 83% Material 6 3 3 12 75% Facilitation 7 3 2 12 83% Usefulness 5 3 3 1 12 67% Discussion on M&E of Panchayat level budgets and performance Content 4 5 2 11 82% Time Allocated 5 5 1 11 91% Material 4 5 2 11 82% Facilitation 5 4 2 11 82% Usefulness 5 4 1 1 11 82% Group Activity – Applicability: Discussion on how today’s concepts can be applied to a department or agency in Egypt Content 6 6 12 100% Time Allocated 9 3 12 100% Material 9 3 12 100% Facilitation 9 3 12 100% Usefulness 9 3 12 100% Hands-On Exercise: Using case study and field visit materials from Days 2 & 3, prepare a monitoring / evaluation plan [team activity] Content 5 6 11 100% Time Allocated 7 2 1 1 11 82% Material 6 4 1 11 91% Facilitation 8 2 1 11 91% Usefulness 9 2 11 100% Team Presentations: on M/E Plan for the Social Welfare Department Programs / School visited Content 8 1 1 10 90% Time Allocated 9 1 10 100% Material 9 1 10 100% Facilitation 8 1 1 10 90% Usefulness 8 1 1 10 90% IDSC TEAM'S WORLD BANK SUPPORTED STUDY TOUR TO PAF BANGALORE ON IMPLEMENTATION OF SOCIAL 59 ACCOUNTABILITY TOOLS IN MONITORING & EVALUATION March 7-11, 2017 - STUDY TOUR REPORT - 27th March 2017 Rating Scale (1 Low; 4 High) Rating of 4 as Day 4: Sessions / Parameters / Participants Total 4 3 2 1 % of Total How would you rate today’s sessions in terms of usefulness? 8 4 1 13 62% Rating of 4 or How would you rate the quality of facilitation by the Trainers? 5 4 3 2 1 Total 5 as % of Total Communication 11 2 13 100% Delivery of presentations 8 5 13 100% Holding interest of participants 8 4 1 13 92% Knowledge of subject 9 4 13 100% Liveliness 8 4 1 13 92% Attention to each participant 10 2 1 13 92% Contextualization 11 1 1 13 92% Management of group work 10 3 13 100% Session-wise feedback Citizen Involvement in Monitoring – PMGSY Road Monitoring by training Citizen groups (PM's Rural Roads Scheme) Content 9 3 1 13 92% Time Allocated 9 4 13 100% Material 9 4 13 100% Facilitation 9 3 1 13 92% Usefulness 9 3 1 13 92% Climate Change Score Card (CCSC) – NGOs / Think Tanks / Research institutions utilizing the Score Card to measure Climate Change and its impact; Policy Recommendations to the Government Content 9 4 13 100% Time Allocated 7 5 1 13 92% Material 11 2 13 100% Facilitation 10 3 13 100% Usefulness 9 4 13 100% Measuring SDG progress with a focus on Health, Nutrition and Education Content 9 2 2 13 85% Time Allocated 7 4 2 13 85% Material 10 2 1 13 92% Facilitation 10 1 2 13 85% Usefulness 8 3 1 1 13 85% Using SA tools to drive Policy and Improvements in Service Delivery: Some examples: a. SBM (multiple tools – CRC, CRC+, Case studies and CSCs), b. BWSSB (Bangalore Water Supply and Sewerage Board) Content 8 4 1 13 92% Time Allocated 8 4 1 13 92% Material 9 3 1 13 92% Facilitation 10 2 1 13 92% Usefulness 9 3 1 13 92% IDSC TEAM'S WORLD BANK SUPPORTED STUDY TOUR TO PAF BANGALORE ON IMPLEMENTATION OF SOCIAL 60 ACCOUNTABILITY TOOLS IN MONITORING & EVALUATION March 7-11, 2017 - STUDY TOUR REPORT - 27th March 2017 Rating Scale (1 Low; 4 High) Rating of 4 as Day 5: Sessions / Parameters / Participants Total 4 3 2 1 % of Total How would you rate today’s sessions in terms of usefulness? 9 3 12 75% Rating of 4 or How would you rate the quality of facilitation by the Trainers? 5 4 3 2 1 Total 5 as % of Total Communication 12 12 100% Delivery of presentations 9 2 11 100% Holding interest of participants 9 3 12 100% Knowledge of subject 11 1 12 100% Liveliness 12 12 100% Attention to each participant 9 3 12 100% Contextualization 10 2 12 100% Management of group work 10 1 1 12 92% Session-wise feedback Team Activity – Discussion and preparation of Note outline by each team Content 10 2 12 100% Time Allocated 9 2 1 12 92% Material 9 3 12 100% Facilitation 10 2 12 100% Usefulness 11 1 12 100% Presentation of Note outline and Discussions Content 10 1 11 100% Time Allocated 8 3 11 100% Material 10 1 11 100% Facilitation 9 2 11 100% Usefulness 11 11 100% The Way Forward – how to integrate the Tools into M&E of government programs and SDG goals in Egypt Content 9 2 11 100% Time Allocated 10 1 11 100% Material 10 1 11 100% Facilitation 10 1 11 100% Usefulness 11 11 100% IDSC TEAM'S WORLD BANK SUPPORTED STUDY TOUR TO PAF BANGALORE ON IMPLEMENTATION OF SOCIAL 61 ACCOUNTABILITY TOOLS IN MONITORING & EVALUATION March 7-11, 2017 - STUDY TOUR REPORT - 27th March 2017