27938 Assessment of Ghanaian Organisations Involved in the Delivery of Monitoring and E va l u a t i o n Tra i n i n g By: Marie-Hélène Adrien Report prepared for: Operations Evaluation Department (OED), The World Bank, Washington, D.C. May 2001 Assessment of Ghanaian Organisations Executive Summary The purpose of this assessment was to review five Ghanaian training organisations to identify their strengths and weaknesses in Monitoring and Evaluation (M&E) and their ability to provide quality training in M&E. The review examined each organisation’s ability to undertake consulting work for government, civil society and the private sector, and identified options to address weaknesses. The five organisations that were assessed were: 1) the Ghana Institute of Management and Public Administration (GIMPA); 2) the Institute of Statistical, Social and Economic Research (ISSER); 3) the School of Administration at the University of Legon; 4) the Department of Planning of the University of Science and Technology (UST); and 5) the Institute of Local Government Studies (ILGS). Each of these organisations was reviewed against the seven following criteria: 1) The mission and mandate of the organisation 2) Areas of strengths and weaknesses in M&E – that is, the areas of M&E in which the Faculty provides training, conducts research or consultancies 3) Faculty experience and expertise in M&E 4) The adequacy of the organisation’s facilities for the delivery of M&E workshops 5) The organisation’s existing clients for M&E courses in particular, and/or general training courses (depending on the data available) 6) The marketing ability of the organisation – that is, its capacity to sell courses to an external target audience 7) The level of stakeholder support– that is, the degree to which either internal or external stakeholders rate favourably the ability of the organisation to deliver M&E training or to engage in M&E consultation. The scale used for the rating of each organisation is as follows: Very good – Good – Basic – Weak –insufficient data available. Exhibit 1.1 summarizes the key findings of the review. Exhibit 1.1 Overall Assessment of the Organisations SCHOOL OF ADMINISTRATION, DEPARTMENT OF GIMPA ISSER ILGS2 UNIVERSITY OF PLANNING, UST LEGON1 1- Mission/ Teaching Teaching Executive training Teaching Research and Mandate training at the Training Research Teaching Training for local government planning at the level local level 1 Assessment was constrained by the limited data provided by this organisation. 2 ILGS is a very new organization; it is too early to judge its M&E capacities. June 2001 i Assessment of Ghanaian Organisations SCHOOL OF ADMINISTRATION, DEPARTMENT OF GIMPA ISSER ILGS2 UNIVERSITY OF PLANNING, UST LEGON1 2-Areas of Impact evaluation Quantitative and Fiscal audits Baseline surveys Too soon to Strengths in qualitative assess Performance Quantitative M&E analysis indicators analysis Design of survey/ Policy monitoring Statistical questionnaire sampling Program instruments evaluation with a Questionnaire Statistical analysis specific design application for Performance Cost-benefit the NGO sector indicators analysis 2.1 Areas of RBM RBM Program Impact evaluation Too soon to Weaknesses in evaluation assess Logical Logical RBM M&E3 framework framework RBM Participatory Log frame processes Quantitative and Practical qualitative experience in methodologies doing M&E work 2.2 Consulting Program Studies and Strategic planning Baseline studies Too soon to strengths4 evaluation for research assess Implementation Ex-ante NGOs Surveys of financial evaluation Policy analysis systems Feasibility studies Feasibility studies and evaluation Baseline studies 3. Faculty Very good 4 Faculty No data available 2 Faculty No permanent experience in members with members with Faculty M&E very good or very good Subcontract good experience experience model Other Faculty (10) Other Faculty (8) have basic or have basic weak experience experience 4. Facilities Very good Weak Very good Good Basic 3 This category is not comprehensive; it includes the following M&E areas: log frame and theory-based evaluation, performance indicators, results-based management (RBM), program evaluation, impact evaluation, cost-benefit-analysis, baseline surveys, rapid review techniques, participatory/stakeholder evaluation, poverty monitoring, statistical sampling and questionnaire design, sectoral evaluation. The assessment focused on existing strengths and future capacity development needs. 4 Note: (First phrase is inconsistent with second one) The assessment of consulting strengths was based on a review of the consulting assignments conducted by Faculty members. June 2001 ii Assessment of Ghanaian Organisations SCHOOL OF ADMINISTRATION, DEPARTMENT OF GIMPA ISSER ILGS2 UNIVERSITY OF PLANNING, UST LEGON1 5. Major clients Public, Private Public, Private Mostly private Public sector Public sector and NGO sectors and NGOs (District Declining Public Assembly) sector clientele NGO 6. Marketing Good Basic Good Basic Too soon to abilities assess 7. Stakeholder Very good Good Very good for all Good Very good, support activities related particularly from to fiscal audit the Government of Ghana (GOG) Weak support for other M&E activities Overall Recommendations to the World Bank We recommend that the World Bank support activities to strengthen the M&E capacities of GIMPA and the Department of Planning at UST, as these two organisations appear to be the most ready and promising in the short-term. We recommend that this support be organisation-specific, and that the Ghanaian organisations targeted for immediate support be asked to submit proposals outlining the support they require to deliver M&E training At a more macro-level, we recommend that the Government of Ghana (GOG) coordinate any support activities with the various bilateral and multilateral agencies involved in Ghana. In addition to the World Bank, three international agencies – CIDA, IDRC, and to a lesser extent, GTZ – have expressed an interest in joining forces in this type of capacity-building initiative. We recommend that the GOG – in collaboration with multilateral and bilateral agencies consider supporting the enrolment of civil servants in M&E courses via a M&E Training Fund – to accelerate the establishment of a core group of personnel with M&E skills inside government ministries. It would also be appropriate to provide twinning or mentoring support for M&E training to several Ghanaian training organisations. Another option would include the facilitation and support of a network of Ghanaian organisations involved in provision of M&E training. Support for a pan-African network of such organisations would be another possibility. We believe that strengthening the M&E capacities of several organisations would have several benefits: • It would reduce the possibility of one organization developing a monopoly on M&E in the region. • It would create an M&E service delivery network with greater geographical range. • It would allow each organisation to build on its strengths and, in the longer term, develop areas of specialization in M&E. • It would allow the organisations to continue to serve their established, loyal clientele. June 2001 iii Assessment of Ghanaian Organisations Recommendations by Organisation Ghana Institute of Management and Public Administration (GIMPA): We recommend the following ongoing support: • Support to update the content of their M&E course. • Support for twinning with academic organisations or consulting organisations that specialize in M&E, for ongoing academic and research exchanges in M&E. • Support for opportunities to publish and share acquired knowledge, as well as to gain knowledge in M&E through membership in evaluation associations. • Support for expanding the repertoire of courses on M&E. University of Legon, School of Administration: We consider the School of Administration a good school in the areas of executive training in management and finance. However, its overall M&E capacities are weak and do not justify capacity building in monitoring and evaluation at this stage. Institute of Statistical, Social and Economic Research (ISSER): We recommend that ISSER be considered for support once there is evidence that M&E has become a more important part of its curriculum, and when ISSER demonstrates a willingness to improve its facilities. In the short term, ISSER could benefit from support in launching its second summer course in M&E and by for the attendance of faculty members at GIMPA’s M&E workshop. Department of Planning, the University of Science and Technology (UST): We recommend the following support: • Support in integrating M&E into the Department’s existing planning curriculum. • Training for Trainers in M&E to ensure a common understanding of the basic concepts of M&E among all faculty members. • Ongoing coaching while the UST M&E course is being adapted, delivered. • Support in implementing an awareness campaign for M&E among District Assemblies in particular, provided that the idea is supported by the GOG. • Support for mentoring other organisations in M&E. It is worth noting that the Department of Planning at UST has local government (district assembly) contacts/fluency and actual/potential Monitoring and Evaluation skills which could meet well the demands of the Institute of Local Government Studies. Institute of Local Government Studies (ILGS): If the ILGS continues to be a credible specialized organisation for local level public servants, it will be a good organisation to support for M&E activities. Our recommendations, at this stage, are: • To monitor the evolution of the ILGS over the course of the next year to eighteen month period, to gain a better sense of its direction, mandate, and expertise. • To assess the extent to which the ILGS has included M&E or M&E related topics as part of its curriculum. • To monitor the GOG’s relationship with the ILGS to determine if there will be continuing support for ILGS as a key organisation to deliver M&E training. • To monitor ILGS’s management of external contractors. June 2001 iv Assessment of Ghanaian Organisations Acronyms AfDB African Development Bank CDF Comprehensive Development Framework CIDA Canadian International Development Agency CMA Central Management Agencies CSIR Council for Scientific and Industrial Research DANIDA Danish International Development Agency DFID Department for International Development (UK) EU European Commission GIMPA Ghana Institute of Management and Public Administration GOG Government of Ghana GTZ German Technical Cooperation IDRC International Development Research Centre ILGS Institute of Local Government Studies ISODEC Integrated Social Development Centre ISSER Institute of Statistical, Social and Economic Research M&E Monitoring and Evaluation MDA Ministries, Department and Agencies MDPI Management Development and Productivity Institute NGO Non-Government Organisation NIRP National Institutional Reform Programme NOC National Overview Committee OED Operations Evaluation Department PMG Policy Management Group PPMED Policy, Planning, Monitoring and Evaluation Department PSC Public Service Commission PSM Public Sector Management PSMRP Public Sector Management Reform Programme PSMRP Public Sector Management Reform Programme TA Technical Assistance TfT Training for Trainers UDS University for Development Studies UST University of Science and Technology WB World Bank WBI World Bank Institute WHO World Health Organisation June 2001 v Assessment of Ghanaian Organisations Contents 1. Introduction 1 2. Methodology 3 2.1 Data Collection 3 2.2 Data Analysis 4 2.3 Limitations 4 3. Ghana Institute of Management and Public Administration (GIMPA) 5 3.1 People Interviewed 5 3.2 Background 5 3.3 Areas of Strength and Weakness in M&E 7 3.4 Clientele 8 3.5 Faculty, Technical Expertise in M&E 10 3.6 Facilities 11 3.7 Marketing Ability 11 3.8 Stakeholder Support and Conclusion 11 4. School of Administration – University of Legon 13 4.1 People Interviewed 13 4.2 Background, Mission and Mandate 13 4.3 Areas of Strength and Weakness in M&E 13 4.4 Clientele 13 4.5 Faculty, Technical Expertise in M&E 14 4.6 Facilities 14 4.7 Marketing Ability 14 4.8 Stakeholder Support and Conclusion 15 5. Institute of Statistical, Social and Economic Research (ISSER) 16 5.1 People Interviewed 16 5.2 Background 16 5.3 Areas of Strength and Weakness in M&E 16 5.4 Clientele 19 5.5 Faculty, Technical Expertise in M&E 19 5.6 Facilities 21 5.7 Marketing Ability 21 5.8 Stakeholder Support and Conclusion 21 6. University of Science and Technology, Department of Planning 23 June 2001 vi Assessment of Ghanaian Organisations 6.1 People Interviewed 23 6.2 Background 23 6.3 Areas of Strength and Weakness in M&E 24 6.4 Clientele 26 6.5 Faculty, Technical Expertise in M&E 26 6.6 Facilities 27 6.7 Marketing Ability 28 6.8 Stakeholder Support and Conclusion 28 7. Institute of Local Government Studies (ILGS) 30 7.1 People Interviewed 30 7.2 Background and Strengths and Weaknesses in M&E 30 7.3 Clientele 32 7.4 Faculty, Technical Expertise in M&E 32 7.5 Facilities 32 7.6 Marketing Ability 32 7.7 Stakeholder Support and Conclusion 32 8. Conclusion 34 8.1 Observations 34 8.2 Recommendations 34 Exhibits Exhibit 3.1 GIMPA Master’s Programmes 6 Exhibit 3.2 GIMPA’s M&E Workshop Contents 7 Exhibit 3.3 GIMPA’s M&E Workshop Clients (1998-2000) 9 Exhibit 3.4 GIMPA’s M&E Workshop Participants by sector (1998-2000) 10 Exhibit 4.1 Legon School of Administration Executive Programme Participants 14 Exhibit 5.1 ISSER Research Areas that Include an M&E Component 17 Exhibit 5.2 ISSER Short-term Courses Related to Monitoring and Evaluation 18 Exhibit 5.3 ISSER’s Participants (1998-2000) 19 Exhibit 5.4 ISSER Staff Profile (partial list) 20 Exhibit 6.1 Description of the Department of Planning (UST) Programmes 23 Exhibit 6.2 UST Training in Monitoring and Evaluation 25 Exhibit 6.3 UST Department of Planning: Profile of Faculty 27 Exhibit 7.1 ILGS Planned Areas of Focus 31 June 2001 vii Assessment of Ghanaian Organisations Appendices Appendix I Terms of Reference for the Assessment of National Training Organisations 36 Appendix II List of Documents Reviewed 37 Appendix III List of People Interviewed 39 Appendix IV Outline of GIMPA’s Program Evaluation Workshop 41 June 2001 viii Assessment of Ghanaian Organisations 1. Introduction As part of its programme of reform to improve public sector management, the Government of Ghana (GOG) has requested the assistance of the World Bank (WB) in strengthening its Monitoring and Evaluation (M&E) functions. In response to this request, the World Bank proposed to the GOG an action plan for strengthening M&E capacities in Ghana (January 2000), and began a series of initiatives to reach that goal. One of these initiatives was the development and delivery of an M&E Training for Trainers (TfT) workshop on Programme Evaluation for the Ghanaian Institute of Management and Public Administration (GIMPA), which was conducted in April 2000. The National Overview Committee (NOC) of GOG and the National Institutional Renewal Programme (NIRP) are responsible for the development of M&E capacities and systems for the GOG. Following discussions of the WB’s proposed action plan, the NOC, in a document dated June 13th, 2000, indicated that: • The M&E capacities of the Policy, Planning, Monitoring and Evaluation Departments (PPMED) in Public Service institutions should be enhanced for improved performance. The training of M&E staff must be given priority attention. Senior Staff of PPMEDs (including Deputy Directors, Directors, and Chief Directors) must also be trained to understand the functioning of the system. • Capacities of local training institutions, such as the Ghana Institute of Management and Public Administration (GIMPA) and the Institute of Local Government Studies (ILGS), should be strengthened to undertake M&E training at national and decentralised levels respectively. Emphasis must also be placed on training of trainers in these institutions. In order to identify the readiness stage and the needs of Ghanaian training organisations that could benefit from Training for Trainers activities, the Operations Evaluation Department (OED) of the World Bank commissioned Universalia, a Canadian management-consulting firm, to undertake a review of five Ghanaian organisations. Funding support for this work was provided by the Evaluation Office of the Ministry of Foreign Affairs of the Netherlands Government. The author of the Guide was Marie-Hélène Adrien, and the OED task manager was Keith Mackay. The overall focus of the assessment was to examine the existing capacities and infrastructure of these organisations for the delivery of M&E training, and to identify their strengths, weaknesses and capacities that would need to be developed via capacity-building support such as M&E Training for Trainers activities. The targeted organisations and departments included: • The Ghana Institute of Management and Public Administration (GIMPA) • The Department of Planning, University of Science and Technology (UST) • The School of Administration, University of Legon • Institute of Statistical, Social and Economic Research (ISSER) – University of Ghana • The Institute for Local Government Studies (ILGS) A sixth organisation, the Management, Development, and Productivity Institute (MDPI) had expressed a strong interest in being considered for Training of Trainers activities. However, this organisation has limited capacities and was not included in this review. June 2001 1 Assessment of Ghanaian Organisations Detailed Terms of Reference for the assignment are presented in Appendix I. This report is organised as follows: Section 2 describes the overall methodology for the assignment. Sections 3 to 7 present reviews of each of the five organisations. June 2001 2 Assessment of Ghanaian Organisations 2. Methodology 2.1 Data Collection The assessment of Ghanaian training organisations was conducted through document review, interviews with key stakeholders, and site visits to the institutions. Document review For each of the organisations, the following categories of documents were collected (to the extent that they were available) and reviewed: • Descriptive brochures and mission statement of the organisation • Programme descriptions (with a special emphasis on programmes or specialization in M&E) • Staff profiles and curricula vitae • Lists of participants in each M&E and M&E-related course or workshop • Evaluations of M&E or M&E-related workshops • M&E or M&E-related course material (class notes, overheads, bibliographies, etc.) In addition, other documents related to the development of M&E capacities and systems in Ghana were reviewed. A detailed list of all documents reviewed is presented in Appendix II. Interviews A complete list of all individuals interviewed is presented in Appendix III. Interviews were conducted with individuals in the following categories: Organisations/Departments under review • Heads of Departments at the ISSER School of Public Administration and the UST Department of Planning • Directors of GIMPA and MDPI • Academic staff targeted for the potential delivery of M&E training (group interviews) Funding/Donor agencies in Ghana • Programme Officers from the Canadian International Development Agency (CIDA), German Technical Cooperation (GTZ) and the European Commission (EU) directly involved in projects with the organisation or department under review Ghanaian Public Sector Representatives • Heads of Policy, Planning, Monitoring and Evaluation Departments (PPMED) of the Ministry of Agriculture, the Ministry of Health • National Coordinator, National Institutional Renewal Programme (NIRP) • Vice-Chairman of the Ghana Public Service Commission and Co-Chairman of the National Overview Committee (NOC) June 2001 3 Assessment of Ghanaian Organisations Representatives from Civil Society and others • Head of Training, Integrated Social Development Centre (ISODEC) • Local consultant External Stakeholders (outside Ghana) • Head of Evaluation Unit, International Development Research Centre (IDRC), Canada • World Bank, Operations Evaluation Department (OED) staff • World Bank Institute (WBI) staff • African Development Bank (AfDB), Evaluation Unit staff • Programme Director, Ghana, Canadian International Development Agency (CIDA) Site Visits A site visit was conducted to each organisation or department under review, to assess the quality of classrooms, the accommodations for residence, and the laboratories for the potential delivery of distance learning. 2.2 Data Analysis In discussions with the World Bank OED Task Manager responsible for the review and the GOG representatives, seven criteria were identified to provide a basis of judgement for the review. Data from all sources were analysed in terms of these criteria: 1) The mission and mandate of the organisation 2) Areas of strengths and weaknesses in M&E – that is, the areas of M&E in which the Faculty provides training, conducts research or consultancies 3) Faculty experience and expertise in M&E 4) The adequacy of the organisation’s facilities for the delivery of M&E workshops 5) The organisation’s existing clients for M&E courses in particular, and/or general training courses (depending on the data available) 6) The marketing ability of the organisation – that is, its capacity to sell courses to an external target audience 7) The level of stakeholder support– that is, the degree to which either internal or external stakeholders rate favourably the ability of the organisation to deliver M&E training or to engage in M&E consultation. 2.3 Limitations Although most of the organisations reviewed prepared documents to support this review, the Institute of Local Government Studies and the School of Administration (University of Legon) could not provide relevant information on their organisations. This affected the scope and the depth of the assessment of these two organisations. In addition, the nature and the richness of the documents provided by organisations varied widely, which made comparative judgment difficult. Some organisations, for example, provided participant evaluation forms and lists of participants, others did not. Some organisations provided the full syllabus of their courses, other simply provided a list of courses offered. June 2001 4 Assessment of Ghanaian Organisations 3. Ghana Institute of Management and Public A d mi n ist ra ti on ( GI MPA) 3.1 People Interviewed • Dr. Lawrence Kannae, Director of Programme, GIMPA, Accra, Ghana • Dr. Sakyi Awuku Amoa, Acting Director, GIMPA, Accra, Ghana • Dr. Stephen Adei, Director and Senior Lecturer, GIMPA, Accra, Ghana • Dr. Joseph Akuamoah-Boateng, Senior Lecturer, GIMPA, Accra, Ghana • Dr. Edward O. Asante, Senior Lecturer, GIMPA, Accra, Ghana • Dr. Christopher Kwaku Addy Nayo, Senior Lecturer, GIMPA, Accra, Ghana • Mr. Daniel Kwabena Boakye, Lecturer, GIMPA, Accra, Ghana 3.2 Background The Ghana Institute of Management and Public Administration (GIMPA) was established in 1961 as a joint project of Ghana and United Nations Special Fund. In line with the GOG’s public sector reform initiative, GIMPA has been restructured into a commercialized organisation with an organisational structure designed to meet its new commercial mandate. The main objective of GIMPA is to provide clients with good quality management training and a wide range of flexible consulting services at a reasonable cost. GIMPA provides the following services: • Executive training and development programmes • Practical-based training in development programmes, a Masters Degree in Development Management, and a BA in Leadership and Governance and Public Administration • Consulting services for local and international expertise • Research and publications on national issues • Regular fora to discuss and resolve contemporary national issues As shown in Exhibit 3.1, GIMPA’s curriculum includes three Masters Programmes with 12 specific options. In addition, GIMPA offers a wide range of short-term courses (three days to three weeks) to mid-level managers and senior managers. June 2001 5 Assessment of Ghanaian Organisations Exhibit 3.1 GIMPA Master’s Programmes EXECUTIVE MASTERS PROGRAMMES COMPETENCIES OPTIONS Executive Masters in Business • Strategic Thinking • Governance and Political Administration (EMBA) Environment for Business • Marketing Management • Entrepreneurship Development and • Human Resources Management Managing SMEs and Organisational Behaviour • Money and Banking • Management Information Systems • Electronic Commerce • Financial and Management Accounting • Financial Management • Business Policy and Strategy • Economics • Quantitative Analysis and Operations Management Executive Masters in Public • Public Administration and • Privatization and Administration (EMPA) Management Commercialisation of Public Institutions • Public Sector Economics and Finance • Municipal Administration and Management • Policy Formulation and Analysis • Decentralization and Local • Human Resource Management and Government Administration Organisational Behaviour • International Relations • Management Information Systems • Strategic Thinking and Management • Governance and Leadership • Public Sector Legal and Regulatory Framework • Public Sector Accounting Executive Masters in Governance and • Political Systems • Management Information Systems Leadership (EMGL) • Governance I: Concepts, Principles • Legal and Regulatory Framework and Practice • Managing Political Parties • Governance II: Globalization and • Negotiation Global Governance • Advanced Leadership: Case Studies and the African Context • Public Sector Economics and Finance • Policy Formulation and Analysis • Strategic Thinking and Management • International Relations SOURCE: GIMPA Documents June 2001 6 Assessment of Ghanaian Organisations In April 2000, a group of 26 GIMPA staff participated in a Monitoring and Evaluation (M&E) Train the Trainer workshop funded by the World Bank. Since then, GIMPA has taken steps to integrate the M&E training into its regular programming. An M&E team was created to adjust the M&E workshop content to the needs of the targeted audience. This team has met on a regular basis under the supervision of GIMPA’s Head of Programmes and has drafted the outline of the revised workshop content. GIMPA staff is presently conducting a needs assessment survey of selected Heads of Policy, Planning, Monitoring, and Evaluation Departments (PPMED) of Ghanaian Ministries to better understand how the M&E workshop relates to the demands of these Ministries. More recently, in February 2001, GIMPA delivered a three-day M&E workshop for the Ghana Public Service Commission. 3.3 Areas of Strength and Weakness in M&E A review of GIMPA’s curriculum and workshop content shows a focus on M&E or M&E-related topics. Since 1998, GIMPA has delivered three workshops on M&E (see Exhibit 3.2) that consisted of 4- to 8-hour modules on issues such as the logical framework, impact assessment, and communication for M&E. Our review of the course outline and instructors’ notes confirms that GIMPA has a good grasp of the various approaches and tools in M&E. Interviews with GIMPA staff and a visit to their document centre also confirmed that GIMPA keeps itself well informed of the M&E approaches utilized by multilateral and bilateral agencies, in order to offer its clientele the most relevant information to monitor and evaluate their donor-funded projects. Exhibit 3.2 GIMPA’s M&E Workshop Contents M&E OF DEVELOPMENT PROJECTS, DEC. 1998 AND OCTOBER M&E OF DEVELOPMENT PROJECTS, APRIL 2000 1998 Projects and Development Projects and Sustainable Development Monitoring and Evaluation Concepts and Approaches Monitoring and Evaluation Concepts and Approaches Designing a Monitoring System Logical Framework Analysis Logical Framework Designing a Monitoring System Project Sustainability Key Questions for Evaluation Key Questions for Evaluation Plan Policy Monitoring and Evaluation Evaluation Process in Practice Project Sustainability PRA Techniques for Monitoring & Evaluation Monitoring & Evaluation Techniques Presentation on M&E Experience of an NGO M&E Experience of an NGO Communication for M&E Communication for M&E Presentations on M&E Experiences Impact Evaluation Group Presentations on Logical Framework Monitoring Productivity & Quality in Development SOURCE: GIMPA Documents June 2001 7 Assessment of Ghanaian Organisations Our review of the workshop content and curriculum, as well as interview data, suggests that GIMPA demonstrates strengths in the following areas: impact evaluation, performance indicators, policy monitoring, as well as program evaluation with a specific application for the NGO sector. In addition, GIMPA has solid experience in training senior public sector staff in policy analysis. GIMPA acknowledges its weaknesses in the areas of managing/evaluating for results, as well as in a holistic view of the logical framework – although a module on this topic is being developed for the workshop. GIMPA’s strength in consulting is based on the experience of individual faculty members who conduct program evaluations for / with NGOS. A sample of their M&E assignments include: • End of Program evaluation for the Ghana Red Cross (2000) • Evaluation and Strategic Planning for Afram Plains Development Organisation and NGO in Community Water (1999) • Evaluation of the Navrongo-Bolgatanga Diocesan Development Office (1999) • Evaluation of Training Needs of NGO in Sub-Saharan Africa (1996) GIMPA faculty members also have experience consulting in policy development, including the following: • Policy proposal and policy development for the Ghana Cocoa Industry • Impact Assessment of the Privatization Programme in Ghana 1996-1998 for the Ministry of Finance and the World Bank (1999) • Gender and Educational Inequalities at the Basic Level in Ghana and Implications for Policy (1995-1997) In addition, faculty members have individually conducted ad hoc planning or policy development consultancies in the health sector, education, and good governance. 3.4 Clientele GIMPA has a reputation that extends beyond Ghana and that, over the years, has attracted students from English West Africa, Ethiopia, and inside Ghana. In fact, many of the senior staff members of the various Ghanaian training organisations that were part of this review have either attended GIMPA’s workshops or graduated from GIMPA. Our review of GIMPA’s M&E workshop clientele for the last three years (see Exhibit 3.3) shows that GIMPA’s clientele cuts across various sectors, including civil and public sectors, bilateral and multilateral agencies, civil society (NGOs and associations), as well as sub-regional organisations, and the private sector. GIMPA reports that their latest enrolment shows an increase in private sector attendance and a slight decline in public sector attendance, perhaps due to GIMPA’s new pricing structure. The distribution of participants for the M&E workshops is presented in Exhibit 3.4. The most recent, fourth M&E workshop commissioned by the GOG’s Public Service Commission, and planned to be delivered in February 2001 is an encouraging sign of the public sector’s increasing interest in this topic. June 2001 8 Assessment of Ghanaian Organisations Exhibit 3.3 GIMPA’s M&E Workshop Clients (1998-2000) SECTOR CLIENT Civil and Public Sectors • Policy Management Group of the Office of the President • National Institutional Renewal Programme • Office of the Head of Civil Service • Ministry of Agriculture • Ghana Public Service Commission • Ghana National Association of Teachers • Ministry of Health • Ministry of Justice Security and Regulatory Services • Ghana Armed Forces • Ghana Police Service • Ghana Prison Service • Customs Exercise and Preventive Services Financial Institutions • Ghana Commercial Bank • International Finance Corporation • Barclays Bank Ghana • State Insurance Corporation Multi and Bilateral Organisations • World Bank • World Health Organisation • Commonwealth Secretariat • Department for International Development • United Nations organisations: UNDP UNICEF FAO • GTZ Private Sector Organisations • Enterprise Support Services for Africa • Ghana Regional Appropriate Technology Industries Services • Unique Ceramics Ltd. • Eyaqueen Enterprises Ltd. Non-Governmental Organisations • World Vision International • Catholic Secretariat • Technoserve • ADRA Sub-regional Institutions • West African Management Development Institutions Network • West African Institute of Financial and Economic Management Professional Associations • Chartered Institute for Marketing • Institute for Personal Management • Institute of Directors • Ghana Employers Association • Association of Ghana Industries SOURCE: GIMPA documents June 2001 9 Assessment of Ghanaian Organisations Exhibit 3.4 GIMPA’s M&E Workshop Participants by sector (1998-2000) M&E WORKSHOP APRIL M&E WORKSHOP M&E WORKSHOP TYPE OF PARTICIPANT 2000 DECEMBER 1998 OCTOBER 1998 Private Sector 4 7 3 Public Sector 3 6 3 NGO 11 22 10 Other 0 0 2 Total 18 35 18 SOURCE: GIMPA Documents 3.5 Faculty, Technical Expertise in M&E Overall, GIMPA’s staff is very well qualified (see Exhibit 3.5). Out of eight senior staff, five hold PhD degrees, one has an MBA, and two have master’s degrees. More important, however, is their level of involvement in M&E. Individual interviews and/or reviews of publications indicate that four of the eight senior staff have extensive knowledge of M&E – having taught, made addresses at conferences, or published in the area of M&E. Exhibit 3.5 Profile of GIMPA’ Senior Staff ACADEMIC NAME POSITION IN GIMPA AREA OF SPECIALIZATION PROFILE Stephen Adei Director General PhD Sustainable Development Sakyi Awuku Amoa Acting Director MA Management Development Lawrence A. Kannae Director of PhD Programme and Policy Programmes Evaluation, Project Planning and Management Joseph Akuamoah Senior Lecturer MBA General management, Human Boateng resource management Edward O Asante Senior Lecturer in PhD Marketing and Information Marketing Gathering and Analysis Christopher Kwaku Consultant PhD International Development, Addy Nayo Strategic management Daniel Kwabena Consultant MA Marketing Analysis, Project Boakye Feasibility Studies Cletus Kwashi Director of Policy PhD Monetary and Financial Theory, Dordunoo Analysis Development Economics SOURCE: Faculty curriculum vitae June 2001 10 Assessment of Ghanaian Organisations 3.6 Facilities GIMPA has seven modern classrooms (air conditioned and equipped with overhead projectors) that seat 35 participants, as well as an air-conditioned conference hall that can seat 150-200. It has satellite video-conferencing facilities and, since the inauguration of this facility in June 2000, GIMPA has conducted four workshops through distance education, mostly with the World Bank in Washington. GIMPA’s library contains 36,000 books and 120 journals. The organisation can accommodate up to 109 people in 105 single rooms and 4 air-conditioned suites. It has full catering facilities and a dining hall that can accommodate 300-350 people. The institute’s clinic is staffed by two nursing officers and a ward assistant, and is visited by a doctor two to three times a week. The Achioma Hospital is available to treat emergencies. Many consider GIMPA’s facilities a key asset. Interviews with competitors and former participants rate the GIMPA’s campus as “attractive and the best option in the region for medium-term courses.” 3.7 Marketing Ability Over the last three years, GIMPA has demonstrated an increased ability to market its services. GIMPA is no longer supported by government, and has been responsible for 100% of its revenues since 1999. This has had several visible implications: • Advertising: During the two-week field mission, there was at least one GIMPA workshop advertisement a day in the local newspapers. GIMPA has also planned a more serious publicity campaign through regular visits to PPMEDs and donor agencies. • Pricing structure: In order to generate profits and be sustainable GIMPA’s pricing structure has been completely modified. At present, attendance fees for a one-week workshop at GIMPA are between 1.5 and 2 times as expensive as other organisations. Nevertheless, senior staff indicate this new pricing structure reflects the context in which GIMPA is operating • Moving from a supply to a demand approach: As is the case with many government- funded agencies, until 1997 GIMPA’s approach was supply-driven, and the courses it offered reflected the skills and strengths of the senior staff. Since 1997, GIMPA has become aware of the need to adapt its services to the needs of its clientele. It now has regular visits and meetings with key public sector individuals (e.g. the Head of the National Institutional Renewal Programme), conducts surveys of key ministries, and carefully reviews evaluations of its workshops. 3.8 Stakeholder Support and Conclusion Throughout all interviews conducted, there was an overall recognition of GIMPA’s professionalism and commitment to teaching and research. Competitors, donors, staff and government representatives all recommend GIMPA as an organisation that has the capacity to deliver relevant M&E training. Some concerns were raised, however, about GIMPA’s strong entrepreneurial spirit and their rather aggressive marketing approach, which some feel could limit public sector enrolment. The concern is primarily related to the fee structure which is higher than in other organisations. Some local government stakeholders also expressed concern that GIMPA’s faculty lacks extensive hands-on public sector experience in M&E. In this regard, GIMPA’s proposes to undertake more needs assessments within the public sector to ensure that any training developed responds to its needs and demands. June 2001 11 Assessment of Ghanaian Organisations Over the last three years GIMPA has demonstrated its commitment to becoming more entrepreneurial and has taken steps to become what its Director identifies as a “regional centre of excellence in M&E.” GIMPA has definitely taken the lead in promoting M&E workshops and could benefit from some ad hoc support in specialized areas to enhance its expertise (e.g. in RBM, some donor approaches to M&E). We recommend the following ongoing support in M&E: • Support to update the content of their M&E course. GIMPA has begun to adapt its program evaluation course to the needs of its targeted audience. With support from an external consultant, GIMPA has outlined a two-week M&E course that includes general introductory sessions on Program Evaluation, as well as specific sessions on M&E in the Ghanaian context. The outline of GIMPA’s course is presented in Appendix IV. In order to deliver the course, GIMPA has highlighted the need for external support in the following areas: – expanding the module on the logical framework approach, in particular to incorporate the language and terminology used by various donor agencies (in order to familiarize participants with the terms used by the World Bank, GTZ, CIDA and other funding agencies) – designing a Result-Based Management (RBM) module, and co-delivering the module in the first workshop – co-delivery of the first M&E workshop with an experienced M&E facilitator, in order to obtain immediate feedback on the delivery of the course and to address changes as needed. • Twinning with academic organisations or consulting organisations that specialize in M&E, for ongoing academic and research exchanges in M&E. At the Pan-African level, GIMPA would like to strengthen its relationships with other African organisations involved in the delivery of M&E training. GIMPA could also benefit from stronger linkages with Northern research organisations or consulting organisations involved in M&E. The overall purpose of the twinning would be to provide coaching on an as needed basis and to ensure continuous dialogue and sharing of ideas. • Opportunities to publish and share acquired knowledge on M&E methods and applications, and the utilisation of M&E findings, as well as to gain knowledge in M&E through membership in evaluation associations. GIMPA acknowledged the benefit of sharing its experience and expertise with other countries. As a recent participant in the Evaluation Capacity Development Workshop jointly organised by the African Development Bank, the World Bank and the Development Bank of Southern Africa (Johannesburg, September 2000), GIMPA indicated that such opportunities provide faculty members with exposure to other approaches and to different perspectives on M&E. GIMPA has expressed an interest in being supported for attending evaluation conferences or for becoming a member of some of the most active evaluation associations. • Expanding the repertoire of courses on M&E. GIMPA could be supported in its effort to become a centre of excellence by being offered other TfT opportunities in more advanced or more specific evaluation courses. Presently, the M&E program includes courses at the introductory level. GIMPA is seeking support to enlarge its curriculum in order to include intermediate and advanced level courses. As an example, GIMPA is discussing with the International Development Research Centre (IDRC) the possibility of receiving train the trainer programs in Outcome Mapping, Institutional and Organisational Assessment, and Impact Evaluation. June 2001 12 Assessment of Ghanaian Organisations 4. School of Administration – University of Legon 4.1 People Interviewed • Dr. Poku, Acting Director, School of Administration – University of Legon, Ghana • E. F. Akpedonu, Executive Secretary, School of Administration University of Ghana – Legon, Accra, Ghana • Akua S. Ejisu, Lecturer, School of Administration, University of Ghana – Legon, Accra, Ghana 4.2 Background, Mission and Mandate The School of Administration at the University of Legon is dedicated to providing intensive two- and three-day courses for senior managers and executives in private and state organisations. 4.3 Areas of Strength and Weakness in M&E At the School of Administration, course outlines and course evaluations were not available. Information on the integration of M&E into the school’s existing programme and curriculum emerged from interviews with the senior staff of the School of Administration. From their perspective, the school does not specialize in M&E – and certainly not in project monitoring and evaluation. Only one of the faculty members conducts M&E consulting assignments on an individual basis, and no one specializes in the teaching of M&E. The School of Administration is very strong in auditing and financial performance measurement. Only three modules contain some evaluation components: • Organisational Goal Setting and Performance Measurement (6-hour module in a 4-day workshop) • Staff Performance Appraisal (3-hour module in a 4-day workshop), the focus of this module is on personnel, rather than on program or project evaluation. • Auditing for Non-Auditors (3-day workshop) 4.4 Clientele The School of Administration targets executive participants who are looking for state-of-the-art professional development courses in administration. Over the years, the clientele have come more from the private sector (the financial sector, banking industries, insurance companies and accounting firms) than the public sector (see Exhibit 4.1). In the last three years the most frequent clients of the School of Administration have been the Bank of Ghana, the Ghana Commercial Bank Ltd., EcoBank Ltd., First Atlantic Merchant Bank, Unique Trust Financial Services, and the IRS Staff Savings Scheme. As the School of Administration was not able to provide a course syllabus or participant evaluations, it was not possible to assess the quality of the course content. June 2001 13 Assessment of Ghanaian Organisations Exhibit 4.1 Legon School of Administration Executive Programme Participants TYPE OF PARTICIPANT 1998 1999 2000 Private Sector 6 54 68 Public Sector 29 24 8 NGO 2 5 0 N/A 1 11 7 Total 38 94 83 Source: School of Administration documents Exhibit 4.1shows a significant decline in public sector attendance at the workshop. Although the School of Administration made no systematic analysis of this trend, one hypothesis for the decline is the increased focus of the material and case studies on private sector examples. Also, “word of mouth” may have promoted the notion that that the course is attended primarily by individuals in the banking industry, which may have caused public sector participants to look for other training organisations to respond to their needs. 4.5 Faculty, Technical Expertise in M&E The School of Administration has 11 faculty members. Although requested, no documentation was made available regarding the profile, background or expertise of the staff. 4.6 Facilities The School of Administration of the University of Legon has very modern premises for professional executive training. These premises include an auditorium (capacity of 450 people), a dedicated building with three main training rooms (capacity of 25 people each), air conditioned and fully equipped with the newest training facilities and equipment, and a dining room and lounge for participants. The School is building a distance-learning centre that should be operational in May 2001. 4.7 Marketing Ability The School of Administration strongly markets its audit courses in trade magazines and in local newspapers. However, we were surprised at the discrepancy between the abundance of newspaper advertisements and the lack of documentation available during our visit. Over the last three years, the School of Administration has demonstrated its ability to become gradually more self-sufficient. As an example, in 1998, 90% of the budget of the School of Administration consisted of funds received from the University. In 2000, the overall budget shows an increase of 11%, and only 65% of that budget come from University funds. These additional revenues are from fees received for short-term executive courses June 2001 14 Assessment of Ghanaian Organisations 4.8 Stakeholder Support and Conclusion Highly praised amongst its stakeholders for its deep understanding of the banking industry, the School of Administration rated poorly in its ability to respond to the needs of the public sector in M&E (beyond fiscal audits). None of the GOG informants rated the School of Administration among the top three organisations that they would support for M&E training of trainers. Essentially, this organisation has established a clear niche with the banking industry and insurance sector, but has very limited knowledge and experience in other public affairs. On the other hand, several former participants from the banking industry, who attended the School of Administration’s workshops, rated the school as the best in Ghana and certainly capable of developing a coherent curriculum in M&E. We consider the School of Administration a good school in the areas of executive training in management and finance. However, its overall M&E capacities are weak and do not justify capacity building in monitoring and evaluation at this stage. June 2001 15 Assessment of Ghanaian Organisations 5. Institute of Statistical, Social and Economic Research (ISSER) 5.1 People Interviewed • Dr. Anarfi, Director, Institute of Statistical, Social and Economic Research, University of Ghana, Accra, Ghana • Dr. Ernest Aryetey, Deputy Director and Associate Professor, ISSER • George Botchie, Research Fellow, Planning, Monitoring and Evaluation of Development Programmes and Projects 5.2 Background ISSER was founded in 1962 and is the main Social Sciences Research Group of the University of Ghana. The primary mandate of ISSER is to conduct research in social sciences to provide input to the university and to the government. A secondary mandate of ISSER is to deliver teaching and training. In addition, the faculty of ISSER conducts research and consulting mandates for various national and international organisations. Historically, ISSER offered courses in two areas: a Diploma and Certificate in Statistics (one and two-year programmes respectively) and an MA and MPhil in Development Studies (one and two- year programmes respectively). However, as part of the restructuring of the University, these two programmes were transferred to the Accra Polytechnic School. At the same time, the government increased pressure on ISSER to become more self-sufficient and to identify mechanisms for money- generation. This pressure, combined with ISSER’s mandate to teach/ train and the restructuring, has led ISSER to develop short-term courses on a fee-for-service basis. Presently, in addition to conducting research, ISSER offers two short-term courses: a 2-week Research Methodology course that targets all sectors, followed by a 3-week course on Data Analysis. ISSER has given these courses three times and has also been called upon to deliver the courses on the premises of three or four other institutions. To date ISSER has trained approximately 150 participants. 5.3 Areas of Strength and Weakness in M&E In 1999, ISSER developed a 4-year Research Priority Plan in an attempt to bridge the demand for research and the specialized skills and interest of its staff. A review of the plan shows that several research areas include M&E components, as shown in Exhibit 5.1. June 2001 16 Assessment of Ghanaian Organisations Exhibit 5.1 ISSER Research Areas that Include an M&E Component ISSER THEMATIC RESEARCH AREAS RESEARCH AREAS (1999-2003) THAT INCLUDE AN M&E COMPONENT Finance for Small-Scale Investments Degree of outreach attained so far by different types of organisations Agricultural Productivity and Development of methodologies to measure, monitor and compare changes in Household Incomes productivity under different farming systems Food and Nutrition Security The impact of Human Resource development on household food security and poverty reduction The impact of trade and infrastructure on household food security The impact of migration on food supplies Human Resource Development Cost-effectiveness studies for specific interventions in the health sector The impact of specific diseases (e.g. malaria, AIDS, schistosomiasis) on productivity The impact of human resource development on poverty reduction and household food security Social Security and Policy Reforms Methods/Approaches for the measurement, analysis and monitoring of poverty and well being of households. The impact of migration on women and children left behind Gender and Life Cycle Issues Cost-benefit analysis of interventions to support women increase their productivity The impact of women’s work on child welfare The impact of child labour on household income and welfare The control and access to resources and services and their impact on household production Governance, Decentralisation and the The impact of public sector reform on the efficiency of government Politics of Development Policy Assessment of different approaches and methods for including poor people and Reforms other marginalized groups in the process of policy formulation Source: ISSER, Medium-Term research Priorities 1999-2003 In addition to conducting research that includes M&E components, ISSER has developed a series of short-term courses that specifically focus on M&E; Exhibit 5.2 provides an overview of these courses. To date, two courses have been implemented; one course on project planning, monitoring and evaluation is being finalised and will be delivered in the summer of 2001. This course is designed for those who are working on the planning, implementation, monitoring and evaluation and maintenance of development projects across a range of sectors. The project planning and appraisal component of the course will deal with issues such as the project cycle, project framework (SWOT Analysis), and various aspects of project appraisal. The project implementation, monitoring and evaluation course will deal with issues such as operational tools for time and cost control, development of management capacity and log frame analysis. June 2001 17 Assessment of Ghanaian Organisations Exhibit 5.2 ISSER Short-term Courses Related to Monitoring and Evaluation TITLE COMMENTS Research Methodology and Report 2-week course designed and delivered since 1997 writing Delivered 3 times in the last 3 years (once a year) A total of 120 participants from public sector, private sector and some NGOs Data Management and Analysis 3-week course designed and delivered since 1997 Delivered 3 times in the last 3 years (once a year) A total of 90 participants, from the Public, private and NGO sectors Project Planning, Analysis and 3-week course presently being developed. The course has never been delivered Monitoring but should be completed and delivered by July 2001. The course emerged as a result of participant evaluations. Outline of course includes quantitative, qualitative analysis, log frame development, data collection and analysis, design of instruments Sources: ISSER, Medium-Term research Priorities 1999-2003, document analysis of courses content Our review of end-of workshop evaluations for the two ongoing workshops is quite positive. Over the last three years, an average of 90% of participants completed an evaluation form. Overall, for the Research Methods workshop, 88% of participants evaluated the course content as excellent or very good, 10% as good, and 2% as average. For the Data Analysis course, 76% of participants evaluated the course content as excellent or very good, and 24% as average. For both courses, 98% of participants would recommend the course to a colleague. Based on a review of the course content and the Faculty member’s experience ISSER shows strengths in the following M&E areas: quantitative and qualitative analysis; statistical analysis and sampling, questionnaire design and survey instruments, data collection methodologies. It is less strong in other M&E areas such as RBM, log frame, processes in evaluation (participatory evaluation, rapid appraisal techniques). From a consulting perspective, ISSER is strong in conducting surveys and feasibility studies as demonstrated by a sample of the assignments carried out by different Faculty members: • Ghana Children 2000 Survey (2000), for the European Commission • The Common Country Assessment (CCA) Indicators (1999), for the WHO and the World Bank • The Accra Urban Food and Nutrition Survey (1997) for the International Food Policy Research Institute, the WHO and the Noguchi Memorial Institute for Medical Research • Socio-Economic Technical and Financial Feasibility Study of Connecting Selected Communities to the Main Ghana Water and Sewerage main lines in Ho District. For DANIDA (no date) • Effectiveness and Impact of Extension Services to the Agricultural Sector. For the Ministry of Food and Agriculture (1998) June 2001 18 Assessment of Ghanaian Organisations 5.4 Clientele As shown in Exhibit 5.3, since 1998 and out of 143 participants (Participants’ lists for 1997 were not available) , 50% came from the private sector, 27% from the NGO community, almost 12 % from the private sector, and 12% did not identify their organisation. This distribution is similar to other Ghanaian training organisations, and shows the difficulty of attracting public sector clientele, in spite of ISSER’s reasonable pricing structure. Exhibit 5.3 ISSER’s Participants (1998-2000) TYPE OF PARTICIPANT 1998 1998 1999 1999 2000 Private Sector 13 10 13 24 12 Public Sector 7 2 6 0 1 NGO 20 7 6 2 4 N/A 13 2 0 0 1 Total 53 21 25 26 18 Source: ISSER documents 5.5 Faculty, Technical Expertise in M&E ISSER staff is highly qualified in research (see Exhibit 5.4). Thirteen out of 16 faculty members have PhD degrees, and all faculty members have a master’s degree. However, although interviews indicate interest and skills in M&E from all 16 faculty members, only 4 out of 16 have conducted research, delivered courses and conducted assignments in areas specifically related to M&E. These included: • Research methods • Statistical forecasting and statistical data analysis • Monitoring and evaluation of development projects • Statistical modelling • Appraisal of development projects • Family planning, evaluation and administration June 2001 19 Assessment of Ghanaian Organisations Exhibit 5.4 ISSER Staff Profile (partial list) ACADEMIC NAME POSITION IN ISSER AREA OF SPECIALIZATION5 PROFILE Clement Ahiadeke Senior Research Fellow, PhD Demography, Epidemiology Population Dynamics Unit and Biometry Ernest Aryeetey Deputy Director and PhD Development Economics Associate Professor L. Oware Gyekye Senior Research Fellow, PhD / LLB Sociology of Development, Social Research Division. Research Methods Nicolas N.N. Head, Computing, PhD Statistical Modelling, Nsowah Nuamah Information and Statistical Forecasting, Data Communication Division. Analysis Edzodzinam Akuyo Junior Research Fellow LLB Gender, Environment and Tsikata Development George Botchie Research Fellow PhD Spatial Development Planning and Policy. PME of Development Programmes and Projects Isaac Osei-Akoto Research Fellow MPhil Modelling for Decision- Making, Economic Evaluation of Social Services. Felix Ankomah Research Fellow MPhil Agricultural Policy, Asante Development Economics. John Kwasi Anarfi Research Fellow PhD Migration Studies, Woman's Studies Al-Hassan Wayo Senior Research Fellow PhD Agricultural Development Seini Policy, Food Security Issues Victor Kwame Senior Research Fellow PhD Agricultural Economics Nayanteng Ellen Bortei Doku Research Fellow PhD Social Development Aryeetey W. Kwadwo Asenso Associate Professor PhD Economic Development Okyere Anthony Wilson Research Fellow MPhil Data not available Kormla Tsekpo Sources: Curriculum vitae of staff and self-reported data at interviews 5 With the exception of four faculty members who have either very good or good experience in conducting M&E activities, the faculty’s M&E experience is concentrated in teaching or research on M&E rather than conducting M&E assignments. June 2001 20 Assessment of Ghanaian Organisations 5.6 Facilities The existing facilities at ISSER are not yet adequate or conducive to workshops or training. The Institute rents facilities in downtown Accra for all its short-term courses and workshops, and is planning to build two training rooms equipped with modern technology. However, there was uncertainty as to the completion date and inspection of the sites confirms that very little construction has been done. There are no facilities for distance learning. 5.7 Marketing Ability Faced with the pressure from the GOG to become self-sufficient, ISSER is trying to learn quickly how to compete for clients in order to remain sustainable. The results of these efforts are still modest. Interviews with the senior staff of ISSER identify several difficulties in sustaining marketing in initiatives. First, the demand for the courses is relatively limited. Second, ISSER’s reputation is based on its research abilities rather than its training abilities. Finally, the existing staff has very limited skills or experience in marketing. 5.8 Stakeholder Support and Conclusion Stakeholders provided mixed reviews regarding the choice of ISSER as an organisation to receive M&E train the trainer support. A few informants see a great deal of merit in strengthening ISSER for the following reasons: • ISSER provides good quality of teaching and delivers a certificate of attendance to its participants, which is well received. • The pricing structure of workshops is very reasonable and attractive to participants. Indeed, ISSER cannot go beyond a pricing structure ceiling set by the University of Ghana • ISSER does follow-up with its participants on an as-needed basis. If they need support to implement what they have learned, ISSER will do what is needed to support them at no cost. • ISSER can tap into a large network of people to respond to various needs. Although the primary people they call upon are University staff, they do not hesitate to call people from other institutions. • Given the GOG’s scarcity of statistics, data analysis and data collection will play a key role within the new government of Ghana  for poverty monitoring, planning, and preparation of Poverty Reduction Strategies, for example. There will be a tremendous need to ensure that people are provided with skills in statistics, data management, collection and analysis –areas in which ISSER has a very strong reputation. June 2001 21 Assessment of Ghanaian Organisations The majority of stakeholders interviewed, however, share the view that ISSER needs a great deal of capacity building in order to provide quality M&E training: • First, most informants noted that ISSER’s mandate is research and its track record is impeccable, but its experience in conducting monitoring and evaluation is limited. From our perspective however, ISSER has a few very experienced faculty members who have conducted large monitoring and evaluation assignments. • Second, informants are inclined to think that ISSER’s faculty has strong skills in research on several topics (including M&E), but do not bring together a tradition of hands-on M&E studies. From our perspective, however, ISSER has solid experience in all the methodological aspects of qualitative and quantitative research and data collection – which are applicable and often essential to M&E. • Finally, and we would concur with this opinion, all informants indicate that ISSER does not have the facilities for delivering M&E workshops in a professional way. ISSER brings together a mix of strengths and weaknesses with regard to its ability to become a training organisation in M&E. Its solid reputation for research amongst stakeholders, its skills in quantitative and qualitative methods, as well as the presence of three faculty members with very good experience in M&E are favourable conditions for further strengthening of its M&E capacities. In that respect ISSER is quite comparable to the Department of Planning at UST. On the other hand, ISSER’s repertoire of tools and skills in M&E is more restricted than the M&E curriculum of UST, the facilities are inadequate, and stakeholder support for M&E delivery is weaker. ISSER has some potential for M&E capacity-building, but not in the short-term. We would recommend that ISSER be considered for substantive support once there is evidence that M&E has become a more important part of its curriculum (such as, for example, the development and the implementation of the third M&E course), and when ISSER demonstrates a willingness to improve its facilities. In the meantime however, ISSER could benefit from support in launching its second summer course in M&E. The Project Planning, Analysis and Monitoring workshop is being finalised, and ISSER has expressed an interest in having the course content reviewed and discussed with external experts in order to ensure that the course is as strong on the practical side as it is on the theoretical side. Another modest way to build ISSER’s capacities in the short-term would be to support the attendance of faculty members at one of GIMPA’s M&E workshops. June 2001 22 Assessment of Ghanaian Organisations 6. U n i v e r s i t y o f S c i e n c e a n d Te c h n o l o g y, Department of Planning 6.1 People Interviewed • Dr. Samuel Boapeah, Head of Department, Department of Planning – University of Science and Technology, Kumasi, Ghana • Jonas Y. Kokro, Director, SPRING (Africa) International Postgraduate Programme • Kwaku Dwumor Kessey, Senior Lecturer • Martin Bawa Amadu, Lecturer • Yaw Nsiah Peprah, Lecturer • Kwasi Osei Agyeman, Lecturer • Kodjo Esseim Mensah Abranpa, Lecturer • Daniel Kweku Baah Inkoom, Lecturer • Isaac Frimpong Mensa Bonsu, Lecturer 6.2 Background The Department of Planning of the University of Science and Technology (UST) has ten full-time faculty members, and offers four full-time programmes and a series of short-term courses. Located in Kumasi, mid-way to northern Ghana, the department carries out two joint programmes with European universities in the Netherlands. Exhibit 6.1 provides a description of the Department of Planning’s programmes. Exhibit 6.1 Description of the Department of Planning (UST) Programmes PROGRAMMES FOCUS AND SPECIALIZATIONS Undergraduate Programme in Development Development Planning and Settlement Planning Planning Developing student’s skills in the following areas: • Increase production base on optimal combination of factors of production • Restructure rural economies • Improve the development and management of settlements • Develop linkages between lower and upper circuit activities • Promote techniques for effective political-technical interaction and community participation Postgraduate Programme in Development Development planning theories, concepts, methods and strategies. Planning and Management Development Workshops, leading to the formulation of district development plans. Postgraduate Programme in National To equip graduates with the necessary skills for the formulation of Development Policy and Planning national development policies from the economic, social, administrative and spatial perspectives as a coordinated and integrated process. June 2001 23 Assessment of Ghanaian Organisations PROGRAMMES FOCUS AND SPECIALIZATIONS Short-Courses Urban Infrastructure Management Refresher Course for District Planning Officers Policy Analysis for Members of Executive Committees of the District Assemblies Planning for the Promotion of Private Sector Development for Proprietors of Small, Medium and Large Enterprises Budget Planning and Management for the District Budget and Planning Officers Source: UST documents 6.3 Areas of Strength and Weakness in M&E At the moment, none of the courses or workshops given at the Department of Planning focuses specifically on M&E. Focus group interviews with UST staff and a review of the curriculum of existing programmes indicates, however, that M&E is integrated into existing courses, and that several aspects of M&E are dealt with in a modularized form inside larger programmes. For example, as shown in Exhibit 6.2, many typical M&E components are presented in 4 to 8-hour segments. As was explained during the focus group, the aim of the Department of Planning is to build planning capacities and M&E is included as part of the planning cycle. Our analysis of the course syllabus and lecture notes shows that some of the M&E modules are presented from a theoretical rather than a practical perspective. The 4-hour module on Results- Based Management (RBM), for example, provides a strong historical perspective and an understanding of the rationale for engaging in RBM; however it offers little insight into how to manage and conduct evaluations with an RBM approach. UST has not kept systematic records of all students’ course evaluations. Given the lack of data, it is difficult to assess the quality of the course delivery. June 2001 24 Assessment of Ghanaian Organisations Exhibit 6.2 UST Training in Monitoring and Evaluation TOPIC DURATION OF COURSE Logical Framework 8hrs. Performance Monitoring Indicators 4hrs. Results Based Management 4hrs. Programme / Project Evaluation 4hrs Fiscal / Financial Monitoring Not given since 1998 Financial Needs and Needs Assessment 8hrs. Participatory Monitoring and Evaluation 4hrs. Reporting 4hrs. Impact Assessment Not given Cost Benefit Analysis Project Evaluation 8hrs. Project Analysis 8hrs. Project Appraisal Techniques 8hrs. Scheduling, Programming and Development Planning Process 8hrs. Prioritization Project Evaluation 8hrs. Scheduling, Programming and Prioritization 8hrs. Source: UST documents The Department of Planning is strong in all aspects of planning for an evaluation, and in ex-ante evaluation, baseline studies, quantitative analysis, statistical analysis, statistical sampling and questionnaire design, as well as in cost-benefit analysis. The Department’s experience in impact evaluation/ assessment, results-based management and logical framework is limited to a couple of faculty members. Consulting experience varies widely among faculty members. Although some faculty members have conducted more than 75 consulting assignments, the majority of faculty members, who are young professionals, have conducted fewer than ten assignments. Our review indicates that the Department of Planning is often called upon to conduct assignments in baseline or general surveys, such as the following: • Baseline surveys in the Thirteen Project Districts of the Rural Enterprise Project (1999) • Population and Development Baseline Study as an input into the Kumasi Water Supply Project (1999) • Baseline Study of Subonpan Valley Bottom Rice Development Project for the IFAD-funded Smallholder Credit, Input Supply and Marketing Project (1996) • Feasibility Study for the Restructuring of Northern Region Rural Integrated Programme (NORRIP) for CIDA, (1999) • Needs Assessment and Baseline Survey of Selected Districts for Population Planning and Policy Implementation for the National Population Council and the Ministry of Finance (1995) June 2001 25 Assessment of Ghanaian Organisations 6.4 Clientele UST has a wide client system and, although no official record of students and participants was made available for this assessment, focus group data and interviews provided some insight into its client system. The major clientele of the Planning Department consists of full-time students involved in development planning. The Department recruits an average of 85 students annually in its undergraduate programme, and an average of 25 students in its postgraduate programmes. The majority of students come from Ghana, with a small percentage (averaging 5% yearly) coming from the West Africa Region. The second client group consists of entry to mid-level managers at the local level. Today in Ghana, at the District Assembly level, all districts are required by law to develop a planning document. UST is recognized as having the strongest planning department of all Ghanaian institutes, and has trained participants from all 110 districts, and has provided follow-up activities to 45 districts through coaching, and review of planning documents. A third group of participants includes NGO members and Civil Society representatives who manage donor-funded projects, and who need to increase their capacities in project planning. This third group represents approximately 10% of the Department’s clientele and typically registers for short-term courses. 6.5 Faculty, Technical Expertise in M&E The staff of the Department of Planning is impressive. Nine out of ten faculty members have PhD degrees in an area related to development planning. The focus group held with all faculty members showed that the staff is relatively young (the majority are under 40 years of age) and energetic, and that they have a deep understanding of the complexity of Ghanaian issues, as well as management issues from developed countries. However, faculty experience in M&E varied greatly. Only two faculty members acknowledged a strong background and experience in M&E; the other faculty members possess little or no concrete experience in conducting monitoring and evaluation activities. Exhibit 6.3 provides additional details on the staff. June 2001 26 Assessment of Ghanaian Organisations Exhibit 6.3 UST Department of Planning: Profile of Faculty ACADEMIC NAME POSITION IN THE DEPARTMENT AREA OF SPECIALIZATION PROFILE Samuel Nyarkoh Head of Department of Ph.D. Development Planning and Boapeah Planning Economic Development Jonas Y. Kokro Director, SPRING (Africa) Ph.D. Decentralized Development International Postgraduate and Participatory Planning, Programme Institutional Capacity Development Kwaku Dwumor Senior Lecturer Ph.D. Development Planning and Kessey Financial Analysis Martin Bawa Amadu Lecturer M. Phil Development Economics and Public Policy Analysis Yaw Nsiah Peprah Lecturer Ph.D. Social Policy Planning Education and Health Kwasi Osei Agyeman Lecturer, Department of Ph.D. Resource Economics and Planning Planning Kodjo Esseim Mensah Lecturer & Researcher Ph.D. Project Analysis and Abranpa Evaluation, Resource Assessment Daniel Kweku Baah Lecturer, Department of Ph.D. Development Planning, Inkoom Planning Project Formulation and Evaluation Isaac Frimpong Lecturer, Department of Ph.D. Monitoring and Evaluation Mensa Bonsu Planning Kofi Diaw Lecturer, Department of Ph.D. Regional / Spatial Planning, Planning Capacity Building and Social Policy Planning Source: Curriculum vitae of UST Faculty members 6.6 Facilities The UST is located in Kumasi and as such is a convenient location for students and trainees from all across Ghana. The Department of Planning has access to a large campus where it can deliver workshops and courses. In addition, the Department of Planning has developed a relationship with a training facility located three miles from the UST. This training centre accommodated five Department of Planning workshops in 2000, and three in 1999. The centre has three large training rooms (for up to 60 participants) that are fully equipped with overhead projectors, screens, LCD projectors, TV and videos, as well as all the required training material (flipcharts, markers, etc.). There are also six breakout rooms, each with a capacity of 25 participants. The centre has a dining hall that can seat 200 and can provide boarding for up to 80 participants. There is no equipment for distance learning. June 2001 27 Assessment of Ghanaian Organisations 6.7 Marketing Ability The Department of Planning has not yet developed a strong marketing capacity. To date, the demand for its planning curriculum has been strong and there was no obvious need to develop a marketing approach. Faculty members regularly share information about their programmes as part of their ongoing support to District Assemblies. In focus groups, faculty identified the Department’s marketing ability as a potential limitation. The Department has not operated in an environment where its products and services needed to be marketed aggressively. District Assembly staff are sent regularly to attend courses to satisfy Ghanaian policy requirements at the local level, and the majority of NGO participants have donor-funded projects that often include allocation for training. 6.8 Stakeholder Support and Conclusion There is wide support for building the M&E capacities of the Department of Planning of the UST. Ghanaian government representatives, donors (GTZ) involved in projects with the Department, and other training organisations acknowledge the quality of its work and the high calibre of its faculty. In addition to commenting on the effectiveness of the Department, stakeholders strongly suggested that its M&E capacity should be developed for the following reasons: • It would support the development of M&E capacities of District Assemblies and other organisations at the local level, and would attract participants from the North of Ghana. • It demonstrates professionalism in carrying out consulting mandates in M&E-related activities. • Unlike some of the other organisations assessed as part of this review, the Department of Planning at UST has a unique understanding of the needs at the local level. From our point of view, notwithstanding its limitations in marketing services, the Department of Planning is a good candidate for M&E capacity building. Support would need to be tailored to the existing situation of the Department. Training for Trainers (TfT) support in M&E would provide all faculty members a basic understanding of M&E. However, since only two or three faculty members have a very good level of understanding and/or experience in M&E, more sustained and targeted training would be needed to strengthen the knowledge of the younger staff. The approach to building the Department of Planning’s capacities in M&E should include: 1) Support in integrating M&E into the Department’s existing planning curriculum. This could be achieved by providing some technical assistance to review the existing curriculum, to gain a better understanding of the M&E needs at the District Assembly level, and to coach faculty members on re-framing some aspects of the courses or workshops. The Department should then be in a better position to identify specific faculty members with sufficient experience in M&E to act as champions of the M&E courses. 2) TfT in M&E to ensure a common understanding of the basic concepts of M&E among all faculty members. It is unlikely, however, that this basic TfT would be sufficient to allow the younger faculty members to become trainers in this area. More specific training and hands-on experience would be required. 3) Ongoing coaching while the UST M&E course is being adapted, delivered. As is the case with GIMPA, UST would benefit from ongoing support as it develops and adapts its M&E course to its main audience, most likely the District Assemblies. The support could come in the form of regular exchanges, co-design, co-facilitation, as well as exchanges on how to market a program, since UST shows some weaknesses in this area. June 2001 28 Assessment of Ghanaian Organisations In addition, the Department’s faculty members identified two areas in which they require support to build their M&E capacities: 4) Support in implementing an awareness campaign for M&E: There is consensus that an awareness campaign should be developed to sensitise local District Assemblies to the need for adequate monitoring systems. Faculty members suggested a six-month period during which the 110 District Assemblies would be visited (either as part of existing activities or as part of the awareness campaign) to discuss the rationale, the objectives and the components of developing strong M&E skills. 5) As is the case with GIMPA, the Department of Planning at UST is hoping to strengthen a Ghanaian network of M&E training organisations: Given the range of personnel who may require training, at both the local and the central government levels, the faculty suggested that M&E capacity building should be planned in a way that would allow each organisation to build on its strengths and knowledge of an existing client group, and to complement each other. As an example, it was proposed that the Department of Planning devote its energies to training local level staff, while collaborating with other organisations that would target central government staff. Other faculty members suggested clustering components of M&E so that each training organisation could specialize in one or two aspects of M&E rather than providing an entire M&E curriculum. Although these suggestions would need to be discussed and decided by the GOG, we would agree with the suggestion of supporting an awareness campaign for M&E. Discussions with the NIRP officials indicate that such a campaign would facilitate the acceptance by Ghanaian Ministries of any policy proposed by the GOG leading to increased accountability requirements for programme expenditures. As for the development of a network of Ghanaian M&E training organisations, from our perspective the purpose of the network would need to be fleshed out. Networks are costly to maintain and rarely sustainable if there is no strong sense of purpose. What is certainly needed is a level of communication among all the different training organisations that would allow each of them to know what the other organisations are offering. It is worth noting that the Department of Planning at UST has local government (district assembly) contacts/fluency and actual/potential Monitoring and Evaluation skills which have the potential to meet well the demands of the Institute of local Government Studies (see section 7 of the report). Therefore, if the Department of Planning receives support to strengthen its M&E capacities, part of the support should be dedicated to mentoring ILGS. This could be through supporting the participation of new full-time ILGS staffers in TfT activities and M&E workshops delivered at UST, or through staff exchange, where fully trained UST Faculty members could be regular trainers/lecturers at ILGS. June 2001 29 Assessment of Ghanaian Organisations 7. Institute of Local Government Studies (ILGS) 7.1 People Interviewed Dr. S.N. Woode, Director, Institute of Local Government Studies (ILGS), Accra, Ghana 7.2 Background and Strengths and Weaknesses in M&E The Institute of Local Government Studies (ILGS) is a very new organisation, founded in June 2000. As a new organisation, most of the curriculum is in the process of being developed. Our interview with the Director provided a perspective of what ILGS is moving toward (see Exhibit 7.1 ). ILGS exists to provide local government studies, training and research as a local government “Centre of excellence.” The main tasks of the ILGS will be: • Initiating and conducting educational and training programmes for local government staff, assembly members and civil society • Documentation and information services • Research, advisory and consulting services The development of best practice case studies will represent the core of research at the Institute. It will initiate or undertake research on behalf of local authorities, and research into aspects of local government that are of concern to practitioners. One key purpose of ILGS research will be the development of an institutionalized knowledge resource that will be included in the training activities of ILGS. ILGS will also offer consulting services to all District Assemblies in a wide range of disciplines. It will make use of both local and external experts in its consulting assignments. ILGS hopes to encourage the exchange of practices and experiences on a wide range of issues of interest to the districts and the decentralization process. The ILGS training curriculum has not yet been formalised. To date, training has been given on an as-needed basis and in response to very specific needs of district assembly managers. As ILGS begins to understand better the needs of its targeted clientele, it plans to develop a curriculum that responds to those needs. Although our review of the future curriculum and agenda of research shows few components related to M&E, the Director indicated that, if the need was expressed, ILGS would contract a facilitator to develop an M&E course. At this stage, it is too early to assess ILGS’s strengths and weaknesses in M&E or consulting. June 2001 30 Assessment of Ghanaian Organisations Exhibit 7.1 ILGS Planned Areas of Focus TRAINING AND RESEARCH AREAS TOPICS TO BE COVERED District Development Planning and District Plan Formulation Management Vision 2020 and the District Assemblies Strategic Planning and Management for Local Government Public-Private Partnerships for Local District Assemblies and Private Sector Partnerships Level Development Partnerships between District Assemblies and Small Enterprise Information Management for Local Computer Applications in District Administration and Local Level Development Level Development Management Information Systems Information Technology Good Governance Ethics and Values in Local Government Managing the Socio-Cultural Environment of Local Government Non-Governmental Organisations and Local Government Partnerships for Effective Governance Natural Resource and Environmental Natural Resources Management Concerns in Local Development Management Land Management and Local Assemblies Water and Sanitation Management at the Community Level Gender, Equity and Local Women in Local Level Development Government Working with Young People: Responsibilities of District Assemblies Leadership, Management and The Chief Executive as a Manager Supervision in Local Government The Role of Presiding Members District Coordinating Directors and Heads of Departments as Managers Financial Management, Budgeting Budgeting and Accounting and Accounting Financial Management for District Level Functionaries Internal Audit and Inspectorate Functions in Local Level Development. Local Level Economic Development Tourism Development Private-Public Partnerships for Infrastructure Services Delivery Local Government and Poverty Reduction Revenue Generation Strategies Human Resources Management in Training and Development in Local Government Local Government Social Services Delivery in Local Government HIV/AIDS and Local Government Administration and Organisation in Administrative Skills Local Government Communication Skills Source: ILGS brochure June 2001 31 Assessment of Ghanaian Organisations 7.3 Clientele The ILGS delivered three workshops in 2000 to approximately 55 participants from District Assemblies. The topic covered was the role of District Assembly Directors and leadership at the District Assembly level. The course content, the list of participants, and participant evaluations were not available for this assessment. 7.4 Faculty, Technical Expertise in M&E The ILGS has no permanent faculty and operates as a virtual organisation. Its five full-time senior staff occupy management functions at ILGS, and contract personnel as needs emerge. Professors and trainers come from existing training organisations – primarily GIMPA or the University of Ghana – or from consulting firms. The advantage of such a structure is reduced overhead costs and the ability to recruit the trainers best suited to each task. However, as the Director of ILGS points out, this structure has severe limitations, as it is dependant on trainer’s availability, makes quality control and follow-up with students more difficult, and does not project the image of an established organisation. Verbal evaluations from former participants indicate that this constant change in staffing can have a negative effect on the quality of the training since “you never know who the trainer will be; some are good, some are less good, depending on whom ILGS has been able to contract.” It is worth noting that the ILGS derives its strength from the synergies of working with other capacity-building organisations. Currently, ILGS is collaborating (exchanging services, developing joint research) with the Institute for Housing and Urban Development Studies in Rotterdam, and the Maestricht School of Management in the Netherlands, under the Capacity-Building for Decentralization in Ghana Project. 7.5 Facilities The ILGS is located 15 minutes outside Accra, on a very spacious campus. The premises are still being developed, but presently include an auditorium with a seating capacity of 200 and two small classes/ meeting rooms with seating capacity of 40. These classrooms are well equipped, though no distance-learning facilities exist. The campus has residence accommodation for 20 participants and ILGS plans to increase the residence if the need emerges. 7.6 Marketing Ability Fully funded by the GOG, ILGS has had no need to market its services. The three courses that were delivered in 2000 were at the request of the GOG and, as such, financially supported by the GOG. The Director of the Centre expressed a strong interest in including M&E in its existing curriculum 7.7 Stakeholder Support and Conclusion ILGS’s visibility in the Ghanaian scene is gradually increasing. All stakeholders interviewed knew of its existence, although several had a limited knowledge of its mandate or curriculum. The former GOG was strongly in favour of supporting this organisation, which was developed specifically to meet the needs of local government. If the support from the new GOG continues, it is clear that ILGS will have to be a player in any M&E capacity building that takes place in Ghana. June 2001 32 Assessment of Ghanaian Organisations Our recommendations, at this stage, are: • To monitor the evolution of the ILGS over the course of the next year to eighteen month period, to gain a better sense of its direction, mandate, and expertise. The ILGS is new and in many cases it is too early to make a judgement as to its capacities in delivering M&E workshops. A more in-depth review of ILGS should be conducted within the next twelve months to see whether progress has been made in implementing the planned areas of training. The planned program is ambitious, so it will be important to review the content and the evaluations of the participants regarding the courses offered. There is a concern that, given the lack of permanent staff, the quality of the workshops could vary according to the facilitator. The ILGS is aware of this issue and is examining several options regarding staffing. It has considered, for example, retaining a pool of permanent trainers contracted for a set number of days a year. It is also considering hiring some full-time faculty members. Interview data indicates that ILGS is planning to gradually expand its in-house pool of trainers. Although no specific ratio of full-time to external contractors was suggested for the long term, the Director expressed a wish to hire two full-time trainers for the new school year (September 2001). He acknowledged that he was hesitant about hiring and therefore increasing overhead costs at a time of political change that could affect the organisation. • To assess the extent to which the ILGS has included M&E or M&E related topics as part of its curriculum. Presently, none of the planned areas of training deal specifically with M&E. • To monitor the GOG’s relationship with the ILGS. The former GOG created the ILGS and was one of its strongest supporters. Interviews with the National Institutional Renewal Program do not suggest that the new GOG will change its attitude toward the ILGS. There is a need to train national government officials in all aspects of M&E, and this seems to be supported by the new GOG. However, as the new government implements its policies and strategies, attention should be paid to whether they will continue to favour ILGS as a key organisation to deliver M&E training. • To monitor ILGS’s management of external contractors. Although the sub-contractor model has the advantages of allowing for greater flexibility and maintaining overhead costs at a reasonable level, quality problems are sometimes inevitable. ILGS will need to develop a good sub-contractor management system to ensure adequate quality control. If the ILGS continues to be a credible specialized organisation for local level public servants, it will be a good organisation to support for M&E activities. However, if ILGS continues to rely on external trainers, direct capacity building support to ILGS staff through TfT would not be necessary. June 2001 33 Assessment of Ghanaian Organisations 8. Conclusion 8.1 Observations The five organisations that were the subject of this review have all shown an interest in being part of an initiative to strengthen their M&E capacities (although in one instance the inability to provide the data requested for the assessment was not consistent with their stated interest). The organisations have different strengths and weaknesses in M&E skills as well as in their consulting expertise. GIMPA, for example, is already well on the way to organising itself for the delivery of an M&E workshop which has been strengthened following the World Bank’s provision to them of a Training for Trainers M&E workshop. The other four organisations are still in the early stages in terms of their ability to deliver M&E training. All of the organisations and their external stakeholders support the notion of healthy competition for M&E training delivery. They have all begun the process of commercialising their courses, with varying degrees of success. Most have found that the private sector and NGO sector are more interested in M&E than the public sector. External stakeholders have mixed feelings about the ability of these organisations to attract public sector participants to M&E workshops. Although the demand for M&E is increasing gradually, many informants question whether the demand is sufficient to convince the public sector to allocate training resources to M&E activities. The five organisations agree that a sustainable organisational TfT in M&E, adapted to the needs of each organisation, should be carefully considered and elaborated in the next 12 -18 months, and that it should include the following components: twinning, delivery of TfT workshops, coaching, support in adapting the M&E program, and opportunities to participate in international activities on M&E. In addition to support for the development of M&E content, the organisations may also need support in their commercialisation efforts. Strengthening the M&E capacities of several organisations would have several benefits: • It would reduce the possibility of one organization developing a monopoly on M&E in the region. • It would create an M&E service delivery network with greater geographical range. • It would allow each organisation to build on its strengths and, in the longer term, develop areas of specialization in M&E. • It would allow the organisations to continue to serve their established, loyal clientele. 8.2 Recommendations We recommend that the World Bank support M&E capacity-building activities with GIMPA and the Department of Planning at UST, as these two organisations appear to be the most ready and promising in the short-term. The decision to provide M&E support to the other organisations should depend on their development in the next two years. We recommend that M&E capacity-building support be organisation-specific, and that the Ghanaian organisations targeted for immediate support be asked to submit proposals outlining the support they require to deliver M&E training. (Although many suggestions for support were identified through the review, developing a proposal will ensure greater detail and also greater ownership of the capacity-building initiative.) June 2001 34 Assessment of Ghanaian Organisations At a more macro-level, we recommend that the GOG coordinate any M&E caacity-building activities with the various multilateral and bilateral agencies involved in Ghana. In addition to the World Bank, three international agencies – CIDA, IDRC, and to a lesser extent, GTZ – have expressed an interest in joining forces in the M&E capacity-building initiative. These agencies could contribute resources to an M&E Training Fund to be used by Ghanaian organisations on a competitive basis, or could even each support a specific Ghanaian M&E training organisation. We recommend that the GOG encourage the enrolment of civil servants in M&E courses – to accelerate the establishment of a core group of personnel with M&E skills inside government ministries. Given the relatively low demand for M&E from ministries, there is concern that the public sector does not see a great need to invest in M&E. Some bilateral and multilateral agencies and financial institutions have expressed an interest in creating a fund for M&E training to ensure regular public sector participation in these courses until the demand for M&E increases as a result of other, ongoing efforts to strengthen the conduct and utilisation of M&E within the GOG. In coordination with the training organisations, course fees could be totally or partially waived for targeted groups of key participants who would otherwise not be able to attend. June 2001 35 Assessment of Ghanaian Organisations Appendix I Terms of Reference for the Assessment of National Training Organisations The Terms of Reference for the Assessment of Ghanaian National Training Organisations were to: 1) Identify the type and amount of training which the four organisations currently provide in M&E and in related areas: poverty monitoring; log frame and performance monitoring indicators; result-based management; programme evaluation (including impact evaluation); cost-benefit analysis; sector review and policy analysis; rapid review techniques; participatory/stakeholder evaluation; other areas of social science research and quantitative analysis 2) Identify the different types of audience for whom training is provided. Especially those in the public sector and the nature of the training – whether it is introductory (awareness- training) or more advanced. 3) Assess the degree to which each organisation systematically conducts self-evaluation of its training, based on levels 1 to 4 of the Kirkpatrick’s model. 4) Review any existing evaluation of training provided – focusing, where possible on participants’ reaction, on the extent of use by participants in their jobs (i.e. Levels 1-3) 5) Assess the technical quality of the training provided –its content, coverage and rigor – including its pedagogical effectiveness 6) Assess the M&E skills and experience of faculty staff – based on their curriculum vitae and on evaluations, research reports and other reports to which they have made an identified contribution. 7) Assess the quality of training facilities, including the extent of reliable access to computers and distance learning facilities. 8) Assess the reach of the organisations in different regions of the country 9) Assess the entrepreneurial and marketing ability of the organisation – how `hungry` is each organisation to provide raining and consulting services on a fee-for-service basis? Does each organisation have a well-developed marketing plan? 10) Assess the extent to which each organisation already collaborates with other organisation in the provision of training. 11) Consult with key stakeholders and client organisation to ascertain their judgement concerning the strengths and weaknesses of each organisation. These include government central ministries, particularly those involved in provision of training for civil servants; sector ministries which have specialists M&E units; in-country donor representatives; donor headquarters (country and sector operational areas and evaluation offices); regional development banks (especially their evaluation offices); other institutions, such as a national evaluation association. 12) Prepare a report incorporating the above analysis, and including: – Identification of organisation’s strengths and weaknesses in M&E –specially their ability to provide quality training in M&E, and the ability of their staff to undertake consulting work for government, civil society and the private sector. – Identify options to address substantive weaknesses. These might include provision of trainer training in specific topics, or the provision of twinning/mentoring support. June 2001 36 Assessment of Ghanaian Organisations Appendix II List of Documents Reviewed List of Approved Courses for the B.A. (IDS) (1997). Tamale: University for Development Studies. Profile of Department of Planning (1997). Kumasi: University of Science and Technology, Faculty of Environmental and Development Studies. ISSER Short Course on Research Methodology and Report Writing Summary Report (1998). Legon: Institute of Statistical, Social & Economic Research. Lecture Notes on Designing a Monitoring and Evaluation System (1998). Achimota: Ghana Institute of Management and Public Administration. Lecture Notes on Monitoring and Evaluation Concepts and Approaches (1998). Achimota: Ghana Institute of Management and Public Administration. Lecture Notes on Techniques for Monitoring and Evaluation (1998). Achimota: Ghana Institute of Management and Public Administration. Workshop on Monitoring and Evaluation of Development Projects Agenda (1998). Achimota: Ghana Institute of Management and Public Administration. Workshop on Monitoring and Evaluation of Development Projects, List of Participants (1998). Achimota: Ghana Institute of Management and Public Administration. 1999 Annual Report (1999). Accra: Management Development and Productivity Institute. Course Evaluation Ratings (1999). Legon: School of Administration, University of Ghana. List of Labour Studies Diploma Recipients (1999). Legon: Center for Development Studies. Management Development Programme, Course Calendar (1999). Legon: School of Administration, University of Ghana. Medium Term research Priorities, 1999-2003. The Micro Foundations of Development Reform (1999). Legon: Institute of Statistical, Social & Economic Research. NBFI Executive Programme, List of Participants (1999). Legon: School of Administration, University of Ghana. Proposed Curriculum for the Diploma in Labour Studies (1999). Cape Coast: University of Cape Coast. 2000 Management Training Programme Guide (2000). Accra: Management Development and Productivity Institute. 2000/2001 Schedule of Training Programmes (2000). Achimota: Ghana Institute of Management and Public Administration. Action Plan Summary (2000). Legon: Faculty of Integrated Development Studies. Annual Report for the Year 2000 (2000). Accra: University for Development Studies. Capability Statement of the Ghana Institute of Management and Public Administration (2000). Achimota: Ghana Institute of Management and Public Administration. Certificate in Administration and Management, Course Brochure (2000). Achimota: Ghana Institute of Management and Public Administration. Collected Résumés of ISSER Faculty (2000). Legon: Institute of Statistical, Social & Economic Research. Course Evaluation Form (2000). Accra: Management Development and Productivity Institute. Danida Center for International Business (brochure) (2000). Legon: School of Administration, University of Ghana. June 2001 37 Assessment of Ghanaian Organisations End of Workshop Evaluation, Report on Monitoring and Evaluation of Development Projects (2000). Achimota: Ghana Institute of Management and Public Administration. GIMPA Executive Masters Programmes Pamphlet (2000). Achimota: Ghana Institute of Management and Public Administration. Institute of Local Government Studies Pamphlet (2000). Ogbojo: Institute for Local Government. Institutional Capability Statement (2000). Cape Coast: University of Cape Coast. Lists of Participants for 9 Management Courses Given in 2000 (2000). Accra: Management Development and Productivity Institute. Monitoring and Evaluation Capacities in Ghana, A Diagnosis and Proposed Action Plan (2000). Washington, D.C.: World Bank. Monitoring and Evaluation of Community Based Programmes/Projects Report (2000). Achimota: Ghana Institute of Management and Public Administration. Monitoring and Evaluation of Development Projects Workshop Announcement (2000). Achimota: Ghana Institute of Management and Public Administration. NFBI Executive Programme, List of Participants (2000). Legon: School of Administration, University of Ghana. Planning and Budgeting Worksop for District Assemblies in the Central Region of Ghana (2000). Cape Coast: University of Cape Coast. Post-Graduate Diploma in Public Administration, Course Brochure (2000). Achimota: Ghana Institute of Management and Public Administration. Private Sector Development Project Final Report (2000). Accra: Management Development and Productivity Institute. School of Administration Welcome Pamphlet (2000). Legon: School of Administration, University of Ghana. Short Courses on Project Planning and Appraisal and Project Implementation, Monitoring and Evaluation Pamphlet (2000). Legon: Institute of Statistical, Social & Economic Research. Short Course in Research Methodology, Directory of Participants. (2000). Lagon: Institute of Statistical, Social & Economic Research. The Way Forward Report (2000). Cape Coast: University of the Cape Coast. Workshop on Monitoring and Evaluation of Development Projects Agenda (2000). Achimota: Ghana Institute of Management and Public Administration. Curriculum Vitae of Trainers in Monitoring and Evaluation (2001). Kumasi: University of Science and Technology, Faculty of Environmental and Development Studies. Management Training Programme Document (2001). Accra: Management Development and Productivity Institute. Record of Training in Monitoring and Evaluation (2001). Kumasi: University of Science and Technology, Faculty of Environmental and Development Studies. Bening, R. B. (2000). Vice Chancellor's Report with Programme and Statistics. Tamale: University for Development Studies. Kotey, N. (1999). Letter of Accreditation for the University of Development Studies. Tamale. June 2001 38 Assessment of Ghanaian Organisations Appendix III List of People Interviewed 1) Kannae, Lawrence, Dr., Director of Programme, Ghana Institute of Management and Public Administration, Accra, Ghana 2) Yame, M.Y., Acting Deputy Director (Training), Management Development & Productivity Institute, Accra, Ghana 3) Zowou, George, Public relation Officer, Wood Industries Training Centre, Kumasi, Ghana 4) Ammoa, Sakyi Awuku, Dr. , Acting Director, Ghana Institute of Management and Public Administration, Accra, Ghana 5) Zhou, Yongmei, Dr., Economist, Public Sector Management Specialist, World Bank Ghana 6) Smutylo, Terry, Director of Evaluation, International Development Research Centre, Ottawa Canada 7) Mostovac, Marc, Canadian International Development Agency, Ghana Desk, Hull, Canada 8) Heindenhof, Guenter World Bank, Washington D.C., U.S.A. 9) Boapeah, Samuel, Dr., Head of Department, Department of Planning – University of Science and Technology, Kumasi, Ghana 10) Agble, Francis, Dr., Vice Chairman of the Public Service Commission and Co-Chair of the National Overview Committee (NOC) Accra, Ghana 11) Jehu-Appiah, Diana, Coordinator of Training, Integrated Social Development Centre (ISODEC), Accra, Ghana 12) Yawson, Doreen, Assistant to the Director of the Ghanaian Evaluation Society. Accra, Ghana 13) Anarfi, Dr. Director, Institute of Social and Economic Research, University of Ghana, Accra, Ghana 14) Poku, Dr. Acting Director, School of Administration – University of Legon, Ghana 15) Zame, M.Y., Acting Deputy Director –Training, Management, Development and Productivity Institute, Accra, Ghana 16) Woode , S.N., Dr. Director, Institute of Local Government Studies (ILGS), Accra, Ghana 17) Appiah-Korateng, Dr. National Coordinator, National Institutional Renewal Programme, Accra, Ghana 18) Gariba, Sully, Dr.,. M&E Consultant, Gariba Associates, Accra, Ghana. 19) Adei, Stephen, Dr., Senior Lecturer, Ghana Institute of Management and Public Administration (GIMPA), Accra, Ghana. 20) Akuamoah-Boateng, Joseph, Dr., Senior Lecturer, Ghana Institute of Management and Public Administration (GIMPA), Accra, Ghana. 21) Asante, Edward, O. Dr., Senior Lecturer, Ghana Institute of Management and Public Administration (GIMPA), Accra, Ghana June 2001 39 Assessment of Ghanaian Organisations 22) Kwaku Addy Nayo, Christopher, Dr., Senior Lecturer, Ghana Institute of Management and Public Administration, Accra, Ghana 23) Kwabena Boakye, Daniel, Lecturer, Ghana Institute of Management and Public Administration (GIMPA), Accra, Ghana. 24) Gossen, Rhonda. Counsellor (Development). Canadian High Commission, Accra. Ghana 25) Foote, Robert. Decentralization Coordinator, Ghana-Canada CIDA Governance Programme, Accra, Ghana. 26) Akpedonu, E.F., Executive Secretary, School of Administration University of Ghana- Legon, Accra, Ghana 27) Ejisu, Akua S., Lecturer, School of Administration, university of Ghana –Legon, Accra, Ghana 28) Ayirebi, Owusu, Assistant Director, Project Planning, Monitoring, and Evaluation Department (PPMED) Ministry of Food and Agriculture, Accra, Ghana. 29) Hesse, J.H. Dr., GTZ. June 2001 40 Assessment of Ghanaian Organisations Appendix IV Outline of GIMPA’s Program Evaluation Workshop Draft 1 The Monitoring and Evaluation Workshop is a 2 week-workshop focusing on Monitoring and Evaluation (M&E). The course is an introductory course, aimed at providing middle level managers of the public and private sectors an understanding of the importance of M&E as well as the necessary skills to begin developing an M&E system in their organization. The workshop will balance theory and practice; as such participants will be asked to apply new knowledge on a concrete project inside their organization. Day 1 • Opening • The Public Sector Reform in Ghana: History, milestones, and achievements-to date. (2 hours) • Overview of M&E in Ghana in the Public Sector: Existing systems and their challenges; organizational structure, policies and practices. (2 hours). This session will be facilitated by a senior Ghanaian Officer, such as the Head of the National Institutional Renewal Program (NIRP) • M&E throughout the Project Cycle (3 hours) Day 2 • History of Program Evaluation; chronology, main trends, sources. (3 hours) • Logical models in M&E: Rationale for logic models; Introduction to different logic models; deep review of the Logical Framework model (vertical and horizontal logic) (4 hours) GIMPA will need assistance in upgrading and delivering the logical framework module Day 3 • Developing an M&E system (3.5 hours) – Objectives – Performance indicators (overview) – Managing the M&E system (skills needed, systems needed) – Roles and responsibilities GIMPA staff will facilitate this session. Some Ghanaian officials from Project Planning, Monitoring and Evaluation Department (PPMED) may be asked to present also. • Managing for result: the result-based approach (3.5 hours) GIMPA will need assistance in designing and delivering the result-based approach module On DAY 3, the assignments will be identified, teams will be formed. Each team will have to make a presentation on Day 9. June 2001 41 Assessment of Ghanaian Organisations Day 4 • Logical models (continued) (3 hours) • Indicators; Purpose of indicators, developing indicators in the Public Sector; use of indicators for monitoring performance. In this session, participants will work with real sets of indicators from their organisation or another Ghanaian Ministry. (4 hours) Day 5 • Introduction to data collection methodologies: Planning for data collection (1 hour) • Developing questionnaires (3 hours) • Conducting effective interviews (1.5 hours) • Conducting effective focus groups (1.5 hours) • Review of week 1 Day 6 • Data collection methodology (continued) • Participatory approaches (3 hours) • Rapid assessments (3 hours) • Review of the week (1 hour) Day 7 • Data analysis and interpretation: overview • Analysing and interpreting quantitative data (2 hours) • Analysing and interpreting qualitative data (2 hours • Data reporting (3 hours) Day 8 • Managing internal and external evaluations: overview (1.0 hour) • Steps in selecting a consultant (2.0 hours) • Developing Terms of Reference for an evaluation (4.0 hours) Day 9 • Presentation of assignments • Forum for discussion on the assignments • Evaluation of the course • Closure June 2001 42