Monitoring and Evaluation Working Paper 5 December 2000 2 1836 Integrating Capacity Development into Project Design and Evaluation - ~~IL Approach and Frameworks Charles Lusthaus Marie-Helene Adrien Peter Morgan GEF Integrating Capacity Development into Project Design and Evaluation Approach and Frameworks Charles Lusthaus Marie-He61ne Adrien Peter Morgan Monitoring and Evaluation Working Paper 5 December 2000 Authors Charles Lusthaus, Universalia Management Group Marie-Helene Adrien, Universalia Management Group Peter Morgan, Consultant Published 2000 Global Environment Facility This paper may be reproduced in whole or in part and in any form for educational or nonprofit uses, without special permission, provided acknowledgment of the source is made. The Global Environment Facility secretariat would appreciate receiving a copy of any publication that uses this paper as a source. Copies may be sent to GEF secretariat, 1818 H Street NW, Washington, DC 20433. No use of this paper may be made for resale or other commercial purpose without prior written consent of the Global Environment Facility secretariat. The designations of geographic entities in this document, and the presentation of materials, do not imply the expression of any opinion whatsoever on the part of the GEF concerning the legal status of any country, territory, or area, or its authorities, or concerning the delimitation of its frontiers or boundaries. The views expressed in this paper are not necessarily those of the GEF or its associated agencies. ISBN 1-884122-82-5 ISSN 1020-0894 ii Preface The GEF Monitoring and Evaluation (M&E) team is tasked with analyzing and documenting GEF results. Until now, conclusions of these efforts have been in the form of evaluation and study reports, annual Project Performance Reports, and GEF Lessons Notes. With the introduction of the M&E Working Papers series, we are publishing reports that are not full-fledged evaluations, but nevertheless deserve attention. Many of the issues and early results that these reports identify will be pursued later in broader evaluations to arrive at more definite conclusions. We expect the M&E working papers to be a valuable catalyst for promoting dia- logue on issues and results of importance within GEF's operational areas and efforts. We therefore look for- ward to your feedback and suggestions. Please contact us through the coordinates listed below and visit the GEF Web site to find out more about the Monitoring and Evaluation program. The present approach and framework paper on Integrating Capacity Development into Project Design and Evalzuation is the result of work carried out under the auspices of the GEF monitoring and evaluation work program during 1999-2000. In the 1998 Project Implementation Review a need was identified to develop an approach for addressing monitoring and evaluation of capacity development in GEF projects in a more sys- tematic manner. The M&E Team engaged the Canadian consulting firm, Universalia Management Group, to provide expertise for the work. Selected projects were included in a desk study of how they approached ca- pacity development. Close communications were established with the GEF-UNDP Strategic Partnership on the Capacity Development Initiative (CDI). The approach paper is intended to make inputs to the CDI, but it is not a product of it. Jarle Harstad Senior Monitoring and Evaluation Coordinator GEF Corporate Monitoring and Evaluation Team 1818 H Street, NW Washington, DC 20433, USA Telephone: (202) 458-2548 Fax: (202) 522-3240 E-mail: geflessons@gefweb.org Web: http://www.gefweb.org iii Acknowledgements The preparation of this approach paper would not have been possible without the support and valuable contri- bution of several individuals and organizations. The work was guided by an interagency working group that met twice during the project. The participants in- cluded: John Hough and Martin Krause (UNDP), Tom Hamlin (UNEP), Tijen Arin and Emilia Battaglini (World Bank), Avani Vaish (GEF Secretariat), and Jarle Harstad and Juha I. Uitto (GEF M&E Team). Others who contributed to the work by providing valuable insights and suggestions included Turhan Saleh (UNDP), Julian Blackwood and Patrick Grasso (World Bank Operations Evaluation Department), Ray Rist (World Bank Institute), Rohit Khanna (World Bank), and Patricia Bliss-Guest, Alan Miller, Colin Rees, and Johan Wide (GEF Secretariat). We would also like to acknowledge the contributions of our colleague at the GEF Secretariat, Elizabeth Mook, for editing the report. Juha I. Uitto Task Manager iv Contents Introduction ...................................................................I The GEF Situation: Its Effects on Monitoring and Evaluation of Capacity Development ............. ....................3 Introduction .............................................................3 Relations to Conventions .............................................................3 Country-Drivenness.3 The Tni-Agency Structure ............................................................3 Reliance on Projects to Support Capacity Development . ............................................................4 A Means Rather Than an End ............................................................4 Designing Capacity Development Interventions for GEF Projects .......................................................5 Structural Constraints, Modes of Operation, and Incentives ............................................................S5 Limitations of Existing Tools for Monitoring Capacity Development ................................. .................6 Application of Usable Knowledge ............................................................6 Need for Support Services .............................................................7 The Indicator Dilemma ............................................................7 Implications for GEF ...................................................................9 Appendices Appendix 1. A framework for a change strategy ................................................................... 11 Appendix II. Planning and evaluating capacity development at the individual level ........................................ 12 Appendix III. Planning and evaluation of capacity development at the organizational level ............................ 13 Appendix IV. Planning and evaluating capacity development a the societal level ............................................ 17 Appendix V. Framework for conducting a functional analysis ................................................................... 18 V Appendix VI. Monitoring the capacity development ............................................................... 19 Appendix VII. LFA and dapacity development models .................... ........................................... 20 Appendix VIII. How to make judgments in order to identify appropriate benchmarks .............. ...................... 21 Appendix IX. GEF cases and examples ............................................................... 22 Appendix X. List of assessment tools ............................................................... 25 vi Introduction The Global Environment Facility (GEF) has previ- issues. The intention is not to focus on strategic issues ously reviewed its performance on capacity develop- concerning the role of capacity development in the ment and associated issues related to the design and GEF context. Nevertheless, strategic concerns are management of its project portfolio. Definitions of highlighted when operational issues so require. No capacity development still vary from one agency to time was available to carry out a detailed review of the next. Although there is currently no single agreed the projects and practices of GEF, and this paper does definition of, or approach to, capacity development in not put forward detailed operational recommenda- GEF itself, a vision emerging in the international tions. It is also not intended to be an analysis of the development community sees capacity development larger stratcgic issucs facing GEF, most of which will as a complex process of innovation and adaptation be addressed by the work of the UNDP-GEF Strate- involving multiple changes at different individual, gic Partnership, the Capacity Development Initiative organizational, and institutional levels' . (CDI), which began in 2000. The 1998 GEF Project Performance Report reviewed This paper does provide a brief review of the current progress on capacity development and recommended status of capacity development work within GEF and a series of actions that included giving greater atten- highlights some areas for improvement that will re- tion to identifying capacity needs and assessing better quire follow-up work. An associated purpose of the the results and qualitative impacts of GEF's work in paper is to engage mid-level managers within GEF on capacity development. Despite this attention, the per- the issue of capacity development and to contribute to ception remains inside GEF that more needs to be the process of rethinking some operational ap- done to integrate capacity development activities sys- proaches. A workshop was held in February 2000 in tematically into GEF-supported projects. This paper, Washington to develop specific proposals for commissioned by the Corporate Monitoring and operationalizing capacity development within the Evaluation Team within GEF, addresses the issue GEF context. once again. The objectives of the paper are: Many of the conclusions in this paper stem from a * To identify ways to integrate capacity develop- GEF workshop held in Washington on October 22, ment objectives at the project planning stage; and 1999. The authors also reviewed a small sample of GEF projects, which GEF operational staff believe * To develop a framework and indicators for reflect some of the strengths, weaknesses, and con- evaluating the performance of capacity devel- straints of GEF's current approach to capacity de- opment activities. velopment. The limits of this paper should be made clear at the These projects are used as examples throughout the outset. The emphasis of the work is on operational paper. The choice and selection of the project men- 1 See Charles Lusthaus, Marie-Hel6ne Adrien. and Mark Perstinger, Capacity Development: Definitions, Issues and Implications for Planning, Monitoring and Evaluation, August 1999, Universalia. I tioned was made by the GEF Secretariat and imple- * A presentation of the GEF situation and how it menting agencies as representative of their best prac- affects GEF abilities to design, monitor, and evalu- tices in planning, monitoring, and evaluating capacity ate capacity development; development. * A discussion of implications and recommenda- As an approach paper initiated by GEF's Monitoring tions, which highlights several operational issues and Evaluation Team, this does not constitute an out- limiting the ability of GEF and its lAs to monitor put of the ongoing GEF-UNDP CDI. It is, however, and evaluate capacity development; and intended to contribute to it. Beyond this introduction, the paper includes three sections: * A series of technical annexes with tools and instru- ments to facilitate M&E of capacity development. 2 The GEF Situation: Its Effects on Monitoring and Evaluation of Capacity Development Introduction funded activities in the focal areas shall be in full conformity with the guidance provided by the COP. The way international funding agencies integrate or GEF is, nevertheless, able to provide funding to eli- mainstream issues such as gender or capacity devel- gible parties also outside of the convention contexts. opment appears to be shaped by a combination of In such a case, it must ensure that assistance is fully factors including: consistent with the guidance provided by the COPs of the two conventions. * Relationship of the issue to broader contextual fac- tors and incentives that shape the purpose and be- Country-Drivenness havior of the organization; This issue relates to ownership. Countries identify * Nature and workload demands of the particular their needs and GEF responds. It is in how the re- issue; sponse takes place that we need to link the issue of capacity development. * Complexity of the issue; and The operational strategy states that: * Professional background and inclinations of the staff. GEF activities will be consistent with, and support- ive of the recipient countries' own actions for sus- The context within which GEF engages in capacity tainable development. GEF programs and projects development is characterized by ten factors, which will be country-driven ... and will be linked with na- should be understood in order to provide guidance for tional sustainable development efforts. Public con- improving planning, monitoring, and evaluation. sultation and effective involvement of local These factors need to be considered when approach- communities and other stakeholders will enhance ing capacity development in GEF. the quality, impact, relevance, and national owner- ship of GEF activities. Relations to Conventions The Tri-Agency Structure GEF is the financing mechanism for the Convention on Biological Diversity (CBD) and the United Na- In addition to the influence of its mandate and overall tions Framework Convention on Climate Change direction, the tri-agency structure of GEF makes (FCCC). The GEF operational strategy2 incorporates reaching a consensus on approaches to cross-cutting the policy guidance of the Conference of the Parties issues such as capacity development more demand- (COP) to the CBD and FCCC. It states that all GEF- ing. Each of the implementing agencies, however, has 2 Global Environment Facility Operational Strategy, GEF, February 1996. 3 specific comparative advantages related to capacity * Country eligibility; development that can be built upon within the GEF context and that can provide complementarities be- * Policy and program framework of the proposed tween the actions of the various implementing project; agencies. * Technical review; * UNDP tends to focus on strategic management and process approaches in the design and man- * Social assessment and consultation; agement of capacity development, as in the case of the Interim Assessment of Biodiversity En- * Capacity building; abling Activities for National Biodiversity Strate- gies and Action Plans. * Training, institution building, planning and policy development, targeted research, linkage of capac- * UNEP gives more emphasis to technological im- ity building to enabling activities and to invest- provements in support of environmental manage- ment; ment, as in the case of the provision of Institu- tional Support for the Protection of East African * Incremental cost; and Biodiversity. * Monitoring and evaluation. * The World Bank, by contrast, works more from the perspective of institutional economics, and Nevertheless, given the need of capacity develop- pays attention to incentives, competition, and the ment interventions for operational flexibility and a influence of institutions or the rules of the game long-term planning horizon to achieve sustainable on organizational effectiveness. impact, the inherent limitations of the project ap- proach remain. The 1998 Project Performance Report Given the organizational structure of GEF, it is diffi- suggested GEF "move from an organizational culture cult to settle on an operationally specific framework based on project approval to one more focused on for capacity development that can accommodate all achieving and measuring project and program re- three perspectives, and yet still be sufficiently useful sults."3 This resulted in a new initiative-Driving for as a guide to decision making for operational staff. Results-that is in discussion between the various There is, nevertheless, need to agree upon a harmo- GEF entities. nized framework for capacity development within which all of the GEF entities would place themselves. A Means Rather Than an End Reliance on Projects To Support Capacity Capacity development is not considered a key objec- Development tive or end result of GEF assistance, as indicated by its mission statement. The effectiveness of capacity development is limited by GEF's reliance on the project as a mechanism to The Global Environmental Facility (GEF) is a transfer resources in support of capacity develop- mechanismfor the purpose ofproviding new, and ment. Project interventions are generally narrow, dis- additional, grant and concessional funding to connected and short term, and GEF has tried to meet the agreed incremental costs of measures to compensate for this deficiency by making specific achieve agreed global environmental benefits in provisions for capacity development in its project the areas of biological diversitv, climate change, cycle. international waters, and ozone layer depletion. Land degradation issues, primarily desertifica- Current guidelines state that projects will be reviewed tion and deforestation, as they relate to the four by the GEF Operations Committee, taking into ac- focal areas will also be addressed. In carrying count the following considerations, as appropriate: out its mission, the GEF will adhere to key opera- 3 t998 Project Performance Report, GEF, p.38. 4 tional principles based on the two conventions, * The transformation process addresses specific de- the GEF instrument, and Council decisions. velopment problems or content areas. In the case of GEF, these can include all the objectives of the GEF is, of course, aware of the importance of capac- Convention on Biological Diversity (CBD) and ity development and most of the projects that it sup- the Framework Convention on Climate Change ports include components that focus on the issue. But (FCCC), international waters, and ozone deple- GEF views capacity development primarily as a func- tion. Capacity development can be seen as tional means toward more substantive program ends. More specifically, it sees capacity development as - an activity (e.g., workshop or training in land- essential to "ensure the sustainability of global envi- use planning for biodiversity conservation); ronmental benefits" and to "reduce the risks caused by uncertainty." Capacity development as a means is - upgrading of skills to collate, retrieve, man- grouped together with activities such as human re- age, and use biodiversity data; source development and institutional strengthening. - human resource development in the area of The combination of its formal mandate, the direction safety in biotechnology; training in informa- it receives from the Conference of the Parties, and the tion technology); or guidance of the GEF Council do not give GEF much specific direction on capacity building, either in terms - a form of technology transfer (e.g., installa- of project selection and design, or in terms of opera- tion of eco-tourism management system to tional management. However, under the ongoing protect a targeted species, influencing the GEF-UNDP CDI, an assessment of the convention markets in benefit of renewable energy tech- guidance on capacity development is being under- nologies). taken. The challenge for funders such as GEF is to help Designing Capacity Development Interventions participants match the scope of their interventions to for GEF Projects the nature and complexity of capacity development issues to be resolved. The keys to meeting this chal- A 1998 consultation by the international working lenge are encouraging participant comprehension and group on capacity development found that among the ownership, managing technical and organizational multilateral agencies interviewed, there was no com- complexity, and sequencing various activities. monly accepted approach to capacity development. However, there are new approaches to capacity de- Structural Constraints, Modes of Operation velopment based on individuals, organizations, and and Incentives institutions. These new approaches have as their foundation some of the following assumptions: GEF faces many of the same "soft" constraints to progress on capacity development as do other fund- * Capacity development interventions frequently ing agencies. Many GEF professional staff have tech- aim to improve the performance of complex "sys- nical backgrounds and training in the environmental tems" (e.g., river basin or watershed protection) and physical sciences or public policy subjects such that involve individual and organizational actors as economics. The GEF Secretariat requires more at the group, community, organizational, network, training or familiarity with public management and and institutional levels. In this sense, capacity program implementation. development is increasingly seen as a multi- dimensional activity that can involve reforms The resulting lack of engagement with capacity de- at each level, as well as efforts to alter the inter- velopment is compounded by a pattern of staff incen- relationships among actors at different levels. tives that continues to emphasize 1) design and approval over implementation and delivery, and 2) * Capacity development is essentially a process of the achievement of tangible, short-term objectives change or transformation that aims to induce vari- over the slower progress of ambiguous, longer term ous actors to take on new responsibilities, skills, activities such as capacity development. behaviors, values, and policies. 5 Furthermore, interviews with some staff indicate that to explain the functions underlying the needs implied the constantly increasing workload mitigates against by fulfilling convention requirements. staff engaging in the complexities associated with capacity development. Over the last years, the GEF As such, we began to outline a tool that could support portfolio has grown rapidly and a number of new more systematic functional analyses. We were also requirements for project approval (i.e., incremental asked to generate tools linking the targets for capacity cost analysis, logframe analysis [LFA]) are now man- building-namely organizational and policy datory. As well, a medium-size funding window was change-and planning and monitoring tools such as added. The number of implementing agency staff did the logical framework analysis or LFA. Increased not grow at the same pace. Hence the timc available emphasis needs to be placed on the operationalization for second level managers in the implementing agen- of capacity building. cies, an important target group for the proposed change process, is increasingly limited. Appendix VI provides further discussions and instru- ment on this issue. Many feel they do not have time to embrace and engage in the proposed change processes implied by Application of Usable Knowledge capacity development. Indications are that the same is true for GEF Operations Committee managers. In GEF, as well as its implementing and executing order to bring these target groups on board as GEF agents, needs to create a broader range of ways to explores improving project monitoring and evalua- learn from their experience with capacity develop- tion associated with capacity development, serious ment. This includes, among other activities, devoting thought should be given to freeing up time and energy more resources to analyzing and patterning insights through delegation, decentralization, and simplifica- from the field as well as sharing lessons across units tion within the GEF project cycle. and agencies. All three implementing agencies are currently trying to improve their capacity to learn Limitations of Existing Tools for Monitoring from their experiences and manage the resulting Capacity Development knowledge. This leads, in turn, to changes in staff skills and incentives, project design, monitoring and As the conceptualization of capacity building be- evaluation techniques, and even the organizational comes clearer, many agencies are now devoting their culture and style of GEF. This paper does not go into attention to developing useful tools for engaging in detail on these points, but simply calls attention to the capacity building.4 One contribution of this approach broader implications of any effort to improve learn- paper is to suggest operational tools for planning and ing and apply the resulting usable knowledge. evaluation of capacity building in GEF projects. A series of tools that might contribute to more sys- The situation is quite clear. Today, we recognize that tematic operationalization of capacity building was many of the compliance problems faced by countries examined with this goal. supported by GEF are linked to more general devel- opment issues. Piecemeal solutions generated exter- We set out some of these tools in the next section. nally are rarely successful. The GEF and donors need While some tools are more advanced in their utility to work with partner countries to figure out the best than others, they do represent a starting point for the ways of solving issues arising from implementing development of such tools for GEF. The tools range conditions within the convention areas. It is the coun- from approaches to capacity development as change tries themselves that need to create ways to learn management to conceptual frameworks and indica- from their experience. It is the countries themselves tors for individual, organizational, and policy level that need to create usable knowledge to apply to their capacity building. In deliberating on capacity build- setting. ing, we concluded that more guidance was necessary 4 For example in November 1999, the Canadian International Development Agency sent out a draft volume of articles, frameworks, and tools that they feel would be useful for operational staff. These will be tried and updated over the next little while. 6 Creating and using knowledge to solve complex de- information to generate more effective projects? How velopment problems requires different ways of plan- can processes of organizational change be monitored ning, monitoring, and evaluating project work. To in ways that can convince skeptical observers ob- begin with, it starts with the assumption that many of sessed about measurable results? And how can this be the problems faced by the countries supported by done when projects are managed in an iterative way? GEF cannot be solved simply with technological so- lutions. The solutions require process, interaction, In addition to the challenge of devising indicators to learning, and problem solving. Also, they require monitor such processes, GEF faces other constraints. funds to support the learning among GEF, its imple- Some capacity development indicators, as they relate menting agencies, and its executing agencies. Learn- to GEF and capacity development, are discussed in ing from experience and using knowledge to promote Measuring Results from Climate Change Programs: GEF goals in partner organizations is a crucial aspect Performance Indicators for GEF 5. The report of the capacity building agenda. stresses that capacity development and institutional strengthening are important and are already sub- Need for Support Services sumed in the other indicators. But it goes on to say that "...efforts to measure capacity building more Any effort at mainstreaming new practices such as specifically would be relatively high in cost and could capacity development needs to be facilitated and sup- be intrusive in nature." ported in a number of critical areas within GEF. The issue must have committed advocates within the or- One way to provide clarity to the evaluation of ganization itself. The problem that capacity develop- change is to develop frameworks that can guide the ment is designed to solve and the benefits it intends to planning and monitoring of capacity development produce must be widely understood. Staff must have work. For example, at the individual level, training is some incentives to engage themselves. Some reason- often used to help one or more individuals develop able level of organizational and technical support that skills to engage in activities that can support the helps staff at the operational level to deal with the change process. How does one describe the chain of added workload and skill demands must accompany events in the development of this capacity and its its implementation. relation to the GEF program objective (i.e., develop- ment problem)? The challenge then is one of capacity building within GEF, of managing a process of organizational inno- One approach could be for GEF and its implementing vation and change that results in altered patterns of and executing agencies to agree to a set of flexible behavior. A number of other donors and international frameworks that allow them to clarify the changes funding agencies (e.g., the Department for Interna- they want. This is the approach taken by the World tional Development in Great Britain) have reviewed Bank Institute (WBI) as it attempts to better under- their practices on capacity development using a set of stand the effects it is having as a result of its training questions put together by the Development Assis- activities. tance Committee of the OECD. This list of questions is attached as Appendix II. Litwicks's (1990) work has helped EDI [now WBI] evaluation teams to identify impact indica- Capacity Development: The Indicator Dilemma tors for a number of the institute's training pro- grams. The approach proposed in this framework In most of the interviews and workshops held in is simple. It starts from changes in knowledge, connection with this paper, one issue kept recurring: skills and performance, which can be observed in How can GEF better monitor and evaluate the out- individual participants as the result of an inter- comes of its capacity work and use that insight and vention (training) ... 6 5 GEF Monitoring and Evaluation Working Paper 4. Measuring Results from Climate Change Programs: Performance Indicators for GEF GEE. September 2000. 6 Suzanne Taschereau, Evaluating the impact of training and institutional development programs: a collaborative approach. EDI, EDI Learning Resources Series. 1998. 7 GEF might wish to consider two other principles with GEF needs to focus as much on process activities as it respect to the issues of performance monitoring and does on final results. It is the drivers of performance measurement of capacity development. First, GEF and capacity that, in many cases, should be the focus needs to supplement its focus on indicators (i.e., per- of monitoring attention. Identifying indicators and formance measurement) with a broader concern for measuring the results of capacity development serves performance management on projects. The use of the needs of multiple stakeholders. As such, they indicators makes more sense when it is part of a wider should be designed and negotiated as part of a partici- effort to induce participants to manage strategically. patory process. Clarifying these processes and in- This, in turn, leads to the second idea that the key tended results is a capacity development activity in objective of performance monitoring is better man- itself. agement at the field level. If that objective is achieved, then GEF must demonstrate that account- ability requirements can also be met. But the reverse is not the case. 8 Implications for GEF The GEF family should engage in more systematic * Sharing personnel with capacity development ex- dialogue on capacity frameworks. perience amongst the GEF Secretariat and its implementing agencies. The GEF family needs to agree on a framework to use at each of the levels. We suggest that the GEF Secre- GEF will need to make efforts to increase staff tariat collaborate with the lAs on developing mutu- skills and knowledge about the use and applica- ally acceptable frameworks. The design emphases tions of any resulting framework for capacity and assumptions of UNDP, UNEP, and the World analysis. Bank all have strengths and comparative advantages that could inform a shared and aggregated approach Once GEF has agreed to specific frameworks, lAs to a capacity development framework. Given its me- will need to use these frameworks to describe their diating and connecting role among the three agencies, capacity development initiatives. Even some basic the GEF Secretariat could play a useful role in think- shared principles and a more common vocabulary ing through a more systematic approach to capacity among the three implementing agencies would con- development for the environment. Indeed, GEF could tribute to effectiveness. Both implementing agency play a bigger role in international thinking about ca- and GEF Secretariat staff need training to make sure pacity development than it does now. As part of this that they are able to 1) understand the frameworks exercise, GEF might wish to review the growing range and 2) use the frameworks in their design and report- of frameworks that are emerging internationally. ing on the capacity development components of their GEF projects. If GEF intends to give more priority to capacity issues, it will need to put in place some capacity Similarly, as implementing agencies begin to include development support services for staff, both in a strategy for change in the project design and then house and externally. follow up by reporting on its implementation, there will be a need to orient implementing agency and These would include some or all of the following: GEF Secretariat staff on change management strat- egies. Some examples of staff skills enhancement * Employing some full-time technical expertise inside include: GEF; (It is difficult to imagine any organization making progress on an issue in the absence of any * Creating continuous capacity development learn- in-house technical expertise and capability.) ing opportunities for staff, e.g., creating opportu- nities where people will learn by being involved * Training or coaching GEF staff on capacity issues; in capacity development activities; or learning through communicating about capacity develop- * Recruiting GEF operational staff with suitable ex- ment among agencies and inside agencies; or perience or knowledge; and learning through coaching; 9 * Promoting inquiry and dialogue about capacity results of change processes are seen only after three development inside GEF; to five years. Effects on the performance of an organi- zation often take even more time. The implication is * Encouraging collaboration or team learning on that many of the benefits of capacity development capacity development; will not be realized within a project cycle. * Establishing systems (technological and proce- Financial implications need to be taken into account. dures) to capture and share agency and inter- Capacity development involves both a content aspect agency learning about capacity development; and a change strategy. To date, project budgets have taken into account the specific capacity development * Encouraging staff to further skills and knowledge activities or the specific technology transfers, but not about capacity development; and the cost of the change process. It is important that GEF recognize the need for planning adequate * Coordinating learning events between the GEF project budgets to ensure that the change process can Secretariat and the lAs to enable them to link theo- occur. The budget increase per project will depend on retical and operational knowledge about capacity the nature and scope of the change expected. For development. implementing agencies, the financial implication will center on ensuring that project designs include bud- GEF may wish to consider the idea that specific gets that reflect the resources required to support the change processes need to be planned into a project change process, and that project reports show ex- for each individual, organization or societal group penses associated with the change process. targeted by a project. The GEF family needs to do more thinking and If the desired result is improving the ability of a work on implementing a more considered ap- targeted government agency to enable it to engage in proach to iteration. the protection of an area of biodiversity, it is as neces- sary to think through the logic of the change pro- Most projects resort to more ad hoc and iterative cess- approaches through circumstance - as initial designs inevitably prove inadequate or incomplete. What * How to engage in participatory work might be helpful for GEF is the adoption of more participative and iterative approaches to project de- * How to create ownership sign, management, monitoring and evaluation. How- ever, in order to work, these approaches require * How to develop new ways of thinking about prob- resources, both human and financial. All the imple- lems, issues and solutions and so forth menting agencies have some form of iterative ap- proach and could share experiences and lessons. -as it is to provide the content activities required within a log frame. One of the key issues to resolve in such approaches is the apparent tension between the needs dictated by There needs to be an altered view of the length of the bureaucratic accountability (e.g., prediction, control, project cycle, or at least the flow of benefits. Project- structure, short-term results, measurement, and re- oriented logical frameworks often do not adequately porting to stakeholders) versus those more suited for consider the time dimension of the change process. In field management and performance (e.g., flexibility, most organizational development interventions, the adaptation, management, and longer time frames). 10 Appendix I A framework for a change strategy There is an increasingly impressive literature on components, while linear, do not act as such on the change management. In general, it identifies four ar- ground. One is constantly clarifying expectations and eas that need planning and thought in an iterative way. communicating the vision as well as trying to get Our experience with capacity development parallels things done and managing the obstacles related to that of management of change. implementing the work required. Similarly, the pro- cesses of consolidation and exiting are central to the In carrying out capacity development work, one needs process of change. to pay attention to each of the four components. The Components of a Change Plan to Support the Project Change Strategy Components Activities that Support the Component and in CD Projects Need to be Considered in Projects Entry: Developing a shared vision of Preparation-make sure organization or group is ready for change process the desired change Collaborative definition of the problem or capacity development area Identification of resistance, potential crisis agreement as to the course of action and preparation Creating a guiding coalition to lead change Communicating the vision for change Implementing change Empowering broad-based action Getting rid of obstacles Changing systems or structures that undermine the change process Encourageing risk-taking and non-traditional ideas, activities, actions Generating short-term wins and visibly rewarding those who made the wins possible Consolidating change Using increased credibility to change additional elements of the system that will not support the overall capacity development objectives Reinvigorating the transformation process with new themes, change agents Exit: Sustaining change Anchoring new approaches in the culture Articulating the connections between new behaviors acquired through activities, and organizational or societal success and sustainablitiy 11 Appendix 11 Planning and evaluating capacity development at the individual level GEF projects often identify individuals or groups of ventions to be clear about the type of change desired. individuals as the target of change, and changes in The framework below was adapted from the work of individuals are often a prerequisite to other GEF Litwick; it identifies the different levels of change to project and program outcomes. Although building consider when doing capacity development work individual capacity is often part of a larger GEF inter- with individuals and the type of change to plan and vention, it is nevertheless important for those inter- monitor. Framework for change' Targets for training Explanation and indicators Individual Changes in individuals knowledge, skills or attitudes Individual performance on the job Application of training on the job Department or organizational changes Affect of application on department or group Department or organizational performance changes Affect on overall organizational performance Policy process changes Affecting policy process Policy outcome changes Changes to policy (the rules of the game) Societal changes Changes to target beneficiary populations 7 Suzanne Taschereau, Evaluating the impact of training and institutional development programs: a collaborative approach. EDI. EDI Learning Resources Series, 1998. 12 Appendix III Planning and evaluating capacity development at the organizational level Although many GEF projects are part of larger inter- development, IDRC and Universalia (1995, 1999) de- ventions that support organizational improvement, veloped a flexible framework that captures many of there is no consistent framework used to help plan and the types of capacity interventions identified in GEF monitor GEF projects that target organizations as an project work. aspect of program work. In the absence of a frame- work, every GEF organizational capacity project uses The Universalia-IDRC framework (shown on the fol- its own approach and language in order to describe the lowing page) is an example of standardizing capacity changes it wants. While there is no inherent difficulty development as it relates to the entity or organiza- in using any type of framework at a project level, at a tional level. The framework suggests that the perfor- programmatic and fund level, it becomes difficult to mance of an organization is affected by three main learn and report on the results. factors (the motivation of the organization, the capaci- ties of the organization, and the external environ- Applying a consistent framework, similar to the one ment). The framework defines eight key capacities suggested for individuals, can help improve commu- that affect performance. By inference, capacity de- nication, planning, and monitoring of the content of velopment work in organizations is a process that organizational capacity development work. While helps organizations develop their abilities to improve there are many approaches to organizational capacity performance. 13 Universalia-IDRC Framework for Change Targets for organizational Explanations and key ideas to develop indicators Examples of indicators change Changes to organizational - Improvement in reaching mission or goals-environmental (Note: All effectiveness indicators are organization-specific performance changes that are the organizational mission and linked to and are not included in this table) GEE program objectives * Costs per client served * More efficient use of resources * Outputs per staff * More relevant to major stakeholders * Timeliness of service delivery * More financially viable * Stakeholder satisfaction * Changes in reputation amongst stakeholders or competitors * Quantity of changes in services and programs related to changing client needs * Number of new financial contributors * Level of innovation and organizational adaptiveness Changes to capacity that affect * Improved ability to: * Existence and use of a strategy to drive the organization performance - Lead strategically * Organizational niche identified - Structure and re-structure * Roles and responsibilities clear - Plan, implement, and monitor financial systems * Planning and monitoring processes embedded in all - Plan, manage and evaluate human resources program management systems - Access needed infrastructure * Training programs prioritize performance areas - Manage organizational processes * Decision-making processes appropriately structured . . . ~~~~~* Standardized work processes in place, and used - Plan, implement and monitor organizational programs , and work * Budget planning used as a management tool - Link to key stakeholders * Personnel appropriately compensated to support - Manage and use information productivity and motivation * Organizational partnerships clarified through written agreement * Adequate staffing plans in place to support strategies Universalia-IDRC Framework for Change (continued) Targets for organizational Explanations and key ideas to develop indicators Examples of indicators change * Joint ventures established linked to performance priorities * Programs and services appropriately linked to mission and goals Changes to organizational * Improved organizational culture * Existence and acceptance of the organizational mission motivation that affect *Improved internal organizational incentive systems * Functioning systems in place to reinforce the organization's * Improved ability to make transition to new organizational stage of development * Existence of an organizational history that support change * Improved mission and vision that is able to drive and innovation members' performance * Incentives systems congruent with organizational priorities * Key values and beliefs driving the organizations are linked to mission * Staff support organizational values and beliefs Understand and interface with * Improved ability to understand the opportunities and * National/local legislation adequate to pursue the environment within which the constraints posed by: organizational mission organization works - Legal/administrative system * Political resources allocation system support - Political system organizational priorities * Values of civil society congruent with organizational - Social cultural mission - Economic systems * Degree of market orientation appropriate to support - Incentives priority work of the organization - Market situation and conditions (competitors) * Labor market incentives provide appropriate pool of workers - Technological systems * Local infrastructure (road, electricity, etc.) reliable * Improved ability to affect the environment within which the organization is operating The implications for planning and evaluation when * Using this type of framework requires a compre- using such a framework are: hensive profile of the organization concerned. * The present and desired level of organizational * There are many potential indicators that can be performance must be identified as part of a plan- used to measure changes in capacity. This type of ning exercise. framework requires the involvement of the orga- nization in developing indicators that best illus- * Planning is based on a logical hypothesis, i.e., we trate the changes they want in capacity and per- can improve performance by building this system, formance. training these people, et cetera. 16 Appendix IV Planning and evaluating capacity development at the societal level Some GEF projects involve capacity development at and second, indicating the "threshold of infornation" the societal level-either through direct policy reform required for awareness. In other words, what is the or by engaging in building stakeholders' awareness of level of knowledge people (targeted stakeholders) the need for change. As is the case with individual and need in order to have a level of "awareness"? Since organizational change, the GEF and its partners need the level of threshold knowledge changes for each to agree on a basic flexible framework that can help in project, it is incumbent on the proposers to suggest the planning and evaluating societal change. level of threshold knowledge needed by each group and why. Some of the GEF projects reviewed indicate that changing the awareness of targeted groups is a critical Some GEF projects propose to build capacity to create factor in obtaining desired outcomes. At the most and enforce policies that support GEF work. Our re- immediate level, we believe any GEF project that view indicates that most of these projects stop at the proposes to increase awareness should meet two basic level of activity and outputs, and do not identify the conditions-first, clearly identifying the targeted results desired beyond the output. stakeholder whose awareness needs to be changed, Framework for Change The ability to identify a problem or a need in which a new policy or a change in policy is thought to support a solution to the problem The ability to create policy processes that generate interest in policy The ability to research the problem and identify alternative solutions to the problem The ability to choose amongst alternative solutions The ability to create a policy if required that would support the solution to the problem The ability to enforce policy 17 Appendix V Framework for conducting a functional analysis Once a country identifies a problem that it wants to Also, capacity development is often construed as only address, it is helpful to describe it in terms of broad training or equipment support to the entities desig- tasks involved to address the problem. If the problem nated to carry out project tasks. These inputs do not is in parks management, it is important to understand necessarily create the dynamic of change nor the dy- the functional responsibilities that are inherent in namic related to leaming that is central to capacity managing a park and to analyze how these functional development. roles are organized within a particular country. Are these functions managed within one organizational An important aspect of capacity development analysis unit or several units? involves understanding the functional requirements for solving a country's problem that are associated In part, capacity development is based on a careful with fulfilling conventional obligations. The existing assessment of the functional needs required to solve a structures should not be taken as given. The capacity development problem. Too often, capacity develop- development needs assessment should address a se- ment is designed as an add-on to projects that is sim- ries of important questions related to the functions that ply there to reinforce existing structures that may or need to be performed. may not be appropriate for the project's purposes. * What functions need to exist and work effectively in order for us to carry out our obligations? * Are these functions integrated into an organization or spread throughout many organizations? * Are the people and organizations responsible for fulfilling the function capable of doing so? * What do they need to fulfill their function? * Are the functions adequately organized and coordinated? * Are there critical bottlenecks and constraints hampering implementation of project actions and reach- ing its objectives? What needs to be done? 18 Appendix VI Monitoring capacity development The target of change is an important element of capac- not adequately link the target of change to the system ity development. As indicated in the paper, capacity change process that is occurring. The table below is a development focuses on at least three levels of tool used to link the target of change to the various change-individual, organizational and societal-and inputs, processes and results of the change effort. It is their interface. Traditional log frame analysis (LFA) meant to complement other tools such as the LFA. provide some insight into the change process, but do Target for capacity Input indicators Process indicators Output indicators Outcome indicators development System level Funds to support Collaborative Policies related to Wide national indicators policy research arrangements hunting of support for created to endangered species enforcement of encourage changes more adequately endangered species to environmental enforced act enforcement procedures Organizational level Equipment Organizational Organization's Ability to adjust indicators purchased to do leaders provide ability to manage biodiversity data as data entry and incentives that and report on required statistical analysis support use of biodiversity data equipment to meet improved informational needs Individual level Adequacy of training Technology training Staff trained in Skills used on the indicators encourages learning targeted technology job how to problem solve 19 Appendix VIl LFA and capacity development models Below is an example of a generic logical framework should be looked at in the context of the other tools for organizational and policy-oriented work. This offered to facilitate monitoring the change process. (capacity Development development) Expected Results Indicators Unit of Change Assumptions TARGET ORGANIZATIONS Enhanced capacity of government of .country" to meet Country determined Goal/Impact additional indicator-country Parks department System needs requirements created driveness by the convention on./.. Improved Increased ability to Outcomes performance of parks respond to Targeted Cag development conventions O Organization peormance information Insttution requirements New administrative systems in place Outputs Improved and used Parks people andChnecpit capacity of parks * Staff skills enhanced systems Change capacity management * New procedures in place Activities * Revise/upgrade Effective activites _ training programs Activities Right diagnosis * Review data management systems * Upgrade equipment l Resources | Training I Technical assistance Inputs 20 Appendix VilI How to make judgments in order to identify appropriate benchmarks One of the important issues involved in planning and the field required, and those with good sectoral experi- evaluating capacity development is the determination ence. In GEF, we find many experts and types of ex- of what is good capacity. What are good capacity perts who offer their opinion on a particular field. Ex- development (processes)? This is an area in which the pert judgment is used primarily when key stakehold- development community is just beginning to pay ers respect the "expert." In these situations, it is the some attention. expert who proclaims whether or not capacity is de- veloped-or if it is developed sufficiently. Experts ren- The ideal is to have benchmarks like those within the der opinions within a context as their basis of judg- ISO system. This way, we can link present levels of ment. Of course, the down side of expert judgment is capacity and capacity development to a norm of be- there are many people willing to judge in areas where havior. Unfortunately, such generalized standards do they are not expert. This is particularly true in a new not exist. Nevertheless, because you can interpret a set field such as capacity development. In using expert of data in many ways, it is important to understand judgment as a basis for decision making, it is impor- how judgments are made with respect to the present tant to know what the expertise of the "expert" is! state of the art of capacity development. From our experience, there are four main decision-making Criteria reference. Criteria reference is quite familiar methods used to make judgments about the interpreta- to the development community. We use it within the tion of data related to capacity development. LFA as pre-established criteria or objectively verifi- able indicators (OVI) upon which we judge projects Comparisons. Comparisons are normally linked to and programs. In criteria reference judgments, we be- baseline work. They involve a comparison of a present lieve a project is successful if it accomplishes what it set of conditions with past data. In general, you look says it is to accomplish-it meets or beats its indicators. for improvements over what has occurred in the past The difficulty with OVIs is that, a priori, you never and provide explanations for prescnt gains and future know whether the "achievement bar" is set too high or potential. Sometimes, using statistical formulations, too low for the context within which the project is op- evaluators try to see if differences between past and erating. present situations are significant. For example, we looked at the extent to which people learned material Market forces. Finally, as more and more products in training sessions using comparisons. Similarly we and services move from the public to the private do- looked at the use of knowledge on the job using pre main, market forces can provide insight into whether and post comparisons. products and services are of value. In the private sec- tor, the market's response will ultimately determine Expert opinion. Experts are those with good insights survival. In the public sector, we are just beginning to into the organization, those who are practitioners in understand the link between markets and public goods. 21 Appendix IX GEF cases and examples Not surprisingly, GEF, along with many other inter- tions about capacity development strategy prior to the national funding agencies, is still searching for a set implementation. of design principles or frameworks that can help staff and project participants to diagnose capacity con- The critical cost benchmarks and their interpretation straints and develop strategies. The pattern of results in the review and approval of the project proposals to date seems mixed. were the subject of debate, much of which high- lighted strongly held, dissimilar views on the pur- A large number of GEF projects are still focused on pose, appropriate content, and resources needed for the provision of a limited package of conventional biodiversity enabling activities. Differences were inputs that include training, equipment, and advice, mainly over the relative emphasis put on building the mainly at the organizational level. The most effective capacity for biodiversity planning versus developing projects in this category (e.g., African NGO-Govern- strategies and action plans for more modest stepping ment Partnership for Sustainable Biodiversity Ac- stones toward the implementation of field projects. tion) accurately diagnosed the nature and source of the broader system constraints and intervened at an A number of other projects did not seem able to appropriate point of leverage. match a capacity development solution to the prob- lem. Capacity development was often used as a term In some cases, training assistance and capacity devel- to describe an activity, but the link between the activ- opment are seen as interchangeable. The Biodiversity ity and the development problem was rarely articu- Data Management Capacitation in Developing lated. As noted earlier in this paper, the logical Countries and Networking Biodiversity Inforrnation framework was not particularly useful in providing projects were based on meeting an urgent need for this type of analysis. For the purposes of planning and capacity building, training and awareness creation in evaluating capacity development, GEF needs to have the area of biodiversity data management in develop- a better "theory-in-use." It must be clearer about the ing countries. In the words of the final evaluation nature and source of the current capacity gaps and report: constraints, about the pattern and configuration of a desired set of new abilities and, finally, how the "sys- [in] those countries where the national capacity tem" in question can move from one state to the other. in biodiversity data management is low, the Those projects that had less success (e.g., Institu- project shouldfocus primarily on capacity build- tional Support for the Protection of East African ing and skills development through more training Biodiversity) had difficulty devising a capacity devel- sessions. opment strategy that attracted participant support and ownership. Other projects reveal the underlying debate over ca- pacity development strategies. In the Interim Assess- Appendices I, II, and III provide frameworks used to ment of Biodiversity Enabling Activities: National design and assess capacity development projects Biodiversity Strategies and Action Plans8, the analy- through different units of analysis: the individual, the sis describes the three implementing agencies and the organizational, and the systemic levels. GEF Secretariat participating actively in consulta- 8 Interim Assessment of Biodiversity Enabling Activities: National Biodiversity Strategies and Action Plans, GEF, 1999, (Roles of the Implementing Agencies) para. 40-4 1. 22 All three implementing agencies are involved in vari- The enabling activities, primarily full-cost financing ous efforts to learn from their field experiences. In the provided for preparation of national reports required African NGO-Government Partnership for Sustain- by the Convention, may be viewed as another form of able Biodiversity Action project, 46 country programs capacity building. Grants of up to $350,000 are pro- from Iceland to Yemen have been implemented vided on an expedited basis, and larger countries have across Europe and the Middle East. Based on these received grants in excess of $1 million. A minority of experiences, a national model was shaped to take into countries have completed the first communication account all the lessons learned since the work began and are requesting a second round of support. (Note: a in 1985. review of GEF support for national communications has been completed and the report is final. It was The current program in Africa draws on these experi- submitted to the 6th Conference of the Parties to the ences, using and adapting field-tested survey tech- Framework Convention on Climate Change in No- niques, technical database management skills, and vember 2000.) Funds are provided primarily for com- established mechanisms to ensure data quality. In the pilation of emission inventories and identification of Southern Africa Biodiversity Support Program climate impacts, and often support the creation of project, sustained efforts were made to learn from national climate teams and units within governments. other similar GEF projects. Priority was given, for These reports and the agencies that prepare them then example, to assessing the results of the first GEFI become a continuing source of awareness within their UNDP African Regional Biodiversity project and the countries. Institutional Support Project for East African Biodiversity. The GEF East African biodiversity co- The Convention has several times given guidance to ordinator participated in one of the planning meetings the GEF to fund specific types of capacity building and a member of the independent mid-term evalua- activities. For example, COP 4 directed the GEF to tion team for this project also contributed to the de- provide financing for, inter alia, identification of pri- sign of the southern Africa project. ority technology needs and capacity building for ac- cess to observational networks. This trend toward The role of the GEF climate program was established such narrowly defined guidance tends to be resisted by the UN Framework Convention on Climate by the GEF because it is perceived as overly specific Change, for which the GEF serves as financial and an intrusion on operational matters; it is also mechanism. The GEF mandate is to contribute to the difficult to integrate this type of direction on an al- reduction of emissions of greenhouse gases in devel- most annual basis consistent with the administration oping countries. Funding is provided through two of a global, multifocal program. Partly in response, basic channels, greenhouse gas reduction projects the GEF developed the Capacity Development Initia- and enabling activities. The latter represent more than tive (CDI) in cooperation with UNDP in an attempt to 95 percent of funding allocations to date, most often provide a more holistic and coherent approach that in the form of financial support and technical assis- would respond to capacity needs across the range of tance for energy efficiency or renewable energy global environmental concerns (although there are projects. Almost all such projects include capacity locally based consultant experts for climate change building elements as a central feature, although typi- and biodiversity for each of four regions). While the cally the majority of funds are devoted to financing project will not be completed until the fall of 2001, a investments in equipment, resource assessment, likely result is the preparation of national capacity and other services. Diverse elements of capacity development needs assessments and potentially a development are employed including curriculum strategic framework for capacity development. The development and training programs; institutional result may for the first time be a GEF focus on capac- enhancement such as the creation of a renewable ity development as an end itself, rather than simply as energy unit in a utility company; technical assistance a project element. for small business planning; support for information centers and services; quality assurance programs on a The future of capacity building in the GEF climate voluntary or regulatory basis; and public awareness program may still be the subject of further delibera- campaigns. tions. At its last COP in the fall of 1999, the parties 23 began deliberations on a capacity building agenda amples of indicators that help targeted beneficiaries that may result in further mandates to the GEF and/or and donors better understand the growth or changes general obligations to provide financial assistance in individual, organizational and societal ability over and technology transfer to the developing countries. time. The approach taken in the Southern Africa The potential content of this agenda remains unclear Biodiversity Support Program seems to reflect the as the proposals so far introduced by Parties (as of standard GEF practice. It was agreed that progress June 2000) are wide ranging and without an overall reports are based on an established set of indicators of framework (submissions can be viewed at the effectiveness, efficiency, and impact developed by UNFCCC website, www.unfccc.de). the program implementation unit for approval by the technical committee within the first six months of the At present, GEF project documents do contain indica- program. The indicators themselves would be re- tors associated with capacity development, but these viewed periodically and updated on the basis of are generally indicators for inputs and outputs and newly acquired information. focus on completed activities. There are few ex- 24 Appendix X List of Assessment Tools Organizational Swanson, R.A., Analysis for Improving Performance: ToolsoforDiaggnosing Organizations and Document- Family Planning Management Development Project, ing Workplace Expertise, Berrett-Koehler, 1994 Management and Organizational Sustainability (MOST), Status Assessment Instrument, Manage- Taschereau, S., Evaluating the Impact of Training and ment Sciences for Health, 1996-1997 Institutional Development Programs: A Collabora- . . ~~~~~~~tive Approach, EDI Learning Resources Series, The Fort, A.L., Want Sustainability? Want Capacity ?A Frame- World Bank, 1998 work and Tool for Measuring Progress. Presenta- tion to the Global Health Council's Global Health, Weisbord, M., Organizational Diagnosis, A Workbook Poverty and Development Annual Conference, 1991 of Theory and Practice, Addison-Wesley, 1978 Fowler, A., Striking a Balance: A Guide to Enhancing Systems the Effectiveness of Non-Governmental Organiza- tions in International Development, Earthscan, 1997 Organization for Economic Cooperation and Develop- ' ~~ment, Criteria for donor agencies self-assessment GTZ. Organizational Development assessment tools in capacitv development, Document #DCD/DAC (GTZ53) (97)31, 1997 Harrison, M., Diagnosing Organizations: Methods, Mod- Stern, G., The Drucker Foundation Self-Assessment Tool: els, and Processes, Applied Social Research Meth- Process Guide, The Drucker Foundation, 1999 ods Series, Vol. 8, 1987 Trostle, J.A., Sommerfeld, J.U., Simon, J., "Strengthen- James, R., Demystifing Organizational Development: ing Human Resource Capacity in Developing Coun- Practical Capacity-Building Experiences ofAfrican tries: Who Are the Actors? What are TheirActions?" NGOs, INTRAC, 1998 in Grindle, M.E., (ed), Getting Good Government, Kirkpatrick, D.L., Evaluating Training Programs. The Capacity Building in the Public Sectors of Develop- Four Levels, Berrett-Koehler, 1998 ing Countries, Harvard University Press, 1997 United Nations Development Program, Capacitv Assess- Leveson H. Orga,zat9onal Diagnosis Harvard Uni- ment and Development in a Systems and Strategic versity Press, 1972 Management Context, Technical Advisory Paper #3, Lusthaus, C., Adrien, M-H., Anderson, G., Carden, F., January 1998 Enhancing Organizational Perfonnance: A Tool Box United Nations Development Program, Building Sus- for Self-Assessment, IRDC, 1999 tainable Capacity: Challenges for the Public Sec- Middleberg, M.I., Guide to Assessing Management Ca- tor. Office of Evaluation and Strategic Planning, pacity Among Non-Governmental Organizations 1996 (NGOs), CARE, 1993 USAID Africa Bureau, Office of Sustainable Devel- Oshry, B., Seeing Systems: Unlocking the Mvsteries of opment, Health and Family Planning Indicators: Organizational Life, Berrett-Koehler, 1995 Measuring Sustainability, Vol, 2. PASCA, Institutional Capabilities Assessment, 1996 25 26 Global Environment Facility 1818 H Street, NW Washington, DC 20433 USA tel: 1(202)473-0508 fax: 1(202)522-3240/3245 www.gefweb.org