GPR 57639 THE INTERNATIONAL LAND COALITION GLOBAL PROGRAM REVIEW Volume 2 Issue 4 THE WORLD BANK GROUP WORKING FOR A WORLD FREE OF POVERTY The World Bank Group consists of five institutions--the International Bank for Reconstruction and Development (IBRD), the International Finance Corporation (IFC), the International Development Association (IDA), the Multilateral Investment Guarantee Agency (MIGA), and the International Centre for the Settlement of Investment Disputes (ICSID). Its mission is to fight poverty for lasting results and to help people help themselves and their envi- ronment by providing resources, sharing knowledge, building capacity, and forging partnerships in the public and private sectors. THE INDEPENDENT EVALUATION GROUP IMPROVING DEVELOPMENT RESULTS THROUGH EXCELLENCE IN EVALUATION The Independent Evaluation Group (IEG) is an independent, three-part unit within the World Bank Group. IEG-World Bank is charged with evaluating the activities of the IBRD (The World Bank) and IDA, IEG-IFC focuses on assessment of IFC's work toward private sector development, and IEG-MIGA evaluates the contributions of MIGA guarantee projects and services. IEG reports directly to the Bank's Board of Directors through the Director-General, Evaluation. The goals of evaluation are to learn from experience, to provide an objective basis for assessing the results of the Bank Group's work, and to provide accountability in the achievement of its objectives. It also improves Bank Group work by identifying and disseminating the lessons learned from experience and by framing recommendations drawn from evaluation findings. The International Land Coalition June 25, 2008 Corporate and Global Evaluations and Methods http://www.globalevaluations.org ©2008 Independent Evaluation Group, The World Bank Group 1818 H Street NW Washington DC 20433 Telephone: 202-458-4497 Internet: http://www.globalevaluations.org E-mail: grpp@worldbank.org All rights reserved This volume is a product of the staff of the Independent Evaluation Group (IEG) of the World Bank Group. It is part of an ongoing series that reviews global and regional partnership programs in which the World Bank is engaged as one of the partners. The findings, interpretations, and conclusions expressed in this volume do not necessarily reflect the views of the Executive Directors of The World Bank or the governments they represent. IEG does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of IEG concerning the legal status of any territory or the endorsement or acceptance of such boundaries. Rights and Permissions The material in this publication is copyrighted. IEG encourages the dissemination of its work and permits these reviews to be copied or otherwise transmitted, with appropriate credit given to IEG as the authoring body. Cover photo: Workers harvesting rice in an irrigated field in Indonesia. Photo by Curt Carnemark, 1993, courtesy of the World Bank Photo Library. ISBN-13: 978-1-60244-108-8 ISBN-10: 1-60244-108-1 Printed on Recycled Paper Independent Evaluation Group Knowledge Programs and Evaluation Capacity Development (IEGKE) E-mail: grpp@worldbank.org Telephone: 202-458-4497 IEG Mission: Improving Development Results Through Excellence in Evaluation The Independent Evaluation Group (IEG) of the World Bank reviews global and regional partnership programs (GRPPs) in which the Bank is engaged as one partner among many for two main purposes: (a) to provide accountability in the achievement of the program's objectives by providing an independent opinion of the program's effectiveness, and (b) to identify and disseminate lessons learned from the experience of individual GRPPs. The preparation of a global or regional program review (GPR) is contingent on a recently completed evaluation of the program, typically commissioned by the governing body of the program. The first purpose includes validating the findings of the GRPP evaluation with respect to the effectiveness of the program, and assessing the Bank's performance as a partner in the program. The second purpose includes assessing the independence and quality of the GRPP evaluation itself and drawing implications for the Bank's continued involvement in the program. Assessing the quality of GRPP evaluations is an important aspect of GPRs, since encouraging more consistent evaluation methodology and practice across Bank-supported GRPPs is one of the reasons why IEG embarked on this new product in 2005. IEG annually reviews a number of GRPPs in which the Bank is a partner. In selecting programs for review, preference is given to those that are innovative, large, or complex; those that are relevant to upcoming sector studies; those for which the Executive Directors or Bank management have requested reviews; and those that are likely to generate important lessons. IEG also aims for a representative distribution of GPRs across sectors in each fiscal year. A GPR is a "review" and not a full-fledged "evaluation." It assesses the independence and quality of the relevant evaluation; provides a second opinion on the effectiveness of the program; assesses the performance of the Bank as a partner in the program; and draws lessons for the Bank's engagement in global and regional programs. The GPR does not formally rate the various attributes of the program. A GPR involves a desk review of key documents, consultations with key stakeholders, and a mission to the program management unit (secretariat) of the program if this is located outside of the World Bank or Washington, DC. Key stakeholders include the Bank's representative on the governing body of the program, the Bank's task team leader (if separate from the Bank's representative), the program chair, the head of the secretariat, other program partners (at the governance and implementing levels), and other Bank operational staff involved with the program. The writer of a GPR may also consult with the person(s) who conducted the evaluation of the GRPP. Each GPR is subject to internal IEG peer review, panel review, and management approval. Once cleared internally, the GPR is reviewed by the responsible Bank department and the secretariat of the program. Comments received are taken into account in finalizing the document, and the formal management response from the program is attached as an annex to the final report. After the document has been distributed to the Bank's Board of Executive Directors, it is disclosed to the public on IEG's external Web site. ii Abbreviations and Acronyms ARNow! Philippines Land Watch NGO CAPRi CGIAR Program on Collective Action and Property Rights CAS Country Assistance Strategy Paper (of the World Bank) CC Coalition Council (of the ILC) CEF Community Empowerment Facility CGIAR Consultative Group on International Agricultural Research CSO Civil Society Organization EC European Commission EE External Evaluation (of the ILC by Universalia) EU European Union FAO Food and Agriculture Organization FIG International Association of Surveyors GPP Global Public Program GPR Global Program Review ICARRD International Conference on Agrarian Reform and Rural Development (2003) IDRC International Development Research Center (Canada) IEG Independent Evaluation Group (of the World Bank) IFAD International Fund for Agricultural Development IFPRI International Food Policy Research Institute IGO Intergovernmental Organization ILC International Land Coalition LAND Land Alliances for National Development Program M&E Monitoring and Evaluation NGO Non-governmental Organization NSP Network Support Program PAKISAMA Pambansang Kilusan NG MGA Samahang Magsasaka (National Federation of Peasant Organizations) PRSP Poverty Reduction Strategy Paper SAFIRE Southern Alliance for Indigenous Resources SARRA South Asia Rural Reconstruction Association UNDP United Nation's Development Program WRAP Women's Results Access Program *** For ILC member acronyms see member table in Annex D. Fiscal Year of Program January 1 ­ December 31 Director-General, Independent Evaluation Mr. Vinod Thomas Director, Independent Evaluation Group ­ World Bank Ms. Cheryl Gray Manager, Corporate and Global Evaluations and Methods Unit Mr. Mark Sundberg Global Programs Coordinator Mr. Chris Gerrard Task Manager Ms. Nalini Kumar Consultant Mr. Ridley Nelson iii Contents ABBREVIATIONS AND ACRONYMS................................................................................................... II PROGRAM AT A GLANCE: INTERNATIONAL LAND COALITION (ILC) ........................................VII KEY BANK STAFF RESPONSIBLE DURING PERIOD UNDER REVIEW .......................................VIII GLOSSARY...........................................................................................................................................IX PREFACE............................................................................................................................................XIII SUMMARY........................................................................................................................................... XV 1. PROGRAM OBJECTIVES, ACTIVITIES, FINANCIAL RESOURCES, AND GOVERNANCE 1 Objectives and Activities ........................................................................................................ 1 Financial Resources................................................................................................................ 2 Governance Structure............................................................................................................. 3 2. THE EXTERNAL EVALUATION OF THE ILC ......................................................................... 6 Scope, Methodology, and Approach ..................................................................................... 6 Independence and Quality...................................................................................................... 7 Impact of the Evaluation ......................................................................................................... 9 3. THE EFFECTIVENESS OF THE ILC...................................................................................... 10 Relevance ............................................................................................................................... 11 International Consensus ..................................................................................................................11 Alignment with Beneficiary Needs and Priorities .............................................................................12 Subsidiarity ......................................................................................................................................14 Comparative Advantage ..................................................................................................................14 Relevance of Program Design.........................................................................................................15 Efficacy ................................................................................................................................... 16 Achievement of Program Objectives ...............................................................................................16 Performance of Individual Program Components ............................................................................21 Efficiency................................................................................................................................ 25 Analytical Evidence .........................................................................................................................25 Efficiency In Relation to a Business Plan ........................................................................................26 Incremental Costs or Benefits to Stakeholders................................................................................26 Obvious Inefficiencies......................................................................................................................27 Sustainability of the Program and Prospects for the Future ............................................ 27 iv 4. GOVERNANCE AND MANAGEMENT .................................................................................. 28 Adherence to Public Sector Governance Standards ........................................................ 28 Relationship with the Host Organization, IFAD ................................................................. 29 ILC with a Similar Relationship to IFAD as in the Past But with Greater Definition of Roles............30 ILC Floated Off as a Legally Independent International NGO..........................................................31 ILC's Future Role .................................................................................................................. 31 5. THE WORLD BANK AND THE ILC PARTNERSHIP ............................................................ 33 6. LESSONS ............................................................................................................................... 36 REFERENCES..................................................................................................................................... 39 ANNEX A. EVALUATION FRAMEWORK FOR GLOBAL PROGRAM REVIEWS............................ 41 ANNEX B. THE EXTERNAL EVALUATION FINDINGS .................................................................... 49 ANNEX C. OUTCOME EVIDENCE TABULATION ............................................................................ 53 ANNEX D. ILC MEMBERS, COUNCIL, AND SECRETARIAT........................................................... 55 ANNEX E. ILC FINANCES .................................................................................................................. 58 ANNEX F. PERSONS CONSULTED .................................................................................................. 60 ANNEX G. RESPONSE OF THE ILC SECRETARIAT TO IEG'S GLOBAL PROGRAM REVIEW... 61 Boxes Box 1. International Land Coalition: Timeline................................................................................... 1 Box 2. International Land Coalition, 2007­2011 Strategic Framework........................................... 3 Box 3. ILC Structure............................................................................................................................. 5 Box 4. Constituency Types ................................................................................................................. 6 Box 5. Key Comments of the Bank's Land Thematic Group on the Preliminary Draft of the External Evaluation................................................................................................................. 9 Box 6. The Land Focus in the World Bank's Land Policies for Growth and Poverty Reduction Paper.................................................................................................................... 13 Box 7. Imputed Logical Chain Corresponding to ILC's Two Main Objectives -- Capacity Building and Facilitation -- and the Predominant Place of Each Component Program ................................................................................................................................. 17 v Tables Table 1. Donations and Pledges to International Land Coalition since Inception, 1996­2007 .... 4 Table 2. ILC Expenditures by Sources of Funds, 2005­2008 (US$ thousands)............................. 5 Annex Table 1. Assessing the Independence and Quality of the Evaluation .............................. 41 Annex Table 2. Providing an Independent Opinion on the Effectiveness of the Program ......... 42 Annex Table 3. Assessing the Bank's Performance as a Partner in the Program ...................... 47 Annex Table 4. Common GRPP Activities ....................................................................................... 48 Annex Table 5. Major Findings of the External Evaluation ............................................................ 53 Annex Table 6. ILC Members, at the Time of the External Evaluation .......................................... 55 Annex Table 7. ILC Coalition Council............................................................................................... 56 Annex Table 8. ILC Secretariat, 2008................................................................................................ 57 Annex Table 9. ILC Statement of Revenues and Expenses ........................................................... 58 Annex Table 10. ILC Balance Sheet at Nominal Value.................................................................... 59 Figures Figure 1. International Land Coalition, Operating Expenses, 2005­2006..................................... 26 The assistance of Marie Charles in mission planning and formatting is acknowledged. Also the technical assistance of Julius Gwyer and Maria Mar with the civil society consultation process is acknowledged. vii Program at a Glance: International Land Coalition Start date The precursor organization, the Popular Coalition to Eradicate Hunger and Poverty, was started in January 1996. The ILC itself was formally constituted and launched along with its new name in February 2003. Mission The ILC is a global alliance of intergovernmental, governmental and civil society organizations. Its mission is to work together with poor rural men and women to increase their secure access to natural resources, especially land, and to enable them to participate at local, national, regional and international levels in the policy and decision-making processes that affect their livelihoods (Strategic Framework, 2004­2006). Objectives Over most of the period under review the language of the objectives has been: (1) To enhance the capacities of its members and partners to help the rural poor, women and men, to gain and maintain secure access to land and related production support services. (2) To facilitate the opening up of space for dialogue with decision makers. While there were some changes in the language in late 2007, the intent has remained largely the same. Activities The ILC has followed six strategic trusts: (a) A Knowledge Program (KP) (b) A Network Support Program (NSP) (c) A Community Empowerment Facility (CEF) (d) A Women's Resource Access Program (WRAP) (e) A Land Alliances for National Development Program (LAND) (f) A program entitled Common Platform on Access to Land (PLATFORM). Supporting these six strategic trusts have been four sets of supporting activities: advocacy activities; policy analysis activities; annual and council meetings activities; and communication, outreach, and management activities (i.e., the Secretariat's activities). WBG contributions The World Bank made one-time contribution of US$1.5 million from the Development Grant Facility in 1998, representing about 8 percent of total donor contributions through 2007. Initially intended to contribute to an endowment fund (which did not materialize due to lack of sufficient contributions), this contribution has financed capacity-building grants to CSOs from the Community Empowerment Facility. Other donor From its founding in 1996 through 2007, the ILC has received financial contributions contributions of $18.6 million from 13 multilateral, bilateral and other donors (including the World Bank). The host organization, IFAD, has contributed 48 percent of total funding. Location Housed at the International Fund for Agricultural Development (IFAD), Rome, which has therefore been the legal entity for the ILC. Governance and The Assembly of Members is the supreme body which meets biennially and management establishes overall policy and strategy. The Coalition Council of 14 members is an executive board which is responsible for overall governance. The Secretariat is responsible for day-to-day management and administration. Most staff are based in Rome; two are based in New York. Latest program- Universalia, International Land Coalition: External Evaluation, 2006. level evaluation viii Key Bank Staff Responsible during Period under Review Position Person Period Global Program Task Team Leader John Bruce 1996­2005 Erick C. M. Fernandes 2005­2007 Klaus Deininger 2007­2008 Director, Agriculture and Rural Alex McCalla 1996­1999 Development Department Robert Thompson 2000­2002 Kevin Cleaver 2002­2007 Mark Cackler (Acting) 2007­2008 Juergen Voegele February 2008 to present Vice President, Sustainable Ismail Seregeldin 1996­1997 Development Network Ian Johnson 1997­2006 Katherine Sierra 2006 to present Trust Fund Operations Not applicable Not applicable Director, Global Programs & Margaret Thalwitz May 2004 to present Partnerships Group Program Manager Position Person Period Director, for both the Popular Bruce Moore 1996 to present Coalition to Eradicate Hunger and Poverty and for the ILC ix Glossary Affiliates For the ILC, organizations having periodic involvement with the Coalition, but who are not members, partners, or donors. Customary tenure The tenure usually associated with indigenous communities and administered in accordance with their customs (as opposed to statutory tenure). Devolution or exit A proactive strategy to change the design of a program, to devolve some of strategy its implementation responsibilities, to reduce dependency on external funding, or to phase out the program on the grounds that it has achieved its objectives or that its current design is no longer the best way to sustain the results which the program has achieved. Donors For the ILC, organizations or entities that have provided financial or in-kind resources to the program, most of which are reflected in ILC's audited financial statements. Efficacy The extent to which the program has achieved, or is expected to achieve, its objectives, taking into account their relative importance. The term is also used as a broader, aggregate measure -- encompassing relevance and efficiency as well -- of the overall outcome of a development intervention such as a GRPP. Efficiency The extent to which the program has converted or is expected to convert its resources/inputs (such as funds, expertise, time, etc.) economically into results in order to achieve the maximum possible outputs, outcomes, and impacts with the minimum possible inputs. Evaluation The systematic and objective assessment of an ongoing to completed policy, program, or project, its design, implementation, and results. The aim is to determine the relevance and achievement of its objectives, and its developmental effectiveness, efficiency, impact, and sustainability. Governance The structures, functions, processes, and organizational traditions that have been put in place within the context of a program's authorizing environment to ensure that the program is run in such a way that it achieves its objectives in an effective and transparent manner. It is the framework of accountability and responsibility to users, stakeholders and the wider community, within which organizations take decisions, and lead and control their functions, to achieve their objectives. Impacts Positive and negative, primary and secondary long-term effects produced by a development intervention, directly or indirectly, intended or unintended. Independent An evaluation that is carried out by entities and persons free from the control evaluation of those involved in policy making, management, or implementation of program activities. This entails organizational and behavioral independence, protection from interference, and avoidance of conflicts of interest. Indicator A quantitative or qualitative factor or variable that provides a simple and reliable means to measure achievement, to reflect the changes connected to an intervention, or to help assess the performance of a development actor. Land administration The set of systems and processes for making land tenure rules operational. Land reform The redistribution or restitution of land, often to the rural poor, for equity, efficiency, or other purposes. Land tenure The relationship, whether legally or customarily defined, among people, as individuals or groups, with respect to land and associated resources. x Legitimacy As a criterion for assessing governance and management, the way in which governmental and managerial authority is exercised in relation to those with a legitimate interest in the program -- including shareholders, other stakeholders, implementers, beneficiaries, and the community at large. Logical framework or A management technique that is used to develop the overall design of a logframe program or project, to improve implementation monitoring, and to strengthen evaluation, by presenting the essential elements of the program or project clearly and succinctly throughout its cycle. It is a "cause and effect" model which aims to establish clear objectives and strategies based on a results chain, to build commitment and ownership among the stakeholders during the preparation of the program or project, and to relate the program's or project's interventions to their intended outcomes and impacts for beneficiaries. Management The day-to-day operation of a program within the context of the strategies, policies, processes, and procedures that have been established by the governing body. Members For the ILC, organizations with voting and decision-making power in the Assembly of Members. Monitoring The continuous assessment of progress achieved during program implementation in order to track compliance with a plan, to identify reasons for noncompliance, and to take necessary actions to improve performance. Monitoring is usually the responsibility of program management and operational staff. Outcomes The achieved or likely short-term and medium-term effects of the outputs of a development intervention. Oversight One of the core functions of the governing body of a program: Monitoring the performance of the program management unit, appointing key personnel, approving annual budgets and business plans, and overseeing major capital expenditures. Partners In most IEG Global Program Reviews, partners are understood as stakeholders who are involved in the governance or financing of the program (including the members of the governing, executive, or advisory bodies). In the case of ILC, the term "partners" is used differently to include organizations that have participated in the Coalition, but who do not have voting authority. Public goods Goods which produce benefits that are non-rival (many people can consume, use, or enjoy the good at the same time) and non-excludable (it is difficult to prevent people who do not pay for the good from consuming it). If the benefits of a particular public good accrue across all or many countries, then the good is deemed a global or international public good. Relevance The extent to which the objectives and design of the program are consistent with (a) the current global/regional challenges and concerns in a particular development sector and (b) the needs and priorities of beneficiary countries and groups. Shareholders The subset of donors that are involved in the governance of the program. Therefore, this does not include individual (particularly anonymous) donors who choose not to be so involved, or who are not entitled to be involved if their contribution does not meet the minimum requirement, say, for membership on the governing body. xi Stakeholders The parties who are interested in or affected, either positively or negatively, by the program. Stakeholders are often referred to as "principal" and "other", or "direct" and "indirect". While other or indirect stakeholders -- such as taxpayers in both donor and beneficiary countries, visitors to a beneficiary country, and other indirect beneficiaries -- may have interests as well, these are not ordinarily considered in evaluations unless a principal stakeholder acts as their proxy. Sustainability When the term is applied to the activities of a program, the extent to which the benefits arising from these activities are likely to continue after the activities have been completed. When the term is applied to organizations or programs themselves, the extent to which the organization or program is likely to continue its operational activities over time. Transparency As a criterion for assessing governance and management, the extent to which a program's decision-making, reporting, and evaluation processes are open and freely available to the general public. This is a metaphorical extension of the meaning used in physical sciences -- a "transparent" objective being one that can be seen through. Sources: Sourcebook for Evaluating Global and Regional Partnership Programs: Indicative Principles and Standards. Independent Evaluation Group ­ World Bank, 2007, for evaluation terms. xiii Preface The organization that is now called the International Land Coalition (ILC) was founded in 1996 following the Conference on Hunger and Poverty in 1995 in Brussels. Its precursor, which arose from the deliberations of that conference, was called the Popular Coalition to Eradicate Hunger and Poverty. The ILC itself was formally constituted and launched along with its new name in February 2003. The International Fund for Agricultural Development (IFAD) agreed to host the precursor Popular Coalition in 1996 -- and continues to host the ILC -- as a separate entity within, but not as a department of IFAD. Similar to other global and regional programs that are housed in international organizations (including the World Bank), IFAD has been the formal legal entity for the ILC. This has, however, been a matter of some debate and is currently being clarified as part of the on-going redirection of the ILC. This Global Program Review (GPR) by the Independent Evaluation Group (IEG) of the World Bank assesses the independence and quality of the external evaluation (EE) of the ILC that was conducted by Universalia in 1996, provides a second opinion on the effectiveness of ILC's work, assesses the performance of the Bank as a partner of the ILC; and draws lessons for the future of the ILC and the Bank's involvement with it. During the preparation of this GPR, there have been intensive discussions among ILC members about its mandate and operations and a number of important decisions may be taken before this report is finalized. If some findings of this report have been overtaken by recent decisions, IEG views that the issues raised in this GPR may still offer food for further deliberation at future Assembly or Council meetings of the Coalition as the changes are implemented. The present review covers mainly the period from the establishment of the ILC in 2003 to the present, including key developments since the external evaluation was completed, but it also touches on aspects of the evolution of the earlier Popular Coalition entity that arose from the original 1995 conference. The ILC was chosen for a GPR for three reasons -- because land is an important topic in its own right; to contribute to a broader IEG study currently underway on the World Bank's contribution to land reform, administration and policy issues in client countries; and to provide lessons for the design and operation of global programs more generally. IEG also anticipated lessons from the review for poverty-focused rural development and for international support of land policy reform in particular. The Review follows IEG's Guidelines for Global Program Reviews (Annex A). It is based on a desk review of relevant documents followed by a visit to the ILC and IFAD in Rome. In addition to the Universalia evaluation, it draws on ILC annual reports, consultant studies, journal articles, Web sites, and reports on conferences and discussions with ILC, IFAD, FAO and World Bank staff (hereafter referred to as IGO staff). These discussions followed flexible but semi-structured questions adapted mainly from the evaluation framework questions in Annex A. Selection was purposive, dictated partly by availability, but based on IEG's knowledge and investigation concerning which individuals had had the most involvement with either ILC or the land issue more broadly, i.e. the "land community" as one respondent described it. IEG has also undertaken a small electronic consultation process with civil xiv society organizations involved in land policy, reform and administration as a part of the above mentioned Land Study. The intent of the consultation for the ILC review was mainly to validate the more substantial survey by the EE. The population consulted consisted of names that had been obtained from a range of sources, including the ILC membership list, and at the second stage, those who then registered to participate. The questions were available on the consultation site in English, French and Spanish. IEG gratefully acknowledges all those who made time for interviews or provided responses to the consultation, and in particular the ILC Director, staff and member representatives. A list of people consulted can be found in Annex F. Copies of the draft GPR were sent to ILC management, to the Bank unit which is responsible for the Bank's involvement with the ILC (the Agriculture and Rural Development Department), to the Bank's Land Policy Thematic Group, and to other Bank units that have responsibility for the Bank's involvement with global and regional programs more generally. Their observations have been taken into account in finalizing this GPR. The formal response received from ILC management has been attached as Annex G. xv Summary Mission, Financial Resources, and Governance 1. The organization, which is now called the International Land Coalition (ILC), was established on January 1, 1996, on the recommendation of the Conference on Hunger and Poverty convened by the International Fund for Agricultural Development (IFAD) in 1995. At first, the organization was called the Popular Coalition to Eradicate Hunger and Poverty. The ILC itself was formally constituted and launched along with its new name in February 2003. The mission of the Land Coalition has been to be a global alliance of intergovernmental, governmental and civil society organizations that worked together with rural poor people to increase their secure access to natural resources, especially land, and to enable them to participate directly in policy formulation and decision-making processes that affected their livelihood, at local, national, regional and international levels. 2. From its inception through the end of 2007, the ILC has mobilized $18.6 million in donations and pledges receivable from donors. The major contributors have been IFAD (which has provided 48 percent of all resources), the Netherlands, the European Commission, Netherlands, the World Bank, IDRC (Canada), Italy, and Belgium. The World Bank was a founding member and provided a one-time financial contribution of US$1.5 million from its Development Grant Facility in 1998. 3. The ILC is governed by an Assembly of Members which meets biennially and an executive body, the Coalition Council, which is composed of 8 civil society members and 6 intergovernmental members. The ILC Secretariat -- headed by a Director and located within IFAD in Rome -- is accountable to the Coalition Council and Assembly of Members. However, IFAD, as the legal entity for the ILC, has also provided some oversight of the Secretariat. For what purposes the Secretariat reports to the Council and for what purposes to IFAD has not always been clear. The External Evaluation of the ILC 4. The Executive Board of IFAD commissioned an external evaluation (EE) of the ILC in 2006. ILC's Assembly, Council and Secretariat fully cooperated with this evaluation. The evaluation team appears to have been largely free to operate independently of the ILC, although a few Intergovernmental Organization (IGO) observers interviewed felt that the ILC Secretariat had too much influence through comments on a survey questionnaire and on the preliminary draft report. The evaluation had both strengths and weaknesses which are discussed in this Global Program Review. The evaluation applied the appropriate criteria of relevance, effectiveness, efficiency, sustainability and governance, and examined in depth the IFAD­ILC hosting relationship, perhaps at some cost to other issues. The final report was issued and publicly disclosed in August 2006. 5. The EE found that the ILC had engaged in many successful activities and produced significant outputs, but was having some difficulty living up to expectations. It noted that the Coalition was still a relatively small entity, in the early stages of its growth, and that there was a xvi significant gap between what it said it wanted to do and what it does or even can do. The evaluation concluded that the ILC was now at a critical point in its evolution. While ILC's mission remained relevant and the majority of its members expressed satisfaction with its progress (although the member CSOs surveyed had some motive to report favorably since they were actual or potential recipients of ILC funds), the evaluation raised a number of issues concerning ILC's long-term viability, effectiveness and efficiency. These issues required immediate attention and called for important stakeholder deliberations and decisions about ILC's future. In particular, the EE found a lack of clarity in ILC's legal status and its relationship to IFAD. 6. Both ILC and IFAD have taken the evaluation seriously. ILC has accepted most of the findings and has drawn on the report to initiate a change process. ILC's response has so far been proactive and promising. It has produced a number of new documents either exploring or charting future directions, and in particular, a paper outlining the options aimed at clarifying ILC's legal status. Membership has doubled since the EE. The Coalition has set milestones for the change process which are reported to have been met ahead of schedule. It has held a number of recent meetings with IFAD on the ILC­IFAD relationship in order to draw up a Memorandum of Understanding to govern their relationship going forward. Donors have recently provided substantial new funding of more than US$7 million since 2006. The Effectiveness of the International Land Coalition RELEVANCE 7. Over most of the period under review the ILC objectives were stated as follows: (1) To enhance the capacities of its members and partners to help the rural poor, women and men, to gain and maintain secure access to land and related production support services. (2) To facilitate the opening up of space for dialogue with decision makers. (There have been some adjustments to these statements recently.) 8. The issue of land rights itself is clearly highly relevant. As an element in poverty alleviation it has moved higher on the international agenda in recent years, driven by some resurgence of attention by donors to the rural sector as well as recently emerging land crises in some countries. It is a significant element in the latest World Bank rural strategy and features in a number of Bank Country Assistance Strategies in some form. The Bank's involvement in the founding of the ILC was consistent with its Rural Strategy at the time and remains so. The potential relevance of an ILC type of global alliance is substantial. The actual relevance so far has been quite modest. 9. Despite its ambitious objectives, the design and implementation of ILC support for land reform and administration has yielded only patchy and opportunistic achievements. However, in assessing such a network, it needs to be acknowledged that making progress on an issue such as land, with its enormous political hurdles, is particularly challenging and will always be a long- term endeavor. Nevertheless, relevance questions arise in a number of areas. First, with respect to legitimacy, a recent global conference, the International Conference on Agrarian Reform and Rural Development (ICARRD) in 2003, with similar aims to the ILC founding conference many years earlier in 1995, did not seize on the ILC as an indispensable and available contributor to the xvii proposed program. While there were widely acknowledged (by observers) political and turf reasons for this, it still leaves some questions about global perceptions of ILC relevance. Second, the confusion for some years after the founding conference about whether or not governments could be members suggests that there would be at least some range of founder viewpoints on legitimacy of the membership structure. Third, the comparative advantage of the ILC is at best mixed. While there is no global network that can claim the full range of activities that the ILC claims, nor the same degree of neutrality in convening power, many do equally well or better in such areas as knowledge exchange and broad-based community capacity building, and some organizations, especially the larger ones, have quite competitive, although to some audiences less neutral, convening power. Fourth, the ILC Secretariat put a greater focus initially on facilitation skills among its staff, and has been short of land policy skills until very recently. Many IGO staff question ILC's relevance on these skills grounds. Fifth, with respect to the subsidiarity principle, the ILC has had a dominant central Secretariat for too long. While this was necessary in the early years for the ILC would not have survived without it, and while members evidently went along with this arrangement, it probably resulted in insufficient regional responsibility. In terms of network evolution, the change is overdue, but it will be a critical test of regional commitment and therefore relevance. EFFICACY 10. ILC's efficacy is extremely difficult to assess due to lack of output and outcome data, including lack of analysis and aggregation of at least some available outcome data from the small-grant Community Empowerment Facility (CEF). Evidence presented in the EE and here is largely qualitative and anecdotal. The rating of a Bank-funded project or program with such limited data would be downgraded on evidence grounds alone. 11. With respect to the first objective of enhancing capacities, there appears to have been some modest progress. But in some cases under both the CEF and the Knowledge Program, there was lack of monitoring attention to deliverable outcomes, and the support provided was not always sufficiently focused on achieving significant gains with respect to land policy or administration. However, the ILC has also contributed to knowledge in global and regional fora by bringing together members and others with land skills to share experiences. But as a knowledge source on land issues, the ILC is not unique. 12. With respect to the second objective of facilitation of dialogue, there has been some scattered success. The ILC's impact appears to be evident, for example in Indonesia; in Nicaragua where ILC brought together NGOs resulting in the formation of a new network; in Guatemala, where ILC had been partly instrumental in facilitating the government and multi- stakeholder dialogue; in Zambia, where there was success in delaying for further consideration a hasty land reform bill with a number of weaknesses. But even in some of these cases, some observers dispute the share of impact that can really be attributed to the ILC as opposed to other local players present both before and after the ILC's contribution. The lack of sustained presence in a theme such as land policy and administration that requires exceptional persistence has been an almost inevitable problem for a small coalition with modest funds and limited skills. 13. More generally, the present review found that some of the documents reporting the outcomes of ILC meetings were disappointing in their lack of focus on action. These were largely restatements of the broad ILC vision, contributing little towards any common agreement about xviii global or regional priorities or to signaling who would do what by when. They suggested insufficient discussion on how to make progress on land policy or land administration issues in priority countries or on where windows of opportunity might be opening up. No doubt some such discussion does take place in informal conference interactions, but it does not seem to find its way into the proceedings. EFFICIENCY 14. Efficiency is even more difficult to assess than efficacy, again due to lack of data. The high share of administrative costs in total costs found by the EE should not be given much significance since most of the Secretariat staff time was not assigned to particular programs or activities with attributable outputs. This accounting problem is now being addressed. However, there are a few indicative findings mostly related to the efficacy findings. As with a lot of small- grant programs, administrative costs are likely to have been relatively high for the modest amounts of money disbursed. Even accepting the challenge of high supervision costs relative to small grant sizes, efficiency could still have been enhanced if these had all been more focused on the highest priority land objectives nested within a broader country strategy. Efficiency for ILC itself has been enhanced by having the host IFAD carry some of the administration and accounting burden, but, of course, this represents an inefficiency for the host organization albeit with some overall returns to administrative scale. Another aspect of efficiency, not directly covered by the EE, is the costs to members and partners. With annual and regional meetings this is likely to have been substantial and significant in comparison with the ILC budget. 15. The mere fact of the length of time taken since the 1996 start date of the precursor coalition to reach the present rather modest level of program maturity suggests some likely efficiency weakness with early costs not yielding sufficient benefits soon enough. But there is insufficient data to attempt any economic analysis along these lines. SUSTAINABILITY AND FUTURE PROSPECTS 16. One aim in the most recent statements of objectives is for the ILC to reach financial sustainability. This may be too optimistic and, in any case, needs to be thought through against the global public good purpose. Clearly contributions by members would demonstrate commitment and allow the ILC to spread its donor resources further. However, the logic of a global public program is that there is an element of global public good externalities that cannot be recovered from beneficiaries since aggregated private benefits are less than total global benefits. With CSOs, the boundary between private and public may be blurred since some members may well be prepared to contribute to a global public good or at least a regional one. But, as it stands, the financial sustainability goal may need some finessing and realism. 17. With the large inflow of new donor funds, the ILC has recently moved closer to financial sustainability as well as less risk through greater diversity in donor contributions. Whether it has moved closer to institutional sustainability remains to be seen. Up to now CSOs have been more receivers than funders, and donor funds have been modest. Meanwhile, the large share of IFAD's contributions was increasingly being questioned within IFAD. Moreover there has been a general lack of enthusiasm for ILC among IGO staff with influence on potential donor funding. ILC's proactive reaction to the EE and the reported meeting of recent milestones in the renewal process may change perceptions. Much organizational change is under way. Whether this will re-ignite xix enthusiasm across all stakeholders remains to be seen but the directions seem promising. At this point sustainability remains uncertain. M&E will now need to deliver output and outcome evidence quickly to sustain further donor commitment. Governance and Management 18. Most CSOs surveyed by the EE were satisfied with the governance arrangements and practice. As with other aspects, IGO staff were less satisfied. There are a number of governance issues and some of these are currently matters of discussion within ILC and IFAD. Three elements are of particular importance. 19. First, the ILC has been strongly guided from the center, that is the Secretariat, and members have tended to go along with this. It is now planned, based on the EE recommendations, to decentralize and strengthen regional alliances and place more responsibility on regional members. This makes sense at this point and should probably have been done several years ago. However, it will be a significant test of commitment. From a situation with no fees beyond indirect or in-kind contributions by some members towards such things as attending and organizing meetings, the new strategy will now impose more direct costs. Governance should benefit from a system with greater regional responsibility relative to the substantial management role of the Secretariat as in the past, but there are considerable uncertainties about the level of response and where this will lead. 20. Second, it is widely agreed by all stakeholders that the quite narrow ILC membership needs to be broadened and this is already underway, membership having recently doubled. But this, along with greater regionalization, will have implications for how ILC will operate, particularly with respect to advocacy since wider membership will mean a wider range of views about land policy and solutions. In other words, there will be more ideas but less consensus. The present review has also identified some issues about the role of advocacy on the one hand and the exclusion of governments as members on the other, which also raises some questions about global relevance since governments are the implementers of land reform and administration policies. 21. Third, the purposes for which the Secretariat reports to IFAD and to the Coalition Council need to be clarified, as well as a mechanism to resolve conflicts between the two if these should arise. It is difficult to see the ILC surviving without being attached to an IGO or some large stable organization. Some variant on the existing arrangement with less burden for the host and greater legal clarity seems sensible, would be consistent with many other global alliances, would help the recruitment of quality staff and interaction with a strong professional land community, and would keep overhead costs down through economies of scale. There is evidence that global alliances that have been "spun off" to be totally independent organizations, or that have rotated among host organizations, have faced more operating and staffing problems, without gaining much in compensation. The costs have generally exceeded the benefits. However, the present review argues that these are largely matters of facilitating operation. The substantive legitimacy provided by the association with an IGO is a false legitimacy. Legitimacy can only come from the quality and proactivity of its membership, from demonstrating positive results, and from the Coalition's reputation for collective knowledge, facilitation and capacity building. xx The World Bank and the ILC Partnership 22. The Bank's roles in the ILC partnership have been as co-founder, a significant one-time financier of $1.5 million in 1998, and a regular member of the Coalition Council. Bank commitments represented 8 percent of funds from 1996 to the end of 2007 and 13 percent of funding up to 2006, prior to the recent substantial commitments from other donors in 2007. The Bank has been in a difficult position in this alliance. As the major donor globally on land policy and administration, the Bank has considerable power. However, Bank staff have had to walk a fine line in a global alliance aimed at capacity building and facilitation with an advocacy element which did not include the Bank's main government clients and which included some members who view the Bank and governments as part of the problem. The present review concludes that Bank's task team leaders could have been more proactive in some areas and would have benefited from a strategically focused terms of reference giving them appropriate direction and continuity of purpose. 23. More specifically, on the positive side, there is evidence from other observers that Bank staff have played an important role in pushing for sound analysis as a basis for Coalition knowledge. They have contributed substantial input through the Bank Land Thematic Group and Bank papers on land issues. On the negative side, it is not clear, first, that the Bank pushed enough over the years to improve the ILC's monitoring and evaluation (M&E) system. Second, the original Bank funding of 1998 is still not closed. The conversion from an endowment fund to standard grant disbursement, and the subsequent hiatus in the use of these funds (due to IFAD funds being used first, resulting in the disbursement of Bank funds over a decade) warranted more proactive fiduciary management. Given the Bank's policy changes on land issues over the years, it would have been better for it to have been closed and taken back to the Development Grant Facility for reconsideration and refinancing. Third, and more debatable, some observers argue that the Bank should have linked ILC work better to its own project interventions at the country level. While there may be some truth in this in some countries, Bank staff have not seen much that the ILC could have offered to the Bank's country operations partly due to the lack of Secretariat skills and limited progress on land policy issues in many countries. 24. There are some potential reputational risks to the Bank if ILC membership grows along with regionalization. There is some risk that either the Secretariat or members, in talking to governments, might use the membership of the Bank to confer legitimacy on positions the Bank itself would not take. This is an inherent risk with all such networks. The present review did not have the resources to check out, at the country level, the few cases of concern mentioned in interviews and these may not have been of great significance. But this is something that the Bank's TTL will need to monitor as the strong central Secretariat control of regional and country- level dialogues is reduced as a result of regionalization. Lessons 25. There remains a case for donors to sustain the ILC for a further period. The land issue is growing rapidly in importance, in some countries in troubling directions. Global sharing of knowledge is therefore potentially of high value. Economic returns to land interventions, based on limited analysis in a few countries, seem generally to be quite high. However, if it is to be sustained, ILC's objectives need to be realistic, readily understood, focused, measurable, and xxi actually measured. Skills need to be aimed at the particular niche of true comparative advantage in land reform and administration. The proposed structure of a widened regionalized network needs to maximize the input of sound analysis. If the ILC of 2012, after five years of experience with the new 2007­2011 Strategic Framework, does not largely achieve at least these elements with reasonable levels of efficiency then it would be time for members, founders and donors to consider terminating the initiative. 26. The main lessons of this Global Program Review for consideration by ILC members and the Secretariat are the following: · ILC membership could usefully revisit the nexus of advocacy and membership. Advocacy as an activity can be advocacy of a vision, to which all members implicitly signed on, or advocacy of solutions, to which no members signed on. Advocacy aimed at solutions cannot be done collectively by a coalition to an outsider without a common position. Agreement on a common position will always be unattainable, especially with broadening membership. Advocacy by individual members within the group to others within the group, which would raise the demand for evidence, seems more likely to lead to advances in understanding. If members were to choose to take this route, those to be advocated to (i.e. governments), would need to be members within the club, including bilateral donors that can offer substantial analytical skills. As governments increasingly accept the broader principles on land policy but face the challenges of the practice of land administration, they will have valuable experience to contribute to the dialogue. Regardless of the outcome of any such membership debate, agreed Coalition language on the boundaries and modalities of membership and advocacy would be helpful to all parties. · ILC's new rather confusing hierarchy of vision, mission, core values and principles, core strategies, goals, and strategic objectives, although voted on by members, seem unlikely to help overcome the past perceived lack of clarity among different players about ILC's role and could benefit from simplification. With decentralization, the objectives would now need to be reflected at the regional level and also be reflected back. The objectives of ILC's renewal and reform process itself are not conceptually parallel to the Coalition's overall objectives and would be better treated separately. · The new objective of financial sustainability would benefit from greater realism and conceptual clarification in relation to the public good element in the program's outputs. · The ILC needs to do better at using ILC meetings and conferences to agree on priorities for action and on who will do what for the next phase of action in priority countries and programs. The reports of such meetings offer little insight to members (or reviewers) into what is planned over the next period. The ILC has also had too many scattered activities that do not seem to lie on any plausible critical path towards significant land policy or administration advances. Moreover, these often seem to be interventions that other players could do equally well and are therefore not playing sufficiently to ILC's comparative advantage. xxii · The Coalition needs to retain sufficient connection to an IGO for stability and the ability to recruit and retain quality staff. However, the legitimacy derived from such a connection, even though sometimes perceived by governments, is a false legitimacy. Any claim to legitimacy will be built from its membership, from demonstrating positive results, and from its reputation for collective knowledge, facilitation and capacity building. · M&E will be the lifeblood of the ILC. More focus is needed on outcomes. Demonstrating results and efficiency has now become urgent. While there should be realism of expectations matching the resources, evidence on performance that remains largely at the process and facilitation levels becomes increasingly less convincing over time in the absence of output evidence on the actual enhanced capacities of intermediaries and at least the early stages of attributable outcome evidence on the welfare of poor landowners or the landless. The three M&E priorities should be (a) to develop logical frameworks and measurable indicators at the output and outcome levels at central and regional levels; (b) to develop M&E methodologies for CSO, country and regional levels; and (c) to develop analytical tools so that outcomes at lower levels, whether quantitative or qualitative, can be aggregated. · The ILC needs better brand recognition. It needs to build Secretariat and regional-level land skills, together with skill linkages to other sources, to ensure that, as a Coalition, it can offer a real and recognized comparative advantage on global land issues. 27. The main lessons from this Global Program Review for the World Bank are: · The World Bank should be more proactive in contributing to maintaining the relevance of the program with respect to role, membership, prioritization at the regional level, fiduciary management, and M&E. (The same could be said of other IGOs.) The Bank should particularly monitor the progress of the recent Coalition changes. · There is need for establishing greater clarity within the Bank about its long and short- term goals for working with the ILC. With greater regionalization of the ILC, the Bank will need to watch more closely the reputational risks but should also gain more opportunities to link ILC members' work with the design and implementation of Bank- supported land projects. Task managers of global programs should have Sector Department-approved terms of reference that formalize these long and short-term goals and should be provided budget allocations commensurate with the task. Staff engagement with the Coalition seems often to have been an unfunded mandate in the past. · The Bank's global program reporting systems need to be more transparent with respect to the final destination of the Bank's financial contributions. In the case of ILC, the Bank's financial support from the Development Grant Facility was passed through IFAD because it has been the legal entity for the ILC which also administers the ILC trust fund. That the funds were destined for ILC was not recorded in the Bank's reporting systems. 1 1. Program Objectives, Activities, Financial Resources, and Governance Objectives and Activities 1.1 The history and origin of the ILC are important in understanding the original motivations and commitment of participants and donors. In 1995 concerns about hunger and rural development prompted IFAD to sponsor a Conference on Hunger and Poverty. While the theme of this conference was much broader than land alone, it did have a significant focus on land assets and the need for the poor to play a greater role in the emerging international dialogue on land rights. The conference led to the formation of the Popular Coalition to Eradicate Poverty in 1996, which changed its name to the International Land Coalition in 2003. This name-change represented a narrowing of the intended focus of the original coalition although land issues had been a major focus of the 1995 conference, according to documentation examined by the IEG mission. Box 1. International Land Coalition: Timeline 1995 Conference on Hunger and Poverty 1996 Formation of "Popular Coalition to Eradicate Hunger and Poverty". 1997 ArNet started. Later this program evolved into the KP and NSP programs 1998 Secretariat established 2000 First Community Empowerment Facility (CEF) projects approved 2003 First Assembly meeting and Popular Coalition renamed International Land Coalition 2004 First Strategic Framework. Secretariat expanded to current size. 2005 Constitution and Governance Framework approved. Second Assembly meeting. 2006 External Evaluation 2006/07 Change process initiated. New Framework, etc. Source: Universalia External Evaluation. 1.2 The vision of the ILC in 2003 was that gaining and maintaining secure access to land, water and other productive assets is basic to implementing durable solutions to hunger and poverty, since these result in greater productivity, increased family income, and sustainable land use. 1.3 The mission in 2003 was (the formation of) a global alliance of intergovernmental, governmental and civil society organizations that works together with rural poor people to increase their secure access to natural resources, especially land, and enables them to participate directly in policy formulation and decision-making processes that affect their livelihood, at local, national, regional and international levels. 1.4 The objectives in 2003, which had remained almost the same since inception in 1996, were as follows: 2 (1) To enhance the capacities of its members and partners to help the rural poor, women and men, to gain and maintain secure access to land and related production support services. (2) To facilitate the opening up of space for dialogue with decision makers. The present Global Program Review assesses the performance of the ILC in relation to these two objectives. 1.5 However, the Coalition's new 2007­2011 Strategic Framework, which has been developed partly in response to the findings of the External Evaluation (EE), contains a more complex hierarchy of statements (Box 2). The original ILC objective (1) of enhancing capacity now features as strategic objective (4), and the original objective (2) of facilitating dialogue now features largely as the new strategic objectives (1) and (2). There are now broad statements of indicators for the five new strategic objectives (see page 10 of the 2007­ 2011 Strategic Framework). 1.6 As will be discussed again later under Relevance, although it was voted on by the membership, it is questionable whether this very complex hierarchy of partly overlapping statements is more accessible and actionable than the earlier more simple version, even allowing for some needed redirection. It is considerably more complex than most. To offer a Vision, a Mission, Core Values, Core Strategies, Goals, and Strategic Objectives all together seems potentially confusing to new members less versed in the development and institutional change jargon. These read as though they were in response to multiple conference comments that somehow all needed to be accommodated. 1.7 Taking a horizontal program-level cut, over most of the period since the establishment of the ILC in 2003, the Coalition has had six strategic thrusts (which have become Core Strategies under the 2007­2011 Strategic Framework): (a) A Knowledge Program (KP) (b) A Network Support Program (NSP) (c) A Community Empowerment Facility (CEF) (d) A Women's Resource Access Program (WRAP) (e) A Land Alliances for National Development Program (LAND). (f) A program entitled Common Platform on Access to Land (Platform). 1.8 Supporting these six strategic trusts have been four sets of activities: advocacy activities; policy analysis activities; annual and council meetings; and communication, outreach, and management activities (i.e., the Secretariat's activities). These activities, although in some cases under review or being modified, largely continue except that the CEF grant program funding (the activity which was supported by the World Bank) is now fully committed. Financial Resources 1.9 The ILC is totally dependent on donor resources. It has no revenues of its own, although membership fees are under discussion. From its inception in 1996 through the end of 2007, ILC has mobilized $18.6 million in donations and pledges receivable through 2010 from 3 Box 2. International Land Coalition, 2007­2011 Strategic Framework Vision Secure and equitable access to, and control over land reduces poverty and contributes to identity, dignity and inclusion. Mission A global alliance of civil society and intergovernmental organizations working together to promote secure and equitable access to and control over land for poor women and men through advocacy, dialogue and capacity building. Core Value · A rights-based approach. and Principles · Recognizing flexible and plural tenure systems. · Subsidiarity and responsiveness. · Mutual learning and accountability. Core · Refocus ILC at the national and local levels to achieve demonstrable impact on the Strategies livelihoods of poor women and men. · Forge strategic coalitions and partnerships as conduits for policy advocacy. · Expand and diversify ILC's membership and strengthen the role of members in the direction and work of the Coalition. · Strengthen and focus ILC as a knowledge network. · Develop transparent systems for results-based and gender mainstreamed planning and management. Goal The goal of ILC is to enhance the capacities of its members and partners as well as their opportunities, at all levels, for pro-poor policy dialogue and influence to promote secure and equitable access to and control over land and other natural resources that are vital to the livelihoods of poor women and men. Strategic (1) All members of ILC provide coherent and coordinated support to global, regional and Objectives national commitments and actions to improve the access of poor men and women to natural resources, especially land. (2) Civil society participates more actively in, and exercises greater influence over, the policy and decision-making processes that affect access of poor men and women to natural resources, especially land. (3) Civil society, inter-governmental organizations and governments identify, share and adopt lessons and good practices that improve the access of poor men and women to natural resources, especially land. (4) All members of ILC have increased capacity for networking, knowledge sharing, dialogue and joint action. (5) ILC becomes an autonomous, decentralized, globally representative, member-led and financially sustainable coalition. its supporters (Table 1) of which US$8.6 million came from IFAD. Thus, including 2007, IFAD has funded about 40 percent. But prior to 2007, since the 1996 inception of the original coalition, the share of IFAD was about 65 percent. Other significant contributors have been the World Bank, Italy, Canada, and Belgium. However, the Bank contributed only once, in 1998. Annex E provides additional financial information. Table 2 shows recent annual expenditures. Governance Structure 1.10 Briefly, the ILC has a governance structure as shown in Box 3 with an Assembly of Members, a Coalition Council, and a Secretariat. The 2006 External Evaluation offers a more detailed organization chart. 4 Table 1. Donations and Pledges to International Land Coalition since Inception, 1996­2007 (US$ thousands) Donor 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 Total Percent IFAD 50 900 1,600 200 1,000 500 600 1,400 965 1,700 8,915 47.9% Netherlands 230 2,000 2,230 12.0% EC (European 2,170 2,170 11.7% Commission) World Bank 1,500 1,500 8.1% IDRC (Canada) 1,141 1,141 6.1% Italy 60 300 400 760 4.1% Canada 68 183 396 647 3.5% Belgium 500 500 2.7% Japan 120 198 318 1.7% Switzerland 35 200 235 1.3% WFP 100 100 0.5% Denmark 64 64 0.3% FAO 21 21 0.1% Total 50 964 3,436 698 1,468 500 600 230 1,400 1,848 396 7,011 18,601 100.0% Source: ILC Secretariat. 5 Table 2. ILC Expenditures by Sources of Funds, 2005­2008 (US$ thousands) 2005 2006 2007 2008 Total Percent IFAD /1 1,262.3 1,948.8 925.0 1,026.0 5,162.1 39.0% EU 0.0 0.0 910.9 1,308.5 2,219.4 16.8% World Bank 372.0 624.3 389.6 94.0 1,479.9 11.2% BSF 0.0 505.3 388.0 239.0 1,132.3 8.5% Netherlands /2 154.0 77.4 16.4 750.0 997.8 7.5% IDRC (Canada) /3 0.0 0.0 17.3 745.0 762.3 5.8% Switzerland 182.0 207.0 256.2 76.0 721.2 5.4% Canada 186.5 106.6 264.3 59.6 617.0 4.7% United Kingdom 28.9 27.3 14.2 0.0 70.4 0.5% Italy 31.3 6.9 10.3 0.0 48.5 0.4% Japan 13.5 6.9 8.0 0.0 28.4 0.2% Others 0.5 0.0 7.8 0.0 8.8 0.1% TOTAL 2,230.9 3,511.0 3,207.9 4,298.0 13,247.8 100.0% Source: ILC Secretariat. /1 The 2007/8 figure is composed of 2nd year of IFAD TAG 906, approved for period 2007/2008 and remaining funds under TAG 416 (= USD 79,000), approved in 1998, dedicated to CEF /2 The Dutch contribution, approved in December 2007, is for the period 2007/2009. Here is reported the portion received and receivable in 2008 only. The total amount of the contribution is US$ 2,000,000. The balance (US$ 1,250,000) will be disbursed in 2009 and 2010 in two equal tranches of USD 550,000 and 700,000 respectively. /3 The IDRC contribution, approved in October 2007, is for the period 2007/2009. Here is reported the portion received and receivable in 2008 only. The total amount of the contribution is C$ 1,100,000. The amount receivable in 2009 is approx USR$ 255,000 depending on exchange rate. A possible addition of C$ 600,000 is being negotiated at present for approval in the course of 2008. 1.11 As shown in Box 4, the ILC has four constituency types: members, partners, donors, and affiliates. Members have full voting rights. Partners have participated. Donors have provided funds but do not have a voting right unless they are also members. Affiliates have had some periodic involvement. As noted in the EE, the difference between partners and affiliates has not been entirely clear and some stakeholders have been unclear as to what their own status has been, based on survey responses to the EE. Box 3. ILC Structure Assembly of A supreme body that meets every two years and establishes overall strategy Members and policy and ensures that the organization remains in compliance with the intensions of the members. Coalition Council The Executive Board with 14 members responsible for overall governance. Secretariat Responsible for management, operations, coordination and administration. Most staff are based in Rome at IFAD; two are based in New York. (See staff list in Annex D.) Partners/Affiliates These are less formal designations. They are entities, mostly CSOs, which have interacted in some significant way with the ILC. 6 Box 4. Constituency Types Members Organizations with voting and decision-making power in the Assembly. Members eligible to be elected onto the Council. At the time of the EE there were 36. (9 IGOs, 27 CSOs, (Not less than 60 percent of members must be CSOs) The numbers have now increased to over 50. Partners Organizations that have participated in the Coalition but do not have voting authority. Future members may be nominated from this group. (Currently 111 partners from 43 countries.) Donors Organizations that have provided resources. They are invited to the Assembly in a non- voting capacity and consulted and may be considered for membership. They may also be partners. (About 15 donors since founding.) Donors can also be members so there is some overlap in constituency types. Affiliates Organizations having periodic involvement with the Coalition but who are not members, partners or donors. (About 40 affiliates.) 2. The External Evaluation of the ILC Scope, Methodology, and Approach 2.1 The Executive Board of IFAD commissioned the 2006 External Evaluation of the ILC on its own initiative, while also discussing this with other members, and contracted the task in March 206 to Universalia, a Montreal-based consulting firm that has done a number of evaluations of Bank-supported global programs. The purpose was to provide IFAD with an independent view of the performance of the ILC in achieving its objectives and to inform the ILC Coalition Council about institutional and program progress. The evaluators were expected to identify lessons and to make recommendations for the future. The TOR are available in the May 2006 Inception Report. 2.2 The evaluation was to focus on the three widely used evaluation criteria of relevance, effectiveness and efficiency. Relevance was to include the continuing relevance of the program as the development and political environment changed. Effectiveness (or efficacy) was in relation to the two strategic ILC objectives and in the light of changing development priorities. Efficiency was mainly focused on the appropriateness of the processes and on the assessment of progress in building an effective coalition. While these definitions in the EE encompass the well established evaluation triangle of relevance, efficacy, and efficiency, there is some definitional overlap. Defining efficacy in the light of changing priorities might require an adjustment in the objectives and therefore become more of a relevance issue. The progress in building the Coalition might be more of an efficacy issue than an efficiency issue. 2.3 The evaluation was carried out by a nine member evaluation team which visited ILC and IFAD in Rome and seven countries in April 2006 (Guatemala, Honduras, Nicaragua, Kenya, South Africa, Uganda, and Zambia). The team visited a total of 62 individuals in 40 organizations, and its final report was issued in August 2006. Overall, the team consulted about 120 people and collected evidence through surveys, interviews, observation, documentation and correspondence as well as from the land reform literature. Four brief case 7 studies were prepared on ILC activities with the EU, FAO, IFAD and the World Bank, which focused on the relationship between the ILC program and these institutions. The evaluation team presented a draft report to the ILC, and took into account the feedback received in the final report. This feedback was mainly but not entirely from the Secretariat. The World Bank's Land Thematic Group also provided comments. Independence and Quality 2.4 The host organization, IFAD, initiated and managed the evaluation process under the leadership of the Assistant President for Programmes but not under the direct leadership of the Evaluation Group. This followed a request for an evaluation by the IFAD Board in keeping with the 2003 IFAD policy for Grant Funding. IFAD prepared the TOR in consultation with the Coalition Council and approved the methodology through an IFAD- managed oversight Evaluation Steering Committee. At the outset, the ILC Secretariat provided input on the proposed methodology to IFAD, and subsequently provided comments on the design of the survey questionnaire to the EE team. The evaluation team consisted mainly of land and civil society specialists and appears to have been largely free to operate independently. A few IGO observers interviewed felt that the ILC Secretariat had too much influence through comments on the proposed survey and on the preliminary draft report. It is noteworthy that the evaluation was intended, first and foremost, to meet the needs of IFAD, not the other stakeholders. The TOR says that "the main objective ... is to provide IFAD ... with an independent review of the progress being made in achieving the mission and strategic objectives of the Land Coalition." 2.5 Most IGO staff interviewed, while by no means agreeing with everything in the EE, felt that it had uncovered a number of important issues. A few felt it could have been more critical. It was carried out in an impressive 90 days (speed was demanded by the TOR) and it addressed most aspects of ILC's work. 2.6 The EE had a number of strengths, including (a) a The External Evaluation thorough review of the relationship between ILC and IFAD, had a number of strengths including the ways in which IFAD exercises control over ILC; but could have focused (b) a thorough review of the organizational structure; (c) useful more in some areas, comments on the Strategic Framework; (d) a well-formulated especially on the ILC's set of inferred results-based indicators in the absence of advocacy role, its available ex ante results indicators; (e) a useful discussion, comparative advantage, albeit not pursued far enough, on the advocacy issue; and and its cost-effectiveness. (f) pointing out a number of anomalies including the confusing language in the early documents on whether governments could be members of the Coalition.1 2.7 The EE also had a number of weaknesses. First, the EE gave too much emphasis to the issue of the legal relationship between ILC and IFAD and initially seemed to recommend the extreme option of spinning off ILC completely disconnected from an umbrella organization -- 1. The ILC Secretariat has responded verbally that the language indicated that there would only be "alliances with governments", which was not intended to imply that governments could be members. Indeed, governments never have been members. However, the language, which varied somewhat in different earlier statements, was never entirely clear to outside observers. 8 without much reference to the experiential evidence from other global networks. (After comments from ILC and IFAD, what seemed to be almost a single recommended option was modified somewhat.) Second, the EE did not sufficiently explore the potential roles of the ILC along the continuum from a predominantly analytical research and knowledge generation role to a more explicit advocacy role, along with the most appropriate membership pattern in relation to these different roles. Third, the EE did not appear to factor into its conclusions the limitations of surveying members and partners who either had, or could potentially, benefit from funding provided by the ILC.2 Fourth, the EE chose as its starting point the year 2003 when the first Assembly meeting took place. While this can be considered the formal starting point of the ILC phase of activity, the work of the precursor coalition had been going for about seven years prior to this. While it would not have been productive, given the weak M&E, to assess the component activities of what was a somewhat different and still incipient precursor organization, the EE should have probed a little more concerning the length of time it had taken the Coalition to reach its present stage of maturity, which has implications for efficiency. The ILC should not be treated today as a relatively new organization. Fifth, the EE gave limited attention to the cost-effectiveness aspect of efficiency and, perhaps to compensate, brought together a range of issues under the heading of efficiency that could have been treated under the heading of efficacy. The EE might have explored more, for example, the cost-effectiveness of a sample of CEF activities, although it is recognized that the lack of input and cost data attributable to particular outcomes make cost-effectiveness analysis difficult at any level. Sixth, the EE did not sufficiently explore ILC's comparative advantage against other possible alternative actors. The present IEG review has tried to advance some of these issues further. 2.8 The EE could not follow a formal results-based framework based on objectives and indicators established at the Coalition's inception because such a framework did not exist. The evaluation team therefore had no choice but to infer sets of indicators from the statements of objectives and program components. These inferred results indicators were well formulated by the EE, but inevitably called for some speculation about what was intended since these were constructed ex post. 2.9 The EE experienced a poor response from its surveys at the "contact" level, possibly because of the very tight timeline. The weakness of the response from ILC "contacts" (as opposed to from members and partners) is one of the more striking evaluative findings, which raises some questions about the level of interest "outside the club." Of 35 Members, 14 (40 percent) responded. Of 66 partners, 22 (33 percent) responded. But of 100 randomly selected ILC contacts only 7 (7 percent) responded, even after three reminders. This does weaken the findings on "contact" questions and seems to suggest a lack of interest. But responses cannot, of course, be forced and low response rates are common in such surveys. 2. The description of respondent categories in the EE is somewhat confusing. Approximately 120 individuals were interviewed in "23 member and 32 partner organizations, and other land related institutions." Within this group, it is not clear what percentage, if any, were outside the ILC family. Those inside would appear to have motivation to report favorably given the availability of grant funding, those outside less so. It is indicated in EE Exhibit 1.1 that, in the category "ILC contacts (other land-focused individuals and organizations)", in the footnote, that this category was included to "solicit the views of leading land specialists who are not currently ILC members, partners or affiliates". The table shows that 100 out of a population of 332 individuals was "consulted". Out of the 100 only 7 responded to the survey. It is not clear what the population of 332 came from, or how the 100 were selected. However, it is clear that the response rate for these "outsiders" was much lower than for "insiders". It is possible that this says something about limited perceived relevance outside the immediate circle of members and partners. But there may be other explanations. 9 2.10 The EE could have gone further in several areas instead of focusing so much on the legal issue. But it identified many of the key issues, and its findings were generally sound. (These are reproduced in Annex B.) The present review is intended to provide a second opinion on the effectiveness of the program and has drawn substantially from the findings of the EE.3 2.11 The World Bank's Land Thematic Group made a number of valid comments on the preliminary draft of the EE (Box 5). Although the evaluation team accommodated these comments to varying degrees in the final report, these comments signal some of the more contentious areas for the Bank's land community. Overall, the Thematic Group found the evaluation "far-reaching, broad in scope and very useful for exploring the accomplishments of ILC." It found the evaluation to be in many ways a strategic planning discussion paper that provided a broad consideration of strategic directions. Box 5. Key Comments of the Bank's Land Thematic Group on the Preliminary Draft of the External Evaluation · The EE could have confronted more clearly the question of whether the ILC has distinctive advantages. · In response to the possibility of changing the World Bank's status to affiliate because full membership repels some social movements, the Thematic Group noted that this was a complicated question. The social movements in question may not be committed to the kind of evidence-driven policy dialogue that the ILC hopes to promote. There is a risk of politicization. · The EE did not sufficiently assess why, after 10 years, the ILC was still at such an early institutional stage. (The EE team responded without substance, simply reaffirming that they took 2003 as their start year when the Assembly and CC were first formed.) · More dialogue and more Bank support to land-related projects had taken place than implied in the EE draft. The Thematic Group noted, for example, that while the draft report said that market-led land reform was seen as an inadequate approach, this was only one limited point of view. (In the case of Brazil, this negative viewpoint was seen as coming from MST, yet other larger rural movements in Brazil including FETAG and CONTAG, which exceeded MST membership, were strongly supportive of market-led reform.) · The issue of the skills of the Secretariat staff, in particular the issue of generalist versus technical skills, warranted more attention. Impact of the Evaluation 2.12 The evaluation has been taken seriously by both ILC and IFAD. Since the evaluation, there has been considerable activity in the direction of implementing the main recommendations. Briefly, this has included a new strategic document and action plan, an operating framework, an important paper on legal status, a regionalization process with regional meetings, a membership drive leading to a doubling of membership, an M&E framework, and development of a communications strategy. In addition, a Memorandum of 3. The actions taken by ILC since the EE have also been highlighted in this report. However, the outcomes of these recent changes have yet to be observed. For example, the very useful review of ILC's legal status is only now contributing to the dialogue with IFAD which is still on-going and the impact of M&E plans is still to come. So, while the recent changes are indeed promising, it is still early to assess impacts. 10 Understanding has been under discussion with IFAD to govern their relationship going forward. The response to the evaluation has been positive and proactive. The milestones set in the Action Plan are reported to have been achieved ahead of schedule. Supporting this renewal process, a large amount of new funding has been obtained, partly due to a substantial fund-raising effort by the ILC, partly due to the EE and ILC's positive response to it, and partly due to the increasingly high profile of land issues globally. The European Commission, IFAD, IDRC, and the Netherlands have committed over US$7 million in one year (2007). There is an air of cautious optimism surrounding the program. 3. The Effectiveness of the ILC4 3.1 The present review follows the current IEG evaluation methodology outlined in Annex A. The product is called a "review" rather than an "evaluation" since it is an assessment with modest resources that is based on the previous more substantial External Evaluation of the ILC by Universalia. A network cannot be evaluated 3.2 Assessing the performance of a network is in the same way a project can. different from assessing the performance of a project. Attribution will always be Some of the recent literature on networks has started to difficult. Members are not address these differences (e.g., Church et al., 2002, and subordinate and will have a Wilson-Grau, 2007).5 Nevertheless, this review still uses range of motivations. Never- the standard Relevance, Efficacy and Efficiency criteria, theless, member interaction and along with Sustainability and Governance, since the capacity alone, without uniqueness of networks does not undermine these demonstrated progress towards fundamental elements of the evaluation framework; it beneficiary outcome, will not be enough to keep a network alive. simply suggests the need to apply them appropriately.6 4. This section of the GPR takes the evaluation as its point of departure but goes beyond it with information from outside sources, including especially interviews carried out for the GPR. See Annex F for persons consulted. 5. Points being made in the recent literature about the evaluation of networks include: (a) Networks operate in a more unpredictable environment than organizations or projects. Outcomes sought are difficult to predict and long-term. They operate within a web of relationships, most of which cannot be controlled by either the secretariat or the collective membership. Yet this very unpredictability is closely allied with the possibility (but not the certainty) of shifts at the frontiers of knowledge. (b) Since networks work through members, some elements of outcome can be validly defined as changes in members' capacities, at least over the short to medium term. However, in the longer term it would be expected that the enhanced capacity would translate into actual impact on poor land user or landless households. (c) Neither the secretariat nor the individual members can reasonably be held accountable for poor impact of individual members on beneficiaries since these two parties have limited influence. Nevertheless, resources can be expected to dry up at some point without collective attributable impact on beneficiaries. (d) A network is not the sum of its parts but a product of the interaction of its parts. (e) A network cannot be "managed" in the same way as a project to achieve stated objectives. Individual parts may be able to guide other parts on the basis of an agreed vision, but none can manage more than their own contribution. Members are not subordinate partners. Effective networks are non- hierarchical. Members must be expected to have a range of motivations and commitment. (f) Attribution will always be difficult. Positive linkages may only become apparent after some years. 6. IEG's evaluation framework for global and regional partnership programs (GRPPs) like ILC, in which the World Bank is only one partner, explicitly recognizes that such programs require special treatment in evaluation because they are different from the Bank's typical investment projects. The evaluation framework adapts the 11 Relevance 3.3 Relevance7 is discussed under five main headings: (a) the existence of an international consensus for global collective action on the issue and for the Coalition itself; (b) the extent to which the objectives of the Coalition are aligned with the needs, priorities, and strategies of beneficiaries; (c) consistency with the subsidiarity principle, that is, the extent to which the Coalition's activities are conducted at the most appropriate level -- global, regional or national; (d) the Coalition's comparative advantage in relation to alternative sources of supply; and (e) the relevance of the program's design, including the rationale for its main activities and its results-chain logic. INTERNATIONAL CONSENSUS 3.4 The statements of the original founding conference and subsequent meetings, the strategy statements of many donors, and the positive member and partner comments in the EE survey suggest an original and continuing consensus at the global level on the importance of the land reform and administration issue itself. As noted earlier, there has been some narrowing of ILC's intent from the original poverty and hunger conference that led to the first incarnation of the Coalition. But this appears to have been a pragmatic shift towards a sharper focus and to have mostly occurred very early on. Does this narrowing really matter? Probably not, since the narrowing was pragmatic and enhanced ILC's focus. But this still leaves some discomfort about original legitimacy, particularly since, as expanded on below, the 2003 International Conference on Agrarian Reform and Rural Development (ICARRD) did not reconfirm that original legitimacy. To outside observers, there was also a narrowing in eligible alliance partners when ILC changed its mission from being a "global alliance of intergovernmental, governmental and civil society organizations" in 2003 to a "global alliance of civil society organizations and intergovernmental organizations" in 2007 (Box 2). It is not entirely clear how or why this latter change occurred. 3.5 There remain three other points about international consensus. First, notwithstanding the positive EE survey response to the questions on relevance, the survey was not representative of a large number of the original founding conferees. Second, when it came to the ICARRD in 2003, a conference of similar significance to the 1995 Conference on Hunger and Poverty, the ILC was not seized upon as the obvious vehicle for promulgating the quite similar resolutions. In fact, the ILC was not even mentioned in the final communications at all. What does this mean for relevance? Interviews with IGO staff and the ILC offer standard evaluation criteria of relevance, efficacy, and efficiency to the special characteristics of GRPPs and supplements these with additional criteria relating to the sustainability, governance and management of the program. IEG acknowledges that there are benefits which arise from "a sense of belonging, active participation, shared responsibilities, etc." that may be hard to capture in assessments of outputs and outcomes, but does not accept that these can be wholly ends in themselves in an environment of scarce development assistance resources and when these are also attributes of virtually all global partnership programs. 7. The EE defined and assessed Relevance in a somewhat different way from typical World Bank evaluations. The EE describes relevance as asking the question: "Is what we are doing related to real identified needs?" Whereas the Bank's Independent Evaluation Group (IEG) defines it (summarized) in the following way: "relevance of objectives is the extent to which the ... objectives are consistent with the current development priorities and with current Bank .. strategies and corporate goals.."; "relevance of design is the extent to which the ... design is consistent with the ... objectives." The Bank formulation limits the freedom of the evaluator to determine ex post what the real needs were at the time of program initiation or are today. 12 convincing and quite complex political and turf-protection explanations, involving a number of players and including the impact of IGO membership, all indicating that there was little that ILC could have done about this. Nevertheless, the lack of some further mandate emerging from a major global conference with similar objectives to the founding conference does raise some concerns about global perceptions by some players of ILC relevance. However in other international fora, such as the United Nations Economic and Social Council (ECOSOC), the ILC has been able to help keep the land issue on the agenda. While the literature and donor documents suggest that the issue of land itself has been of rising While the literature and importance over the last decade, it is not yet clear that this donor documents suggest that has lifted the perceptions of ILC's importance. Third, both the issue of land itself has IFAD and Bank staff seem at best to be of two minds about been of rising importance whether the ILC has been relevant. IFAD staff are reported over the last decade, it is not by the EE as being quite negative. The present review found yet clear that this has elevated similar views although not without some positive elements perceptions of ILC's and readiness to remain open to the possibility of importance regeneration and growth. 3.6 Some practitioners have pointed out that in the last few years the pragmatic design of workable policy and the associated practice of cost-effective land administration is becoming more of the challenge than the broad equity-focused principles of land policy which many governments increasingly and largely accept. In other words, for governments, often the biggest challenge, in a highly confused land situation (such as in Ghana or Kenya) is not the principles of the policies but how to actually apply them on the ground. ILC Secretariat skills and member capacity will need to stay ahead of this gradual shift away from principle towards the practice. The lack of government membership in the ILC may become more of a handicap in maintaining relevance as governments gradually gain greater understanding of the management challenges from their learning by doing experience. 3.7 That said, there is a recent more positive sign of international consensus presumably reflecting a perception of relevance. A number of donor agencies and respected international institutions have come forward with funding including IDRC, Canada, IFAD, EC and Netherlands. There has been a very substantial surge in funding. ALIGNMENT WITH BENEFICIARY NEEDS AND PRIORITIES 3.8 Coalition objectives and component programs appear to be substantially consistent with the strategy statements of beneficiary country rural strategies, which in recent years increasingly articulate the importance of land access and rights and the means to achieve these. They are also largely consistent with the Bank 2003 Land Policies for Growth and Poverty Reduction paper (Box 6). However, it is difficult to rate such consistency with country strategies in the case of an organization that places great importance on creating a space for dialogue, because some divergent perspectives are necessary in order to have a debate and are therefore by definition not wholly consistent with existing country strategies. Indeed, some CSO members see the main purpose of their involvement as firmly pushing reluctant governments in new directions, an activity that, taken to an extreme, can make IGO members who have other relationships uncomfortable. 13 Box 6. The Land Focus in the World Bank's Land Policies for Growth and Poverty Reduction Paper In 2003, the Bank's Research Group produced a Policy Research Report on Land Policies for Growth and Poverty Reduction. Though not a formal Bank policy document, it has contributed to the understanding of land right issues within and outside the institution and also provides the most recent guidance to Bank staff on the subject. The main messages of the paper rest on three principles: · The importance of the security of tenure in improving the welfare of the poor and the incentives to invest in land improvement, · The importance of land markets in getting productive but land poor producers access to markets, · The role of governments in promoting and contributing to socially desirable land allocation and utilization. The paper found that the focus on formal title in the earlier Bank 1975 paper was inappropriate and that greater attention would be required to the legality and legitimacy of existing institutional arrangements. It also concluded that dealing with efficiency would not automatically resolve equity issues. Source: World Bank, "Land Policies for Growth and Poverty Reduction," June 2003. 3.9 Indeed, Coalition members could profitably explore the ILC's role in advocacy. How far should its advocacy go? In exploring this, it is important to separate the Coalition Council and Secretariat from the membership. One can presumably view advocacy as making a potentially positive contribution to development to the extent that what is being advocated are effective and efficient strategies based on strong evidence to achieve growth and equity objectives. Since different members have different objectives, advocacy by individual members carries risks for all. But it also carries risks for the Coalition as an alliance. The EE mentions some criticism from its interviews and surveys questioning the Coalition's legitimacy in acting as an advocate and facilitator on behalf of civil society in relation to its work with the African Union. An IGO staff member also mentioned some discomfort about the ILC's role with respect to facilitating dialogue in Madagascar. Some, but by no means all members are believed to have joined the ILC partly to try to push the World Bank more towards a particular advocacy role at the country level, which the Bank has resisted. 3.10 The EE made some sound proposals on this issue (p. 62), but perhaps did not push it strongly enough. The evaluation recommended that the ILC better define its advocacy role -- that it should stick to being a facilitator of dialogue at the country level (and presumably also a conveyor of peer-reviewed knowledge and other similarly neutral things), but should continue to advocate the broad principles of better land access for men and women at the global level. This makes sense since all members have implicitly signed on to the ILC vision and objectives. The EE notes that both the above roles imply relative neutrality. The present IEG review concludes that it would be helpful for the Coalition to agree on language regarding the boundaries of advocacy, including some specificity with respect to positions that might be advocated, ground rules for advocacy, and processes for reaching Coalition consensus on anything to be advocated. 14 SUBSIDIARITY 3.11 Up to now, the ILC has been dominated quite strongly by the Secretariat. Indeed, some CSOs responded to IEG that they had found this excessive.8 This was probably highly desirable in the formative years, since it is doubtful that the ILC would have survived without strong central leadership. However, both the EE and the membership have come to the view that the loci of dialogue space and knowledge generation should now shift down to the regional level. While this decentralization probably makes sense in terms of both efficiency and responsiveness to beneficiaries, it remains to be seen whether the regional members are sufficiently prepared and resourced to take up this additional role with cost burdens when they were predominantly recipients of support in the past. Decentralization per se does not assure substance. It still puts form before content, although it may ease some transfer of lessons. 3.12 There remains the question of how transferable the lessons are between regions. If there are few lessons that are transferable, then the remaining central ILC secretariat would have limited value. However, the predominant view of those interviewed, which the academic literature seems to support, was that there are relevant lessons that can be transferred between regions and that more advanced developing countries can offer those at a lower level of development glimpses of future directions that could usefully impact the design of earlier steps on the ladder. COMPARATIVE ADVANTAGE 3.13 There are some questions about the skills of ILC's staff. The World Bank thematic group, in their comments on the EE, noted that the EE could have confronted more clearly the question of "whether the ILC has distinctive advantages ... or has filled a previously unfilled gap." There are three important points here. First, the ILC Secretariat seems to have the skills for managing global or national dialogues, which are important in any network program. However, the ILC is not unique here: FAO, IFAD, DFID, the World Bank and others have also initiated many global dialogues in recent years. Nevertheless, the ILC has a valid claim that at least some stakeholders see ILC as more neutral as a convener than other agencies in an issue with great political implications, both national and global. There are certainly some CSOs that would come to a meeting convened by ILC that would not come to a meeting convened by the World Bank. Second, some professionals in the land policy field have expressed concerns that the ILC, while quite strong on facilitation skills, has been weak in land administration and land policy analysis skills, at least until recently with the recruitment of one land-specialized staff member. Third, other organizations such as FAO, CAPRi, Oxfam, DFID, the International Association of Surveyors (FIG) and some University-based organizations also serve as useful knowledge sources. Indeed, few IGO interviewees go to the ILC Web site first for information. Still, no other network or organization seems to offer the full array of ILC's areas of focus on land reform and administration, plus facilitation support, although many different organizations can provide one or more elements of it. 3.14 In its draft membership strategy, the ILC claims that this strategy will enrich ILC's "unique status." But disaggregating its various component programs and roles, it is questionable whether the ILC has yet reached the point of being sufficiently unique and 8. Comments to IEG included: "ILC is like an exclusive club"; "ILC left out representative organizations ... and chose some private NGOs ... the criteria of choice ... not being transparent at all." 15 influential in the facilitation of global, regional, or national dialogue to really stand out in the global land field. It is still seen more as a potentially useful player among many. In its CSO capacity-building role, it is more difficult to see the ILC having a comparative advantage over many other donors and organizations who can also directly or indirectly offer support for strengthening the capacity of NGOs and community groups. In the area of community empowerment, it is also difficult to see the ILC Secretariat or members as having a significant advantage over many other players. Since community empowerment is now at the heart of much of development, there are widespread skills in this area in most countries and within most institutions. But it is arguable that the ILC can better provide some elements of such support which have a particularly land-oriented empowerment and capacity focus -- offering financial support and capacity-strengthening processes that NGOs and their communities can use to dialogue with government departments, such as the CEF support to the Kenya Mau Community Forest Association which helped both the CSO and supported groups negotiate with the Forest Department. In its purely knowledge-sharing role, the ILC is clearly only one of a number of sources. This review found some good published material from the ILC such as that on common property resources and a few quite strong country level studies supported. But other strong material is available elsewhere, too. RELEVANCE OF PROGRAM DESIGN 3.15 As designed, the Coalition program was insufficient in scale to achieve its very ambitious objectives, including positively impacting the poor at country level, when only some individual CSO members had significant potential to reach the household level on any scale. It would have been better to set out shorter-term measurable and achievable objectives within a frame of more ambitious and less certain longer-term aims. However, it needs to be acknowledged that the ILC was not a project but a process that was set in motion with uncertain outcomes regardless of scale, but with a valid conviction that effective land policy and administration were of great importance for addressing poverty. Unfortunately such uncertainty did not trigger, as it should have, strong M&E for rapid feedback to facilitate mid- course corrections. 3.16 The New Strategic Framework, 2007­2011. With respect to this recent shift in program direction, the new 2007 hierarchy of vision, mission, core values and principles, core strategy, goals and strategic objectives is very complicated. This array of intents has the potential to reduce The new hierarchy of vision, mission, values, the clarity of role, confuse potential members, and make strategy, goals and evaluation difficult. A simpler formulation would be a longer- objectives is too term Vision statement and a set of shorter to medium-term complicated and warrants Objectives that would articulate a basis for measurable revisiting. It may confuse achievements. Some statements of intent refer to the reform of potential members and the ILC itself and are therefore not logically parallel to the result in less clarity on others. It may be better to treat these separately. They are the what ILC is trying to means for how the ILC would change as opposed to intended achieve and make global outcomes. evaluation more difficult. 16 Efficacy 3.17 The EE found in its survey that most CSO members and partners indicated that the ILC was "working successfully towards its mission and objectives." This gives limited qualitative evidence since it suggests only forward movement rather than a point reached. It is also worth noting that several of the respondents were members and partners who either had, or could potentially benefit from ILC funding. Among the CSO staff who were either members or had knowledge of the ILC, and who responded to the IEG study consultation, ILC members/partners had a significantly more favorable opinion than the non-members who reported knowledge of the ILC on four ILC-related questions -- relevance of the ILC to the organization's objectives, relevance in bringing global experience, efficacy in enhancing capacity, and efficacy in facilitating dialogue. Written, open-ended comments from both members and those non-members who had knowledge of the ILC were fairly evenly balanced with about half being neutral about the ILC, about one-quarter being positive, and about one-quarter being negative. 3.18 The typical view among IGO staff interviewed was more negative. Most felt that the ILC had been of some value in a few countries in facilitating dialogue and in supporting capacity building of member and partner CSOs and in enhancing global knowledge in a small way, but the ILC Secretariat has not had sufficiently skilled staff to help advance informed land policy dialogue at sufficient depth for IGOs and other donors to build upon for the design of promising land project interventions. Most felt that the idea of a global network was still attractive, but that the experience so far had been disappointing. ACHIEVEMENT OF PROGRAM OBJECTIVES 3.19 Efficacy in the achievement of objectives is assessed here against four main questions: (a) the extent to which the stated objectives were actually achieved, including the achievement of any implicit, unstated objectives and any unintended impacts; (b) the attribution of such achievements to the ILC;9 (c) the extent and type of factors influencing the observed outcomes, including the location of the program and mid-course corrections in the direction of the program as a result of learning; and (d) linkages between Coalition program activities and country level land-related activities. 3.20 Logical Chain. However, to explore those questions better, IEG first attempted to reconstruct, ex post, a highly simplified theory-based logical chain. The logical chain postulated has two main trunks corresponding to the two main objectives, the capacity- building objective and the facilitation objective (Box 7). This very simplified chain is an attempt to reconstruct what seems to have been at least the framework of the program design logic, although nowhere has this been explicitly expressed as such. Such a logical chain, or 9. It is important to distinguish between the respective roles of the Secretariat and the coalition members but attributable outcomes arise in the final analysis from the synergy between the two. Therefore, a review such as this must in the end focus more on assessing the efficacy of the organization as a whole, rather than on diagnosing which parts have contributed most to its strengths and weaknesses. However, IEG has not avoided commenting on the respective roles of the Secretariat and coalition members where this has been important and has signaled the different roles in a number of places, Notwithstanding the original intent that the Secretariat would be the facilitator and the members the life-blood and the action arm of the Coalition, the Secretariat has been quite dominant in the view of many observers, which was probably necessary in the early stages. 17 Box 7. Imputed Logical Chain Corresponding to the ILC's Two Main Objectives -- Capacity Building and Facilitation -- and the Predominant Place of Each Component Program (a) The ILC as a coalition receives financial resources, member participation input, and a measure of global legitimacy from donors and members and passes on resources to members. (b) This leads to a range of interventions by both the Secretariat and participating members and other stakeholders towards the objectives, particularly in the direction of the capacity building and facilitation objectives. The vehicle for these interventions are the main component programs (e.g. KP, NSP, CEF, WRAP, LAND). (c) The capacity building branch of the chain: Through grants, direct TA from the Secretariat, the Web site, ILC studies, and interaction at meetings, CSO stakeholder capacity on land issues is enhanced. The Community Empowerment Facility does this directly. Other components do it both directly and indirectly. The Knowledge Program inputs increase understanding of land issues at country and regional level. The Women's Resource Access Program builds the capacity of women to become active on women's land issues. The Network Support Program inputs build network linkages that further enhance knowledge and encourage collective action on leading land issues. These activities, many of which can be applied to both branches of the logical chain, result in improved quality of advocacy, knowledge, and skills in the area of land reform in priority countries. · These inputs and outputs are eventually reflected, through legislation or institutional development (in the broadest sense), in local level improvements in land ownership pattern, or asset security, or more effective rental law. · These changes are reflected in measurable increases in assets of the poor and/or incentives to invest in assets for productive purposes (provided that pricing policy, supporting services and infrastructure are also supportive). · These changes, probably after some years, are finally reflected in higher level outcomes in the form of the expected enhancements in income and welfare of the poor and also in their continued capacity to demand and contribute to pro-poor change at community level. (d) The facilitation branch of the chain: Knowledge is generated through the interactions and spread of skills mainly among ILC members and partners but including affiliates and outsiders. The Land Alliances for National Development Program lies predominantly in this branch of the logical chain helping to establish country level dialogues. · This facilitation feeds into the capacity branch above at the point of national level changes. From that point on, these facilitation benefits run more or less parallel to the capacity-building benefits and outcome indicators would be largely the same leading towards impact at the level of poor households. Facilitation contributes to knowledge transfer and capacity building although it also has an end in itself. Source: Constructed by IEG. some more detailed variant, could in the future form the basis for M&E design including input, output and outcome indicators. Having such a chain could also prompt questions about the ambitious level of the program's objectives. The matrix recently developed in the 2007ILC Monitoring and Background Paper represents the start of this process, although this focuses predominantly on outputs rather than outcomes. 3.21 The present review turns now to the efficacy of ILC's program and network measured against stated objectives and backed by some triangulation across the ILC component programs frame. It will be recalled that the first objective over most of the period under review was "to enhance the capacities of its members and partners to help the rural poor, 18 women and men, to gain and maintain secure access to land and related production support services." The second objective was "to facilitate the opening up of space for dialogue with decision makers." Capacity Enhancement 3.22 The first objective calls for answering three somewhat separate questions: · To what extent was member and partner capacity enhanced either directly or indirectly through such support as knowledge, networking, community empowerment, support for women, land alliance, and the common platform; · To what extent did the poor actually gain, or stand soon to gain, greater land access as a result of this support; · To what extent did the poor gain in the area of related production services. 3.23 The evidence on the enhancement of capacity, directly or indirectly, suggests mixed success. The EE survey found that respondents noted the success in strengthening capacity, although it should be recalled that some respondents had benefited from grants in this direction. The EE does not indicate the percentages of respondents who reported positively, neutrally or negatively. Areas of capacity building cited included policy reform, experiences from other countries, increased visibility and credibility of CSOs through their international connections (only obliquely a capacity issue), enhanced ability to structure and present evidence to advocate for their cause, and greater capacity to service their own members with knowledge. 3.24 Lacking the resources to do a full sample assessment, the review mission looked through a small selection of CEF cases on file and Web site reports. There seems to have been a range of quality. Some interventions made some advances on land issues such as the titles issued to 2,691 farmer beneficiaries from 21 groups in the Philippines who benefited partly due to support from the Philippines Land Watch NGO (AR Now!) and the National Federation of Peasant Organizations (PAKISAMA). Other interventions -- such as the Southern Alliance for Indigenous Resources (SAFIRE) in South Africa or the South Asia Rural Reconstruction Association (SARRA) in Bangalore, India -- seemed only distantly relevant to land policy advancement or could have been readily supported by any number of community-driven development organizations or projects. On capacity building, IEG's consultation with CSOs found a high level of satisfaction from ILC members but a lower level of satisfaction from non-members with knowledge of the ILC. While the present report focuses predominantly on the more recent achievements, there were capacity outcomes earlier that were supported by the then Popular Coalition to Eradicate Hunger and Poverty. For example, the Coalition along with others supported a conference in 2000 that set up LandNET Africa which has been a regional network of some value in sharing knowledge, in drawing attention to land issues, and with links to east, west, and southern African networks. 3.25 Direct capacity support is not the only way the ILC has enhanced capacity. Many CSOs have been able to attend international and regional conferences due to the facilitation and funding of the ILC. While difficult to quantify, discussions with IGO observers suggest that such attendance appears to have had some capacity building benefit for individual attendees and CSO responses to the EE survey referred to this. 19 3.26 Poverty Impact. On the second question of whether the poor actually gained, or stand to gain, the evidence is, again, very limited. There are a few cases, not well assembled by the ILC for evaluation, where ILC support did contribute to some forms of secure land rights for a small number of households, for example in one of the India CEF grants and in Philippines. There are many other cases where the interventions are only a few years old and it would not yet be reasonable to expect any impact on land rights for the poor. In some cases, there is reasonable expectation from some intermediate outputs that a number of poor should benefit in due course. 3.27 Support for Production Services. On the third question, the present review found little indication of any significant support in relation to production services, small CEF grants did offer some localized technical or marketing services such as the agricultural technical support in India and the marketing support in South Africa. No doubt some other grant- supported activities which were not reviewed also provided some support. But this aspect of the objectives was given the least attention. Facilitation of Dialogue 3.28 The second objective calls for answering the question: To what extent was dialogue with government decision-makers facilitated, either through advocacy or through the various elements of the six main ILC programs. The EE found that many members and partners perceived that the ILC had been particularly successful in achieving this objective, including the appreciation of ILC-sponsored dialogue between partners with opposing views and the opportunity to attend international events. Some noted that the ILC had been able to maintain its reputation as a neutral player. Examples cited include: In Guatemala, where the ILC had been partly instrumental in facilitating the government and multi-stakeholder dialogue In Nicaragua where the ILC brought together NGOs for a regional workshop resulting in the formation of a new network (Grupo Tierra) In Zambia, where support for the Zambia Land Alliance dialogue succeeded in delaying a hasty land reform bill with a number of weaknesses to allow further consideration and evidence More broadly in Africa where, in the absence of any regional facilitating network, the ILC has provided a networking function, but not the only one, through conferences, exchange visits, and shared information. 3.29 There was some criticism noted in the EE, primarily coming from IGOs, that the ILC had not sufficiently reached out to all key players, including some social movements, when facilitating dialogue. The new strategy proposes greater effort in this direction. On this second objective, IEG's consultation with CSOs carried out as a part of the Land Study found somewhat less satisfaction from members than with the capacity building objective, but a majority still rated it effective. Some comments were positive such as "ILC actions ... contributed to the opening of the dialogue with civil society where government's awareness has just been raised. This ... translated into a five-year plan." Other comments were less positive such as the ILC "seems to deal only with [xxx] cluster of NGOs ... it has to try to understand the complex web of people's organizations and NGOs dealing with agrarian reform." Elsewhere, the present review has quoted another view that ILC is too much of "an exclusive club." 20 Attribution 3.30 On the input side, the direct grant resources for the CEF program were clearly disbursed. But the impact across the range of different cases is unclear. Some aspects of attribution in individual cases present uncertainties. In some cases, such as the policy dialogues in Guatemala, Zambia, and Indonesia (a significant National Forum in preparation of a joint position for the ICARRD Conference), there seems to be some qualitative or anecdotal evidence that the ILC played a significant direct or indirect role. However, some IGO staff interviewed questioned the share of the overall outcome in such cases that could validly be attributed to the ILC. They argued that a number of other local players, often operational well before ILC's involvement and long after it, played the dominant roles. They noted that simply showing that organizations who were members of the ILC made important contributions does not necessarily demonstrate ILC network attribution. Reports of some ILC 3.31 One piece of evidence, which the present review meetings give little indication examined and which raises some questions about the real of what the coalition plans to extent of the impact attributable to the ILC, lies in the texts of actually do. These reiterate the resolutions arising from regional ILC meetings. The most the vision but say little about recent one is the draft resolution document from the Uganda action. Others focus more on meeting on October 30 to November 2, 2007. This is coalition internal processes. disappointing for an outsider looking to understand what the Meetings should lead to ILC plans to do in Africa on land issues. This lists seven actionable and monitorable challenges that largely reflect the various levels of mission decisions towards achieving objectives. statements that the ILC produced some years ago -- such things as the need for a lobbying platform, the need for resources, and the need for coordination among CSOs. It then lists thirteen priority areas for collaboration, the majority of which are again simply restatements of the ILC mission such as the need to build alliances, the need to improve access of women to land, the need to defend the rights of indigenous people, and the need to strengthen global partnerships. Such statements may have been useful back in 1996 at the formative stages, but they provide little incremental value for 2007. The same document then has a section headed Decentralization with somewhat more substance. But this is an ILC reform process section, it does not signal priority activities towards the ILC's objectives. In other words it is inward looking, suggesting how the ILC will handle the shift towards regionalization, such as who would be the members of the steering group, who would be the nodal CSO, what are the overall tasks of the alliance, and the need for collaboration with farmer organizations. While this is not without value to guide the change process, it still says nothing about planned regional action on land policy or land administration. It says nothing about priority countries or issues. It says nothing about potential ILC members' input into particular land legislation that might be moving towards finalization. It says nothing about the source of resources. It says nothing about where windows of opportunity might be opening up at country level. It says nothing about areas for further study. From a knowledge perspective, there was more substance in the papers presented by individual participants at this meeting. But in terms of making incremental gains towards collective action in the direction of stated goals, this draft resolution was a disappointing document for the eleventh year of a Coalition's global work. The frustration of IGO staff is understandable. As noted earlier, governments are increasingly looking for support in how to implement reforms and are less persuaded by the repetition of general principles. The EE findings on some elements of reporting are in the same vein. For 21 instance, page 65 of the EE found that "ILC reports to IFAD and to members are mostly narrative discussions about process and outputs of individual activities, not about outcomes. Reports would be ... more meaningful if they provided ... planned and actual achievements." 3.32 Similar comments, although less strong, could be made about Coalition Council Resolutions. For example, the tenth session in 2007 was more about ILC structure and processes than strategy and priority programs aimed at furthering ILC's agreed objectives. The Summary of Decisions document of the Council has some important and necessary resolutions about the legal status of the ILC. It points to ways to transform the ILC into a more member-led Coalition. It endorsed the proposed draft business plan and a new communications strategy. It selected an option for a land indicators approach. It gave guidance on future M&E design. It agreed that members should share administrative costs. And it agreed upon a selection committee for selecting a new Director. (The current Director is leaving in August 2008). These decisions were all needed. But the outside evaluator still searches in vain for where the CC sees its strategic priorities on land for support by region or by country or by issue, where the windows of opportunity might lie, and how progress might best be made towards these opportunities. If these program-related questions are now to be determined separately by region, then the regional documents do not appear to be meeting this need either, as shown in the Africa case. Factors Influencing Observed Outcomes 3.33 A central question here is the influence of the IFAD relationship. The EE did a very extensive review of this, in particular in their Exhibit 6.1 that analyzed the degree of ILC's dependence on IFAD. Of particular significance is the question as to why the ILC Secretariat should "take responsibilities ... for certain management activities within IFAD and to represent IFAD as may be requested by the Assistant President." Briefly, the present review argues that a solid umbrella organization was essential through the ILC's early years of uncertain financial support and even uncertainty about legitimacy. Moreover, given the high share of administrative resources provided by IFAD, the close links were almost certainly a net benefit to the ILC. The question now is the extent to which those links should be severed or re-tied in a different configuration. This issue of the ILC­IFAD relationship is taken up more fully in the Governance section below. Linkages to Country Operations 3.34 In interviews with Bank staff, there has been disappointment at the extent to which the ILC has been able to provide capacity building or facilitation platforms that would support World Bank lending. There is probably a degree of unreasonable expectation here -- that a small Secretariat could provide a technical pool in the way that FAO does -- but it is also the case that Bank staff do not appear to feel that much has been gained from the ILC for project work in any areas, even in facilitation and knowledge generation. PERFORMANCE OF INDIVIDUAL PROGRAM COMPONENTS 3.35 The paragraphs above have focused predominantly on efficacy in the achievement of the stated objectives. At the risk of some repetition, it is also useful to slice the cake in another direction and to assess the performance of the six component programs, each contributing towards one or more of the strategic thrusts. In the following paragraphs, some of these programs are reviewed together as a package. 22 Knowledge Program (KP) and Network Support Program (NSP) 3.36 Unfortunately, the limited evidence on knowledge interventions is not well assembled by the ILC. Annex C attempts to assemble some of the main points but this is tabulated by objective not by program. The EE suggests that these two programs have contributed to shaping ILC's public visibility. They have carried out research, conferences, meetings, consultations, training and publications. The EE survey also suggests that CSO members use the documentation more than IGO members. Based on our interviews, it appears that IGO staff tend to look for more outside-ILC sources such as Oxfam, the International Federation of Surveyors (FIG) (whose site has published some Bank papers), ILC reports on grant FAO, some CGIAR sources, and the internet more broadly. Our support cases do not own review of the ILC's Web site and knowledge-related reports offer enough evidence found some useful material such as the work on common property about outcome and the in pastoral areas. However, overall the ILC Web site does not attributable links to appear to particularly stand out from the others. inputs. 3.37 A recent ILC paper entitled Sharing Knowledge and 10 Building Networks presents the results of programs supported with supplementary funds provided through IFAD and the Government of Italy. Twenty-two grant-supported activities are reported, each given about two pages in length. Again, this is a frustrating document for an evaluator for the following reasons: It provides no costs or evidence of resource use, although total costs but with no breakdown can be found by searching in other documents. In many cases it is not clear what the outcome was; what is written under the heading "Results" is often something quite different. It is not always clear what has been actually completed and what is underway or even still just a proposal. In cases where there are some defined outcomes, it is not clear what the attribution might be to the (unreported) resources applied. The language is often very vague such as "took action to establish a forum" "establishment of a network." No clear objectives are stated. The heading "Content", which was presumably meant to show what the content of the project was in input terms, in many cases, is simply some background description. While some project reports do indicate how the generated knowledge was used, others omit this section; perhaps these are the ones not completed, it is difficult to tell. 3.38 It would be fair to note that some of the outputs reported in this paper appear to be relevant and useful, for example, the "Reform of the Sugarlands" paper for the Philippines and some of the national consultations. However, in no cases was it possible to assess attribution, since no resources are given along with the outline of the activities. For future credibility with donors, it will be essential for such reports to indicate objectives, resource use, and evidence of attributable outcomes, even if achievement is still in the form of outputs rather than outcomes. 10. www.landcoalition.org/pdf/nsp kp sum05.pdf 23 3.39 In addition to the above there have been a number of useful training programs that appear to have enhanced knowledge and network support, some of which have been discussed earlier or feature in Annex C. Community Empowerment Facility (CEF) 3.40 The EE noted that, based on its surveys, many stakeholders saw the CEF as the flagship of the ILC's contribution. One would expect this since this is where stakeholders have received the most direct funding benefits. The EE noted that "there is wide agreement that CEF has achieved remarkable results with comparatively few resources." At the time of the EE, the CEF had funded 41 projects out of 239 applications since 2000. The total number of completed projects is now 53. An independent evaluation of the CEF is currently underway by a consultancy firm to be completed later in 2008. Hopefully the final report will be able to classify and assemble evidence on outcomes, even if much will have to remain largely qualitative and anecdotal due to limited case evidence. 3.41 Some EE survey respondents reported some reservations about the CEF, including limited ILC capacity and in-country resources, limited assistance in implementation as opposed to planning, and particularly, the short timeframes of support. Most evaluations of community-driven development interventions globally, such as IEG's 2005 evaluation of Community-Based and -Driven Development, have shown the need for sustained support for more than one donor project cycle. Indeed, an IEG survey associated with the above quoted report suggested that most Bank staff feel that support at community level is needed for at least ten years to reach some level of sustainability. While this does not necessarily imply that ILC support for CSOs, who then support community initiatives, should be this lengthy, the EE findings are consistent with a general concern about "staying the course" to the point of some degree of (local) institutional sustainability. It suggests the need for better definition of sub-project exit strategies. 3.42 The IEG mission's informal review of a very small sample found quite good logical frameworks in some proposals, suggesting that at least some CSOs had a good focus on measurable outcome. Unfortunately, those outcome indicators were often not picked up again in the same CSOs' closing reports. This did not seem to be demanded by the Secretariat review process. Across about six of the cases that the IEG mission reviewed, there was a mixture of promising if small and short-term initiatives (the better ones included leadership training in Indonesia and support for the Nepal Community Self-Reliance Center). But other cases (such as the support in Andhra Pradesh, India), while not without value, seemed predominantly agricultural interventions of low land policy or administration priority. Another case cited earlier, the South Africa SAFIRE support, was, strangely, support for trade in natural products with no obvious close land policy connection. The APLR program in Georgia, the Zambia Land Alliance program, the FENACOOP in Nicaragua, and FSRDA in South Africa are four cases cited by the EE where CEF initiatives are reported to have contributed well to capacity building related to land (see also Annex C).11 However, notwithstanding a number of CEF grants that appear to have achieved some measure of what 11. A CEF grant for Guyana indigenous peoples seems to have come to nothing when an aged but indispensable overseas anthropological consultant fell ill and was unable to fly to attend court. This was a seemingly high risk intervention. However, it was arguably worth the gamble since it did seem to lie closer to the critical path to land policy advancement (through legal precedent) than many others. 24 they set out to do, many do not appear to be on any obvious critical path to significant advances in land policy or administration. Moreover, many could easily have been supported by any one of a number of community development funds. For a small alliance with a focus on land policy, and claiming comparative advantage in that area, the CEF seems to have been rather unfocused and therefore probably not very efficient. Women's Resource Access Program (WRAP) 3.43 This is a smaller and newer program than the others. The EE reported thirteen community workshops, covering five countries (Nepal, India, Indonesia, Cambodia, and Kenya). Funding was from the Japanese Government to IFAD under a Women in Development Trust Fund. The present review found the documents produced professional and useful as learning material. It is too early to expect outcome evidence, but there appears to have been some useful training on a small scale. But again, one searches in vain for evidence of a significant direct line of logic to the advancement of gender-relevant national land policy or administration in order to differentiate this support noticeably from a host of other women's empowerment programs supported by others. Land Alliances for National Development (LAND) 3.44 This program, started in 2003, has worked mainly in Guatemala, the Philippines, Indonesia and South Africa. The aim has been to establish dialogue with important stakeholders. Briefly, the objectives were to develop or strengthen mechanisms for dialogue on land, to improve policy formulation through informed dialogue, to support joint action on user rights and land access, to exchange knowledge and to monitor land policy processes. The main results to date have been in Indonesia and the Philippines. In the former, the ILC hosted a forum prior to the ICAARD 2006 Conference which helped government and CSOs develop positions on land prior to the conference. This appears to have been a well-seized window of opportunity that represented a significant step forward in the consolidation of thinking by the Government of Indonesia (GOI) on land reform. Such conferences sometimes do provide the incentive for useful steps in the development of national policy positions. In the Philippines, the ILC held meetings with both CSOs and the government to open up debate about land conflict, including conflict between farmers and indigenous people. This appears to have had some value, but the extent to which the advances can be attributed to the ILC as opposed to other players is difficult to assess. In South Africa there was less progress attributable to the ILC, since the government decided to organize its own consultations and some CSOs could not agree on the market element of the approach. The EE has an important recommendation on the LAND program with which IEG fully concurs -- that the LAND program should become the flagship program into which the other more supportive programs should feed. Clearly, the elements of the LAND program lie at the very heart of ILC's mandate. Toward a Common Platform on Access to Land (PLATFORM) 3.45 This program was not sustained. There was a stakeholder consultation process to reach some common positions on land and a document was produced. The EE does not give it much attention other than to suggest that the document should simply be treated as a background document and the program dropped as a named ILC component program. Nevertheless, there appears to be some evaluative evidence from this experience. If advocacy is to remain an objective, then presumably coming to common positions, i.e., a common 25 platform, or agreeing to disagree and formally recording this, would be of value. It is somewhat unclear how much this particular dead-end reflects lack of ability to come to common positions (therefore raising questions about the role of advocacy), or how much it came to a halt simply due to lack of resources and interest. Either way, it is not without significance that a coalition with a claimed role in advocacy was not able to sustain such a component. Collaborative Action on Land Issues (CALI) 3.46 The Collaborative Action on Land Issues program (CALI), a joint IFAD/Belgian Survival Fund/ILC initiative carried out action research. For example, in 2005, with support through the Uganda Land Alliance and the Ministry of Lands, CALI encouraged wide input into the formulation of the new land policy. In Niger, in 2007, CALI launched support for The Promotion of Local Initiative for Development in Aguie. CALI is piloting methodologies at village level to help small farmers obtain land tenure security. Efficiency 3.47 There is very little that can be said about ILC efficiency due to lack of evidence. Efficiency is assessed to the extent possible against four main questions: (a) quantified economic analysis to the extent available, whether this be economic rates of return or comparative total or partial cost-effectiveness analysis against "industry standards"; (b) evidence of efficiency when measured against any available formal or informal business plan, i.e., efficiency against own targets; (c) evidence of any attributable incremental costs or benefits to stakeholders including borrowers or donors at country level; and (d) the presence of any obvious signs of wastage or inefficiency. There is very little evidence in any of these categories. ANALYTICAL EVIDENCE 3.48 Inputs. As shown in Table 1, ILC's resource mobilization from 1999 to 2006 fluctuated between a low of about US$0.5 million in 2000 to a high of about US$3.5 million in 2006. However, there has been an impressive inflow in 2007, when four different donors collectively committed about US$7.0 million. This has brought IFAD's contribution down to about 24 percent of the total in 2007, well below the 35 percent target set by IFAD to reduce its share. This represents a huge increase of about five times the average commitment between 1999 and 2006 and will impose a substantial incremental management burden on the ILC. This burden would be even greater if, as suggested by some, the ILC were to shift towards being less dependent on IFAD management systems. 3.49 Administrative Costs. The EE found that administrative costs were high at 65 percent of total expenditures for 2004 and 2005. But this seems Administrative costs are a largely due to undifferentiated assignment of staff time costs high percentage of total costs (Figure 1). Staff time spent on direct support to program at 65%, but this is not activities has not been allocated to those programs and better, indicative of program more program-differentiated data is not available, as the EE inefficiency, merely noted. Better recording of resource allocation in the ILC is inaccurate cost assignment. clearly needed. The Secretariat is aware of this and says it is working with IFAD to achieve better cost assignment by program activity. 26 Figure 1. International Land Coalition, Operating Expenses, 2005­2006 100% 35% 36% 40% 80% Grants 60% 17% Consultants for 24% 11% Operations 12% General Expenses 14% 40% 10% for Operations Staff 33% 37% 20% 30% 0% 2004 2005 2006 Source: ILC. See also Annex E. 3.50 The EE made no attempt to calculate an economic or financial rate of return or comparative cost-effectiveness. This would have been extremely difficult to do, given the weak M&E data, and assessing attribution would also have challenging. What might have been possible would have been some sample indicative analyses of a cross section of representative small grants. One thing that is clear is that the ILC has not become a financially sustainable organization. But this is true of most global public programs and part of the rationale of the outside donor support is precisely the global public good element. We pick this up again in the sustainability section. EFFICIENCY IN RELATION TO A BUSINESS PLAN 3.51 It has not been possible to make an assessment of efficiency measured against a business plan. There has been no business plan offering sufficient detail linked to program content. With the emergence of the proposed indicators in the new ILC documents this may be possible in future. But it will still depend on the willingness and ability of members and partners at the regional level to provide evidence on their program inputs, outputs and outcomes. INCREMENTAL COSTS OR BENEFITS TO STAKEHOLDERS 3.52 Again, there is insufficient evidence with respect to incremental costs or benefits to stakeholders. The extent of documentation by potential and actual CSO recipients of small grants was substantial -- thick ring binders for each one approved, but less voluminous for those rejected. Because of the small size of the grants, this is likely to have been a significant burden, but apparently seen as worth it by many CSOs. However, based on experience, the management burden with grants at all levels appeared to be no more onerous than many other such small grant programs observed. 27 3.53 A cost that should not be ignored in assessing efficiency is the aggregate staff time and travel cost of all members and partners to attend meetings and carry out other forms of interaction. The Assembly and Council meetings use significant resources. 3.54 With respect to human resource use efficiency, the difficulty of offering long contracts or regular staff status in some cases due to uncertainty of funding has probably been somewhat negative for efficiency of human resource use at the Secretariat level. OBVIOUS INEFFICIENCIES 3.55 Clearly the servicing of the small-grants program mentioned above is an area for potential efficiency gains by focusing more directly on the core ILC objectives. The proposed greater decentralization may enable some cost reduction by the Secretariat, thereby offering better leverage from limited resources through the application of the principle of subsidiarity. However, this involves passing costs out to regional CSOs with limited resources. It is difficult to forecast how cost-effectiveness will change in the next few years with this regionalization. It will be important to collect M&E evidence from the regional level to assess this. 3.56 Although not quantifiable, the mere length of the program period from the earlier coalition in 1996 to the present time suggests the likelihood of a degree of inefficiency in overall resource application, in which the earlier start-up costs yielded only gradual and modest benefits stretched out over a long period. The long delay in the use of the $1.5 million of World Bank funds, and the opportunity cost of those funds, is an element in this program-length efficiency effect. But we can go no further than qualitative statements about this. 3.57 From the above discussion, it is concluded that efficiency is very difficult to assess but that, over the first ten years, efficiency of resource use is at best questionable. If this were being assessed as a Bank project it would have been given a low rating on efficiency simply on the grounds of lack of evidence. Sustainability of the Program and Prospects for the Future 3.58 Sustainability and risk is assessed against four main questions: (a) financial sustainability; (b) the risks of not achieving the intended objectives; (c) the risks to sustaining the Coalition program itself; and (d) questions related to whether the Coalition should, in fact, be sustained. 3.59 The ILC's most recent statement of strategic objectives has, as item (5), to achieve financial sustainability, but it is not entirely clear how financial sustainability is defined. There are two sides to the financial sustainability coin for a global public program. Clearly contributions by CSO members would be desirable as an indicator of commitment and strong financial donor support would indicate, as it seems to, commitment to the issue. However, the reason some IGOs support the ILC is because they see at least the knowledge aspect as a public resource worth subsidizing because, without it, beneficiaries and investors would underfund. If members were ever able to fully support the ILC, highly unlikely at present, it might suggest either that the public good element was small or that CSO members themselves had in mind contributions towards global or regional public benefits. 28 3.60 With respect to the risk of not achieving objectives, while not so great for the process elements of the objectives such as putting in place facilitation and dialogue space, the risk appears still quite large for reaching beyond that to effective and efficient impact on the access of poor men and women to secure land assets. 3.61 With respect to the risks of failure to sustain the Coalition program, although the major CEF grant program, supported partly by the Bank, is now fully committed, the recent surge in donor contributions suggests an increased interest in financially sustaining the program for the coming two to three years. However, until the recent surge in funds, the ILC had struggled to maintain a sufficient and steady flow of funds. Currently it appears that some potential donors are sitting on the fence waiting to see what happens so it may be in three to four years time that the uncertainty on sustaining resources could arise depending on performance and response. 3.62 With respect to the future, the change process in train gives some hope for greater sustainability. It is proactive, promising and worth monitoring by members and donors. But unless M&E can be built up quickly to better show progress along the logical chain, and unless there is more focus on land policy and administration action, donors will lose interest and will be justified in losing interest. 4. Governance and Management 4.1 Governance and management is assessed here against two main questions: (a) adherence to generally accepted standards of public sector governance such as legitimacy, efficiency, accountability, fairness, transparency, and financial management; and (b) the relationship between the ILC and its host organization, IFAD. Adherence to Public Sector Governance Standards 4.2 Legitimacy. As noted by the EE, membership had been quite narrow up to the time of the evaluation. Comments to the effect that the ILC was like an exclusive club came to IEG in the consultation with members and others with Legitimacy can only knowledge of the ILC. However, membership has approximately come from the doubled since then. While almost certainly desirable there are membership and the potential issues with a broadened membership which may pull in quality of knowledge and facilitation. Legitimacy members with more advocacy and political focus than analysis drawn from a host focus.12 This issue has implications for the question, discussed organization is a false below, of openness of membership category and whether legitimacy. governments and bilateral donors should be now considered for membership. 4.3 The often cited legitimacy provided by the umbrella of a UN associated agency may be a perception with some governments and others, but it is a false legitimacy since IFAD does not control ILC decisions. In the end, legitimacy can only come from the quality and 12. A reading of some CSO web sites showing little analytical content or reference behind extremes of strategic advocacy suggests that such concerns are valid. 29 proactivity of its membership, from demonstrating positive results, and from the Coalition's reputation for collective knowledge, facilitation, and capacity building. 4.4 Efficiency in Governance. The efficiency of service provision to the limited number of stakeholders appears to have been sound. Based on the EE surveys, most CSOs are satisfied with ILC Secretariat efficiency although it is questionable whether respondents had a common view on what efficiency would look like. With respect to the efficiency of coalition collective decision-making, the efficiency problem of reaching agreement across a wide range of members is difficult to assess quantitatively but there is some evidence from comments and meeting documentation that the Secretariat had a fairly dominant role. However, it should also be noted that most members went along with this and do not appear to have pushed for changes. With decentralization the efficiency of the respective decentralized alliances will inevitably have a bearing on the efficiency of the whole alliance. 4.5 Accountability. The accountability of the various parties does not appear to have been very clearly defined at the outset, particularly the accountability of members. Overall, accountability up and down the chain of command within the organization seems to have exhibited some weaknesses. Some stakeholders have felt that the ILC has been excessively centralized and to have had a rather paternalistic Secretariat. The Secretariat is aware that this has to change, as reflected in the new membership guidelines and the proposed shift of responsibility to the regional level. This will pose new and different accountability issues that will call for agreement and guidance. 4.6 Fairness. The CEF is the most obvious place to look at the issue of fairness. There is no evidence of any bias in funding allocation or that certain recipients were privileged. There was a reasonable balance across regions, and perhaps too much spread across the range of activities. The application review process, while as noted lacking in rigor on some aspects, does not appear to hold inherent bias. Applicants were given the chance to come back with further information when requested. The criteria were available to applicants and appear to have been applied consistently. 4.7 Transparency. The ILC has made material widely available and meetings have been open to all members. However, as noted, the Secretariat has tended to dominate until recently and the confusion among some members and partners about what the ILC's role should be may be indicative of some lack of internal transparency that needs to be addressed. As noted, there have also been assertions that ILC membership has been too exclusive. 4.8 Financial Management. There appears to have been no irregularity in this regard. The financial record keeping and flow of funds has been through IFAD systems which are audited annually. Relationship with the Host Organization, IFAD 4.9 ILC's position in relation to IFAD has been a matter of some contention within IFAD. A Memorandum of Understanding between IFAD and ILC is currently under discussion to govern their relationship going forward. There have been a range of views about what ILC is and what it should become. Some IFAD staff interviewed felt quite strongly that it should be "spun off" to operate less under IFAD's umbrella. However, views 30 on this issue often depend on the precise parameters in any postulated relationships. For a small network such as ILC still some way from reaching maturity, there would be risks associated with a significant adjustment in hosting arrangements, as well as significant cost implications. On-going discussions are aimed at reducing and clarifying IFAD's role. Up to now the ILC Director has been recruited by IFAD in consultation with the Coalition Council and has reported to IFAD's President. The President has been approving the annual work plan and budget albeit upon recommendations of the Coalition Council. IFAD has taken the lead, but along with the Council, in the performance review of the ILC Secretariat Director. There are also less substantive perception issues. For example, all ILC Secretariat staff have IFAD e-mail addresses. This has sometimes confused correspondents but is fairly standard practice for hosted global programs. 4.10 The EE made a number of specific recommendations calling for the ILC to resolve its status overall and its status in relation to IFAD. While the resolution of some outstanding relationship elements is not unimportant, the close relationship with IFAD was beneficial for a new and small global program in the earlier years. Moreover, the present review argues that some relationship with a large and stable host of standing will continue to be needed. Approximately 70 percent of Bank-supported global programs are hosted by either the World Bank or other large international organizations. 4.11 It is not the place of this review to recommend a particular organizational structure or relationship and much will have already been decided by the time this report is issued. However, based on the EE findings, mission discussions and document review, a recent ILC legal review, and evidence from some of the other global public programs, we outline some of the issues with respect to two of the main options that have been on the table. ILC WITH A SIMILAR RELATIONSHIP TO IFAD AS IN THE PAST BUT WITH GREATER DEFINITION OF ROLES 4.12 The advantages of such an arrangement for the Secretariat include: somewhat easier recruitment; depending on the arrangement, reduced overheads with the sharing of some of IFAD's facilities; and, although difficult to define, some greater degree of overall stability. However, as noted earlier, the purported legitimacy derived from a close link to a UN- associated agency such as IFAD is invalid. 4.13 The disadvantages for the Secretariat are few. These include the risk of excessive management interference from IFAD; the belief by potential donors that ILC is "IFAD's baby" with, perhaps, some reduced motivation to fund; and, perhaps in a few cases, potential members who would join if the ILC was totally de-linked from any international body. Any reduced motivation to fund would seem to be belied by the recent surge in funding. 4.14 The advantages for ILC members and partners include again, some degree of stability and, perhaps, reduced burden of management responsibility on members. However there is the risk of excessive paternalism. The disadvantages for ILC members and partners are essentially the obverse of these points. However, paternalism or the perception of paternalism would be much diluted by the development of a widely recognized brand name. The ILC has not been able to do this so far. By contrast, other global programs -- such as the CGIAR, CGAP, and the Cities Alliance -- have successfully developed a brand name even though they are located in an IGO, in this case the World Bank. 31 4.15 The advantages for IFAD include arguably a measure of control over ILC and particularly the resident Secretariat, but the control is not something IFAD appears to want; and the ability to more easily tap the ILC Secretariat skills. The disadvantages for IFAD include a larger management burden; a greater perceived implicit commitment for IFAD funding possibly exploited by other donors free-riding; the risk of ILC speaking for IFAD; and, arguably, some perceived constraints in the freedom for IFAD to make wholly unbiased input in Council meetings. ILC FLOATED OFF AS A LEGALLY INDEPENDENT INTERNATIONAL NGO 4.16 The advantages and disadvantages for the various players are largely the obverse of the previous points. Briefly, the task of the Secretariat in recruitment and management outside the institutional shelter of IFAD would be greater. Recruiting quality staff would be somewhat more difficult. As has been found by other global programs, recruitment logistics would be more challenging e.g. staff visas etc. Probably there would be reduced stability but there may also be some advantages for the ILC to have to face the buffeting of independent survival. The reduced management intervention by IFAD would need to be compensated for by greater management input from members and the Council. Funding would probably have less of a stable core from the parent institution, although a more IFAD-linked arrangement does not necessarily imply any funding commitment. 4.17 This review concludes that the link to IFAD in the earlier years was extremely important in sustaining the ILC and that there are likely to be continuing advantages. Being able to recruit and keep highly qualified staff is important and is enhanced by a link to some IGO. For other donors, the perceived stability and associated financial channels provided by a link to IFAD or some other IGO is probably, on balance, positive, although so far many donors appear to have happily been free-riders on IFAD's support. ILC's Future Role 4.18 Looking to the future, this issue revolves largely around the tension between knowledge and advocacy. Two extremes can be envisaged with a range of intermediate positions. At one ILC members should consider extreme the ILC could simply be a forum to exchange reassessing the nexus of advocacy well-analyzed experiences. In addition, if the ILC also and membership. Advocacy of promoted the higher-level principles implicitly accepted principle is becoming less useful by members at the outset, few members would take than advocacy of (evidence-based) exception since such broad advocacy is consistent with practice. This may be better done with all players inside the fence. the strategies of most players on land issues. But, when Governments will increasingly going beyond this to the more specifically action-oriented have knowledge to offer. level, advocacy becomes more problematic. 4.19 ILC members and partners by no means agree on how most effectively and efficiently to reach the broader goals through land policy and land administration design under particular conditions.13 For example, some argue that one member, the World Bank, has 13. The present review does not have the space to go into an intermediate level of advocacy which could be defined as the increasingly well established lessons of broad relevance such as: start with what systems exist; there is rarely a single solution; simple solutions have rarely worked; the policy reform must be owned at many levels; 32 been, and perhaps still is, too focused on market-based reform. Others argue for more or less land redistribution or for associated policies to land reform that represent total trade protectionism. There is a question whether a coalition of broad and broadening membership can collectively advocate any solutions, below those of the broad principles noted above, without losing wide membership support. With the shift to regionalization, the extent of ILC's advocacy role warrants re-examination. If the aim is purely knowledge generation and dissemination, then presumably there would need to be membership that spanned all main players including governments, bilateral donors, and even the private sector. Without this, one or more sets of contributors to knowledge would be excluded. 4.20 This issue was discussed at the Uganda Assembly meeting in April 2007, where there was a working group to discuss membership. The eligibility of governments to become members was voted on by secret ballot and rejected by the Assembly. The two main arguments made were (in favor) "to build government commitment through active membership" and (against) "but the goal stated in the Strategic Framework indicates that governments are the primary targets." As suggested by this last point, the reason why governments are excluded at present is predominantly because of ILC's advocacy role -- with a collective advocacy role there must be at least one outsider to advocate to. While respecting that this was a membership decision, the present review suggests that it would be worth collectively revisiting the logical elements of these arguments during the process of developing new directions for the ILC. The members could re-examine the extent to which there really are net benefits in having internal network dialogue which omits the key player of governments who are gaining increasing experience in land reform, and which omits the other key player of bilateral donors who often have strong analytical skills and cross-country experience. Governments might be much more likely to have ownership of decisions that arise from findings and recommendations emerging from programs in which they are members.14 In addition, they provide at least one aspect of the reality check. 4.21 The complexity of objectives noted in the 2007­2011 Strategic Framework does not contribute much to clarifying the role of ILC. The mission statement refers to advocacy, dialogue and capacity building. The core values talks of coalitions and partnerships as conduits for policy advocacy. The goal statement talks of capacity enhancement, opportunity enhancement for dialogue and "influence", presumably a less conspicuous term than "advocacy". Lower in the hierarchy, the five Strategic Objectives again use the term "influence". They talk of coordination on support for national policy action, greater influence over policy, sharing lessons, capacity for networking and joint action and a fifth objective that defines how the ILC would change towards a more member-led, decentralized, coalition. This mix now warrants simplification, focus and prioritization with translation down to, and up from, regional level directed at action. But this is not simply better presentation of a set of agreed aims. It will call for the membership to work towards consensus on the role. what works in one country may not work in another; access to legal support is important for the poor; working towards the right process comes before developing the particular options; codification should be delinked from collateralization; good governance usually has to be tackled at many levels; land inventories are useful for many purposes; there is often high risk in land reform for socially excluded groups; etc. (Palmer 2007). 14. An internal Bank reviewer has commented on an earlier draft of this review that another route to achieving the ILC's objectives without having governments as members could be (a) by truly embedding the ownership and implementation of the program in civil society and (b) by joining forces and building effective coalitions with other advocacy and capacity-building coalitions while each focuses on its areas of comparative advantage. 33 5. The World Bank and the ILC Partnership 5.1 The World Bank was a founding member and contributed a total of US$1.5 million. (US$1.0 million in June 1998 and US$0.5 million in December 1998). The contribution was to IFAD, termed in the letters exchanged the "focal institution", for passing on to ILC since the ILC was not its own legal entity. This was at the time when the ILC was still the Popular Coalition to Eradicate Hunger and Poverty and the grant, under the Bank's FY 1998 and FY 1999 Development Grant Facility, was for the CEF component which provided funds for capacity building and empowerment of CSOs. Originally, as set out in the terms and conditions letters, these were to be contributions to an endowment fund and the ILC was to receive only the income. When not enough funding came forward from other donors to establish a viable endowment fund, it was agreed by the Bank that the funds could be drawn down as a grant in the normal way, a less sustainable arrangement. However, since the ILC also had a grant from IFAD for the same purpose they chose to draw down IFAD's funds first and then the Bank's. IFAD were perceived as more likely to fund further once fully disbursed. This resulted in the Bank funds sitting unused (although invested) for some years and they were disbursed mainly between 2005 and 2008. As a result of this delay this fund is not yet closed 10 years after commitment. 5.2 The World Bank's relationship with the ILC is examined at five levels: (a) the Bank's contributions at the global/program level; (b) its contributions at the country/activity level; (c) the efficacy and efficiency of Bank oversight; (d) managing the reputational risks to the Bank; and (e) the formulation and implementation of a disengagement strategy, as appropriate. 5.3 The Bank's Contribution at the The Bank has made important Global/Program Level. In this program the Bank contributions in knowledge and appears to have been a somewhat passive partner in the in emphasizing sound analysis, area of coalition guidance. However, strong skills were but it has been weak in terms of made available for meetings which non-Bank observers oversight and proactivity. It is have noted were useful for knowledge enhancement, not clear what expectations the global experience and particularly analytical grounding of Bank had in these respects at the land policy and administration evidence. However, it outset. should be noted that the Bank has also engaged with a number of other partners in the land area, including the Global Land Tool Network (GLTN), the International Association of Surveyors (FIG), the African Development Bank, the Organization of American States, LandNet Americas, the Permanent Committee on GIS Infrastructure for Asia and Pacific (PC-GIAP), and a number of others. The present review did not have the resources to assess the relative impact on Bank and country goals of engaging with these various partners, nor the efficiency of such engagements, but this would be a valid resource efficiency question. 5.4 Land policy is an increasing area of focus in the Bank's rural and urban programs. With the Bank's large presence in rural strategy development in many borrowing countries, with its focus on poverty, and with its considerable analytical skills, the Bank has a considerable comparative advantage in the area of land. However, the Bank has not seen the ILC as the only option for global advancement in the land field. Indeed, the Bank's Vision to Action rural strategy document had actually visualized a network "with FAO and IFAD on 34 land reform, which [would] facilitate the exchange of experience among countries that are introducing negotiated or market-assisted processes for land reform." It is not clear whether, in this instance, the ILC was being seen as a part of IFAD. 5.5 The Bank's Contributions at the Country/Activity Level. Linkages between the ILC and the Bank's country operations seem to have been quite weak. But, as in other IGOs, Bank staff have pointed out that the lack of linkage to projects is largely due to lack of skills offered by the ILC and the relatively few countries in which the ILC has both strong partners and a sustained presence. This review has not had the resources to check in the field as to whether such claims are valid but they seem consistent with what other IGO staff report. Globally, there appears to be a much greater link on land project design between FAO and the Bank than between ILC and the Bank. Given the larger scale of the FAO land group in terms of staff numbers and budget compared to the ILC this is not surprising. 5.6 In assessing the technical input to land issues it is important to distinguish the ILC Secretariat from the membership with respect to both responsibilities and potential. The Secretariat is not tasked by the Strategic Framework of ILC to give technical advice and the membership is the main source of skills and knowledge. This would seem to be a loss to the global knowledge base. Operating across many countries and with many players the Secretariat has a unique opportunity to gain and spread global experience. While members are also potentially useful for technical assistance, they often do not have the global spread of experience that a Secretariat staffed with skilled land specialists could have. Moreover most IGO staff perceive individual CSO members as accessible as knowledge sources anyway, with or without the ILC umbrella. Thus, as in the case of other network-type global programs such as the Consultative Group to Assist the Poor (CGAP) with respect to microfinance, many IGO staff look towards the Secretariat as a potential pool of experience on land reform and are disappointed, while also remaining open to the individual skills of members. While it is important now that Bank staff stay in touch with ILC members and the Secretariat as the ILC shifts to its regionalized approach, so far this review does not see evidence that the Bank is ignoring a significant global pool of support for land work. Another reason for the limited connection to Bank-supported projects might also be attributed to the lack of government membership in the Coalition, although this is difficult to prove. It is rare for the Bank to be involved with a network alongside such IGOs as IFAD and FAO without both governments and bilateral donors being also involved. 5.7 World Bank Oversight. There have been both strengths and weaknesses. With respect to strengths, based on interviews with observers outside the Bank, the Bank representatives have pushed hard for an improved analytical base in land policy work and they have played a strong role in sharing knowledge. While the Bank has not provided any funding since 1998, Bank staff members have been active in attending meetings. The ILC has been discussed by the Bank's Thematic Groups on many occasions. ILC staff members have invariably been invited to knowledge events at the Bank related to land policy. 5.8 However, there have also been several weaknesses in this area. First, IEG is not aware of any terms of reference for the Bank's task team leaders from either the ARD Sector Board or Department. There have simply been TORs from managers for each staff member's missions, usually combined with other work. Second, even after ten years there has been no assessment by the Bank of ILC's performance. Third, IEG found it extremely difficult to 35 trace the Bank's financial contributions for ILC within the Bank's reporting system because these were labeled as IFAD funds and the name of the Coalition did not appear. Fourth, in recent years the time allotted to ILC activities by Bank staff seems not to have been accounted for at all. Fifth, there appears to be some areas where a more proactive role as a member would have been warranted. In particular, the Bank could have pushed more persistently for better early design of M&E. Sixth, changing the financial contribution an endowment fund to a fully disbursing grant and permitting to drag on for ten years seems to be casual fiduciary management. Land issues and the Bank's strategy on land have changed a lot over the intervening decade, so that there was a case for reappraising commitments at an interval of considerably less than 10 years. It is not clear why the Bank accepted to have the fund sitting unused for so long. More generally, while it is difficult to prove, this review senses that all IGOs, including the Bank, have tended to sit back and "go along for the ride." A more proactive approach by all might have triggered some of the current change activity sooner. 5.9 Reputational Risks. These may be of some significance, although Bank staff did not themselves volunteer it as a leading concern. Staff of another IGO felt strongly that his organization faced considerable reputational risks, while staff of yet another IGO saw only some risks. The perceived risks are for ILC staff or members, explicitly or implicitly on behalf of the ILC, to take positions on land policy at a country level beyond the broad principles which members accepted. IGO staff were concerned that, in some cases, their organization's reputation might be used to get the attention of national policy-makers, but that the directions being pushed were not necessarily those that the organization would have taken. However, while one or two examples were cited, these did not seem particularly egregious, and the present review did not have the resources to follow up these cases to triangulate with other observers.15 But IGO staff and managers clearly perceive some risk, unless the boundaries of advocacy are more clearly drawn. The regionalization of the ILC, with less central control from the ILC Secretariat, may be perceived as raising this risk. It is important to note, as the ILC Secretariat has done verbally, that there are reputational risks for other stakeholders, too. For example, some members take reputational risks being in a network alongside IGOs that some of their partners on land issues do not wish to negotiate with, at least not within a network, and that they themselves have some discomfort with. 5.10 Disengagement Strategy. The Bank has had neither an engagement nor disengagement strategy for its involvement with the ILC. It might be too much to expect a detailed Bank exit plan in something so uncertain and, at the country level, as politically charged and long-term as support for a global land alliance. Nevertheless, the lack of documentation concerning the Bank's own perceived role in the ILC -- even an approximate documented outline of mileposts or a postulated exit scenario -- is symptomatic of a weakness in entry strategy. One has to define clearly why one has entered and what is expected before one cannot really say when or why or how one might exit or shift gears. In 15. A very small example, but the mission did notice in documentation that one of the ILC member CSOs supported with a grant had been involved in public demonstrations at which a government land office had been locked and shut down by demonstrators. While this may have been a pragmatic way to get attention to a valid land issue, it is plainly not something that would have been supported by any IGO members of ILC. But neither the Secretariat nor the collective of members can reasonably be held responsible for an individual member's actions although it would become a different story if such actions became the norm. 36 other words, exit strategies are a useful element in the design of entry. We suspect that if there had been an exit strategy, or at least some mileposts on a long road, the Bank would have revisited its engagement some years ago. But the present review does not advocate this now. 6. Lessons 6.1 Over the past ten years, the ILC (or its earlier incarnation) has contributed in modest ways to facilitating dialogue, generating and disseminating knowledge, and building CSO capacity on land issues. While the evidence is limited and scattered, this work appears to have made a moderate contribution on land reform or administration in some countries. M&E has been too weak and insufficiently assembled to present convincing evidence on overall outcome measured against objectives or on efficiency. Considering a decade has passed since the precursor coalition started operation, progress to date can at best be assessed as modest. There is a substantial difference in the perceptions of the two sets of major players. CSO members have been generally positive about ILC's performance. IGO staff have been more negative. The review has been cautious in interpreting CSO responses to either EE or IEG consultations since many were either potential or actual recipients of funding from the ILC and would therefore seem unlikely to respond too negatively. 6.2 The ILC is now faced with a pivotal moment that, along with the recent surge in funding, could signal a rebirth. The Secretariat and the more active members appear to be seizing this opportunity. The attitude of many observers appears to be that this may be the last chance for the ILC to show to the global donor community and to itself, i.e. its collective membership, why it should survive and grow as a coalition. While that may be, it also appears to be the last chance for the key players in the donor community: first, to decide that the issue is important enough; second, to determine that the recent changes in ILC direction are sufficiently promising; third, to help with sustained commitment of resources; and finally to pursue more proactive intervention at governance meetings to finally make and sustain the ILC as a network that has a strong global brand-name. Founders, including the World Bank, surely have a particular responsibility to contribute to strategic direction. It is therefore a pivotal moment for both the ILC itself and its founders, and actual or potential supporters. 6.3 There remains a case for donors to sustain the ILC for a further period. The land issue is growing rapidly in importance, in some countries in troubling directions. Global sharing of knowledge is therefore potentially of high value. Economic returns to land interventions, based on limited analysis in a few countries, seem generally to be quite high. However, if it is to be sustained, ILC's objectives need to be realistic, readily understood, focused, measurable, and actually measured. Skills need to be aimed at the particular niche of true comparative advantage in land reform and administration. The proposed structure of a widened regionalized network needs to maximize the input of sound analysis. If the ILC of 2012, after five years of experience with the 20070­2011 Strategic Framework, does not largely achieve at least these elements with reasonable levels of efficiency then it would be time for members, founders and donors to consider terminating the initiative. 37 6.4 The main lessons of this Global Program Review for consideration by ILC members and the Secretariat are the following: · ILC membership could usefully revisit the nexus of advocacy and membership. Advocacy as an activity can be advocacy of a vision, to which all members implicitly signed on, or advocacy of solutions, to which no members signed on. Advocacy aimed at solutions cannot be done collectively by a coalition to an outsider without a common position. Agreement on a common position will always be unattainable, especially with broadening membership. Advocacy by individual members within the group to others within the group, which would raise the demand for evidence, seems more likely to lead to advances in understanding. If members were to choose to take this route, those to be advocated to (i.e. governments), would need to be members within the club, including bilateral donors that can offer substantial analytical skills. As governments increasingly accept the broader principles on land policy but face the challenges of the practice of land administration, they will have valuable experience to contribute to the dialogue. Regardless of the outcome of any such membership debate, agreed Coalition language on the boundaries and modalities of membership and advocacy would be helpful to all parties. · ILC's new rather confusing hierarchy of vision, mission, core values and principles, core strategies, goals, and strategic objectives, although voted on by members, seem unlikely to help overcome the past perceived lack of clarity among different players about ILC's role and could benefit from some simplification. With decentralization, the objectives would now need to be reflected at the regional level and also be reflected back. The objectives of ILC's renewal and reform process itself are not conceptually parallel to the Coalition's overall objectives and would be better treated separately. · The new objective of financial sustainability would benefit from greater realism and conceptual clarification in relation to the public good element in the program's outputs. · The ILC needs to do better at using ILC meetings and conferences to agree on priorities for action and on who will do what for the next phase of action in priority countries and programs. The reports of such meetings offer little insight to members (or reviewers) into what is planned over the next period. The ILC has also had too many scattered activities that do not seem to lie on any plausible critical path towards significant land policy or administration advances. Moreover, they often seem to be interventions that other players could do equally well and are therefore not playing sufficiently to ILC's comparative advantage. · The Coalition needs to retain sufficient connection to an IGO for stability and the ability to recruit and retain quality staff. However, the legitimacy derived from such a connection, even though sometimes perceived by governments, is a false legitimacy. Any claim to legitimacy will be built from its membership, from demonstrating positive results, and from its reputation for collective knowledge, facilitation and capacity building. 38 · M&E will be the lifeblood of the ILC. More focus is needed on outcomes. Demonstrating results and efficiency has now become urgent. While there should be realism of expectations matching the resources, evidence on performance that remains largely at the process and facilitation levels becomes increasingly less convincing over time in the absence of output evidence on the actual enhanced capacities of intermediaries and at least the early stages of attributable outcome evidence on the welfare of poor landowners or the landless. The three M&E priorities should be (a) to develop logical frameworks and measurable indicators at the output and outcome levels at central and regional levels; (b) to develop M&E methodologies for CSO, country and regional levels; and (c) to develop analytical tools so that outcomes at lower levels, whether quantitative or qualitative, can be aggregated. · The ILC needs better brand recognition. It needs to build Secretariat and regional- level land skills, together with skill linkages to other sources, to ensure that, as a Coalition, it can offer a real and recognized comparative advantage on global land issues. 6.5 The main lessons from this Global Program Review for the World Bank are: · The World Bank should be more proactive in contributing to maintaining the relevance of the program with respect to role, membership, prioritization at the regional level, fiduciary management, and M&E. (The same could be said of other IGOs.) The Bank should particularly monitor the progress of the recent Coalition changes. · There is need for establishing greater clarity within the Bank about its long and short- term goals for working with the ILC. With greater regionalization of the ILC, the Bank will need to watch more closely the reputational risks but should also gain more opportunities to link ILC members' work with the design and implementation of Bank-supported land projects. Task managers of global programs should have Sector Department-approved terms of reference that formalize these long and short-term goals and should be provided budget allocations commensurate with the task. Staff engagement with the Coalition seems often to have been an unfunded mandate in the past. · The Bank's global program reporting systems need to be more transparent with respect to the final destination of the Bank's financial contributions. In the case of ILC, the Bank's financial support from the Development Grant Facility was passed through IFAD because it has been the legal entity for the ILC which also administers the ILC trust fund. That the funds were destined for the ILC was not recorded in the Bank's reporting systems. . 39 References 16 Alilio, Martin S., with, Ib C. Bygbjerg, and Joel G. Breman. "Are Multilateral Malaria Research and Control Programs the Most Successful? Lessons from the past 100 years in Africa." American Journal of Tropical Medicine and Hygiene, 71, 2004 Bell, K. C. "World Bank Support for Land Administration and Management: Responding to the Challenges of the Millennium Development Goals" Article of the Month, November 2006, International Federation of Surveyors CAPRi, "CGIAR System-wide Program on Collective Action and Property Rights." Policy Brief Number 5, January 2007 Church M. et al. "Participation, Relationships and Dynamic Change: New Thinking on Evaluating the Work of International Networks." Working Paper No 121. Development Planning Unit. University College, London 2002. Available on: dpu@ucl.ac.uk Deininger, K. "Land Policies for Growth and Poverty Reduction: Key Issues and Challenges Ahead." Article of the Month, December 2004, International Federation of Surveyors ------. "Land Policies for Growth and Poverty Reduction." World Bank Policy Research Report, June 2003 Co-publication of the World Bank and Oxford University Press De Wit, P., Bellini, M., and Hatcher, J. "The FAO land programme in the Sudan: from emergency interventions to sustainable development" February 2005, Reforma Agraria, FAO, Rome (and other papers in that series) Fuys, A, and Di Gessa, S. "Common Ground. The Challenges to developing inclusive land policies" International Land Coalition, April 2007 Hatcher, J. "Peasant Movements and International Organizations: Facing Contemporary Agrarian and Land Reforms" January 2005, Masters Thesis, European School of Advanced Studies ICARRD. "For a New Agrarian Reform Based on Food Sovereignty." Submitted at the "Land, Territory and Dignity" Forum, Porto Alegre, International Conference on Agrarian Reform and Rural Development (ICARRD) March 6­9, 2006 ICARRD. "Statement by Civil Society Organization -- Conference Declaration." International Conference on Agrarian Reform and Rural Development (ICARRD), Porto Alegre, March 2006 IFAD. "Advancing Together -- The International Land Coalition's Plan of Action in Response to the External Evaluation." Approved by the 8th Coalition Council, Rome, 7­8 November 2006, IFAD Rome ------. "Report on Lessons Learned by the International Land Coalition." Submitted to IFAD Governing Council February 2007 ------. "Progress Report on the Global Mechanism of the United Nations Convention to Combat Desertification." Executive Board Rome 12­14 December, 2006, IFAD ------. "Progress Report on the Operations and Institutional Change Agenda of the ILC." IFAD December 2007 ------. "IFAD's Plan for its Future Relationship with the International Land Coalition." IFAD 2006 ILC (International Land Coalition). "Constitution of the International land Coalition." ILC 2007 ------. "ILC Monitoring and Evaluation Background Paper." ILC 2007 16. Beyond those of most relevance to the report, this reference list does not repeat the many ILC documents available on the ILC Web site that have been looked at. 40 ------. "Land, Dignity and Development: Putting a Pro-Poor Land Agenda into Practice" ILC Assembly Report, April 2007 ------. "Working at the Boundaries -- International Land Coalition's engagement with the land rights of indigenous people's and their neighbors" July 2007, International Land Coalition, ------. "Strategic Framework 2007­2011. Putting a Pro-Poor Land Agenda into Practice". ILC 2006 ------. "Biennial Report, 2005 to 2007 of the International Land Coalition" 2005 ------. "Report of Electronic Consultation on Draft European Union Land Policy Guidelines" ILC Rome Italy, May 7, 2004 ------. "Securing Access to Common Property Resources in a `Modernizing' World." Brief ------. "Land Alliances for National Development: Programme Description and Guidelines for Establishing LAND Partnerships" Paper ------. "Sharing Knowledge and Building Networks for Collective Empowerment" ISS/ICODC. "Land Poverty, Social Justice and Development" A Joint Project of the Institute of Social Studies, Inter-Church Organization for Development and cooperation in collaboration with Coirdaid, FIAN, Nivib and 11.11.11, The Hague, 9­14 January, 2006 Karikari, I. B., "Ghana's Land Administration Project (LAP) and Land Information Systems (LIS) Implementation: The Issues", Article of the Month, February 2006, International Federation of Surveyors Land Watch Asia. "Report of Regional Strategy Meeting. Land Watch Asia" 28­30 March 2007, Tagaytay City, Philippines Moore, B. "Making Land Distribution Productivity Enhancing" ILC. Lecture to WBI/SADC/SARPN. July 2007 Mulvaney, P. "Beyond the World Food Summit: Can Civil Society Organizations collectively rise to the challenge?" Development in Practice, Vol. 7 (3), 1997 Nori, M. "Mobile livelihoods, patchy resources and shifting rights: approaching pastoral territories" ILC Issues Paper. April 2007 Palmer, R. "Literature Review of Governance and Secure Access to Land." Governance and Social Development Resource Centre commissioned by DFID. 2007 Universalia. "External Evaluation" (of the International Land Coalition), Final Report, August 2006, Universalia Consultants UPA. "New Vision for Land Reform" Common Minimum Programme, 27th May, 2004, United Progressive Alliance, India Yiri, K-N, "Customary Lands Administration and Good Governance -- The State and the Traditional Rulers Interface" Article of the Month, May 2006, International Federation of Surveyors 41 Annex A Annex A. Evaluation Framework for Global Program Reviews Note: This evaluation framework is a general framework that has been designed to cover the wide range of such programs in which the World Bank is involved, encompassing policy and knowledge networks, technical assistance programs, and investment programs. It is not expected that every global program review will cover every question in this table in detail. Annex Table 1. Assessing the Independence and Quality of the Evaluation Evaluation Questions 1. Evaluation process To what extent was the GRPP evaluation independent of the management of the program, according to the following criteria: · Organizational independence? · Behavioral independence and protection from interference? · Avoidance of conflicts of interest? Factors to take into account in answering these questions include: · Who commissioned and managed the evaluation? · Who approved the terms of reference and selected the evaluation team? · To whom the evaluation team reported, and how the evaluation was reviewed? · Any other factors that hindered the independence of the evaluation such as an inadequate budget, or restrictions on access to information, travel, sampling, etc.? 2. Monitoring and evaluation framework of the program To what extent was the evaluation based on an effective M&E framework of the program with: · Clear and coherent objectives and strategies that give focus and direction to the program? · An expected results chain or logical framework? · Measurable indicators that meet the monitoring and reporting needs of the governing body and management of the program? · Systematic and regular processes for collecting and managing data? 3. Evaluation approach and scope To what extent was the evaluation objectives-based and evidence-based? To what extent did the evaluation use a results-based framework -- constructed either by the program or by the evaluators? To what extent did the evaluation address: · Relevance · Governance and management · Efficacy · Resource mobilization and financial management · Efficiency or cost-effectiveness · Sustainability, risk, and strategy for devolution or exit 4. Evaluation instruments To what extent did the evaluation utilize the following instruments: · Desk and document review · Consultations/interviews and with whom · Literature review · Structured surveys and of whom · Site visits and for what purpose: for interviewing implementers/beneficiaries, or for observing activities being implemented or completed Annex A 42 Evaluation Questions · Case studies · Other 5. Evaluation feedback To what extent have the findings of the evaluation been reflected in: · The objectives, strategies, design, or scale of the program? · The governance, management, and financing of the program? · The monitoring and evaluation framework of the program? Annex Table 2. Providing an Independent Opinion on the Effectiveness of the Program Every review is expected to cover the first four criteria in the following table: (a) relevance, (b) efficacy, (c) efficiency, and (d) governance and management. A review may also cover (e) resource mobilization and financial management and (f) sustainability, risk, and strategies for devolution or exit if the latter are important issues for the program at the time of GPR, and if there is sufficient information available on which to base an independent opinion. Evaluation Criteria and Questions Relevance: The extent to which the objectives and design of the program are consistent with (a) current global/regional challenges and concerns in a particular development sector and (b) the needs and priorities of beneficiary countries and groups. 1. Supply-side relevance -- the existence of an international consensus that global/regional collective action is required. To what extent does the program reflect an international consensus on the need for action, on the definition of the problem being addressed, on priorities, and on strategies for action? Is the original consensus that led to the creation of the program still present? Is the program still needed to address specific global/regional public concerns? Take into account the origin of the program in answering these questions: · Is the program formally responsible for implementing an international convention? · Did the program arise out of an international conference? · Is the program facilitating the implementation of formal standards and approaches? · Is the program primarily donor-driven? Did donors establish the program with little consultation with developing countries? · Is the program primarily Bank-driven? Did the World Bank found the program and then seek other partners? 2. Demand-side relevance -- alignment with beneficiary needs, priorities, and strategies. To what extent are the objectives consistent with the needs, priorities, and strategies of beneficiary countries as articulated in the countries' own PRSPs, and in donors' strategies such as the World Bank CASs, and the UN Development Assistance Frameworks? To what extent has the voice of developing and transition countries been expressed in the international consensus underlying the program? 3. Vertical relevance -- consistency with the subsidiarity principle. To what extent are the activities of the program being carried out at the most appropriate level -- global, regional, national, or local -- in terms of efficiency and responsiveness to the needs of beneficiaries? To what extent are the activities of the program competing with or substituting for activities that individual donors or countries could do more efficiently by themselves? Pay particular attention to those programs that, on the face of it, are primarily supporting the provision of national or local public goods. 43 Annex A Evaluation Criteria and Questions 4. Horizontal relevance -- the absence of alternative sources of supply. What is the comparative advantage, value added, or core competency of the program relative to other GRPPs with similar or complementary objectives? To what extent is the program providing additional funding, advocacy, or technical capacity that is otherwise unavailable to meet the program's objectives? To what extent are the good and services being provided by the program in the nature of public goods? Are there alternative ways of providing these goods and services, such as by the private sector under regular market conditions? 5. Relevance of the design of the program To what extent are the strategies and priority activities of the program appropriate for achieving its objectives? What are the major activities of the program: · Policy and knowledge networking? · Financing country and local-level technical assistance? · Financing investments to deliver national, regional, or global public goods? (See Annex Table 4.) Has the program articulated an expected results chain or logical framework, along with assumptions that relate the progress of activities with the achievement of the objectives? Does the results chain identify the extent to which the achievement of the objectives depends on the effective functioning of bureaucracies, markets, or collectivities? If so, to what extent are these assumptions valid? For programs providing global or regional public goods, is the design of the program consistent with the way in which the individual efforts of the partners contribute to the collective outcome for the program as a whole -- whether "best shot", "summation", or "weakest link?" Efficacy: The extent to which the program has achieved, or is expected to achieve, its objectives, taking into account their relative importance. 6. Achievement of objectives To what extent have the stated objectives of the program been achieved, or has satisfactory progress been made towards achieving these objectives? To what extent are there implicit objectives that are well understood and agreed upon by the partners and to which the program should also be held accountable? To what extent are there any positive, unintended outcomes of the program that have been convincingly document? To what extent have these assessments by the program or the evaluation been evidence-based? 7. Progress of activities, outputs, and outcomes. To what extent has the program or the evaluation measured the progress of activities, outputs, and outcomes? How did the program or the evaluation aggregate its outputs and outcomes at all levels -- global, regional, national, and local -- to provide an overall summary of its results? To what extent have factors such as changes in the location of the program, its legal structure, or governance processes affected the outputs and outcomes of the program? To what extent have there been outcomes that can be uniquely attributed to the partnership itself -- such as the scale of or joint activities made possible by its organizational setup as a GRPP, or its institutional linkages to a host organization? 8. Linkages to country or local-level activities. To what extent has the program established effective operational linkages with country-level activities, taking into account that: · The desired nature of these linkages will vary according to the objectives, design, and implementation of each program? · Positive outcomes at the country or local level are generally a joint product of both global/regional and county- level activities? Annex A 44 Evaluation Criteria and Questions Efficiency or cost-effectiveness: Efficiency -- the extent to which the program has converted or is expected to convert its resources/inputs (such as funds, expertise, time, etc.) economically into results. Cost-effectiveness -- the extent to which the program has achieved or is expected to achieve its results at a lower cost compared with alternatives. 9. Efficiency To what extent is it possible to place a monetary value on the benefits arising from the activities of the program? To what extent has the program or the evaluation conducted impact evaluations of representative program activities? To what extent has the program or the evaluation analyzed the program's costs in broad categories (such as overhead vs. activity costs), and categorized the program's activities and associated benefits, even if these cannot be valued in monetary terms? 10. Cost-effectiveness To what extent is the program measuring up against its own business plans: · Has the program cost more or less than planned? How did it measure up against its own costing schedule? · Have there been any obvious cases of inefficiency or wasted resources? To what extent is the program delivering its activities cost-effectively in comparison with alternatives: · How do actual costs compare with benchmarks from similar programs or activities? · Are the overhead costs of governing and managing the program reasonable and appropriate in relation to the objectives and activities of the program? How does the program compare with traditional development assistance programs: · For beneficiary countries, has receiving the development assistance through the GRPP increased the transactions costs compared with traditional development assistance programs? · For donors, has delivering the development assistance through the GRPP reduced donor costs by harmonizing efforts among donors or by reducing overlapping work (such as through joint supervision, monitoring and evaluation)? Governance and management: Governance -- the structures, functions, processes, and organizational traditions that have been put in place within the context of a program's authorizing environment to ensure that the program is run in such a way that it achieves its objectives in an effective and transparent manner. Management -- the day-to-day operation of the program within the context of the strategies, policies, processes, and procedures that have been established by the governing body. Whereas governance is concerned with "doing the right thing," management is concerned with "doing things right." 11. Compliance with generally accepted principles of good governance. To what extent are the governance and management structures and processes well articulated and working well to bring about legitimate and effective governance and management? To what extent do governance and management practices comply with the following seven principles: · Legitimacy -- the way in which governmental and managerial authority is exercised in relation to those with a legitimate interest in the program -- including shareholders, other stakeholders, implementers, beneficiaries, and the community at large? · Accountability -- the extent to which accountability is defined, accepted, and exercised along the chain of command and control within a program, starting with the annual general meeting of the members or parties at the top and going down to the executive board, the chief executive officer, task team leaders, implementers, and in some cases, to the beneficiaries of the program? · Responsibility -- the extent to which the program accepts and exercises responsibility to stakeholders who are not directly involved in the governance of the program and who are not part of the direct chain of accountability in the implementation of the program? 45 Annex A Evaluation Criteria and Questions · Fairness -- the extent to which partners and participants, similarly situated, have equal opportunity to influence the program and to receive benefits from the program? · Transparency -- the extent to which a program's decision making, reporting, and evaluation processes are open and freely available to the general public? · Efficiency -- the extent to which the governance and management structures enhance efficiency or cost- effectiveness in the allocation and use of the program's resources? · Probity -- the adherence by all persons in leadership positions to high standards of ethics and professional conduct over and above compliance with the rules and regulations governing the operation of the program? 12. Partnerships and participation To what extent has the program identified a complete list of stakeholders, or "stakeholder map", including the agreed- upon or perceived roles and responsibilities of the categories of stakeholders identified? To what extent is this a routine programmatic function, updated regularly, and transparently available? Has the program adopted primarily a shareholder model of governance (in which membership on the governing body is limited to financial and other contributors), or a stakeholder model (in which membership also includes non- contributors)? To what extent, if any, is the program's legitimacy being sacrificed in order to achieve greater efficiency, or vice-versa? 13. Programs located in host organizations To what extent is the location of the program in the Bank or other partner organization adversely affecting the governance, management, or other aspects of the program, such as compliance with the principles of transparency and fairness? For which functions is the program manager accountable to the host organization and the governing body of the program, respectively? Are conflicts of interest being managed appropriately? To what extent does the host organization play such a dominant role in the program, thereby reducing the incentives of other partners to participate effectively, or reducing the ability of the host organization to look at the weaknesses of the program objectively? Resource mobilization and financial management: Resource mobilization -- the processes by which resources are solicited by a program and provided by donors and partners. Financial management -- the processes that govern the recording and use of funds, including allocation processes, crediting and debiting of accounts, controls that restrict use, accounting, and periodic financial reporting systems. In cases where funds accumulate over time, this would also include the management of the cash and investment portfolio. 14. Resource mobilization To what extent has the program succeeded in raising financial resources commensurate with its objectives? And from what sources -- the Bank, bilateral donors, foundations, etc.? To what extent has the program succeeded in diversifying its funding beyond a small number of donors? To what extent are the sources of funding for the program (including donor restrictions on the use of resources) affecting, positively or negatively: · The strategic focus of the program? · The outputs and outcomes of the program? · The governance and management of the program? · The sustainability of the program? Annex A 46 Evaluation Criteria and Questions 15. Financial management Are there any issues that have emerged during the course of the review in relation to: · The quality of financial management and accounting? · The methods, criteria, and processes for allocating funds among different activities of the program? · Financial management during the early stages of the program? Sustainability, risk, and strategy for devolution or exit: Sustainability -- When applied to the activities of a program, the extent to which the benefits arising from these activities are likely to continue after the activities have been completed. When applied to a program itself, the extent to which the organization or program is likely to continue its operational activities over time. Devolution or exit strategy -- a proactive strategy to change the design of a program, to devolve some of its implementation responsibilities, to reduce dependency on external funding, or to phase out the program on the grounds that it has achieved its objectives or that its current design is no longer the best way to sustain the results which the program has achieved. 16. Sustainability of the benefits of the program's activities What is the risk, at the time of evaluation, that the development outcomes (or expected outcomes) of the program will not be maintained (or realized)? This depends on (a) the likelihood that some changes may occur that are detrimental to maintaining or realizing the expected outcomes, and (b) the affect on the expected outcomes if some or all of these changes actually materialize? 17. Sustainability of the program This will depend on a number of factors, such as the continued legitimacy of the program, its financial stability, its continuity of effective management, and its ability to withstand changing market or other conditions. To what extent is there still a sufficient convergence or accommodation of interests among the major partners to sustain the program financially? To what extent has the program developed institutional capacity such as performance- based management, personnel policies, learning programs, and knowledge management that help to sustain a program? In what areas could the program improve in order to enhance its sustainability, such as better marketing of the program's achievements in order to sustain its reputation? 18. Prospects for continuation and strategies for devolution or exit To what extent should the program be sustained? Is the continuation of the program the best way of sustaining the results achieved? Should the design of the program be modified as a result of changed circumstances, either positive or negative? What other alternatives should be considered to sustain the program's results more cost-effectively, in the light of the previous evaluation findings with respect to relevance, efficacy, efficiency, and sustainability: · Reinventing the program with the same governance? · Phasing out the program? · Continuing country or local-level activities with or without devolution of implementation? · Seeking alternative financing arrangements, such as revenue-generation, or self-financing to reduce dependency on external sources? · "Spinning off" from the host organization? 47 Annex A Annex Table 3. Assessing the Bank's Performance as a Partner in the Program Evaluation Questions 1. Comparative advantage at the global/regional level. To what extent is the Bank playing up to its comparative advantages at the global/regional level -- its global mandate and reach and convening power? To what extent is the Bank's presence as a partner in the program catalyzing other resources and partners for the program? 2. Comparative advantage at the country level. To what extent is the Bank contributing multi-sector capacity, analytical expertise, and country-level knowledge to the program? To what extent has the Bank's country operations established linkages to the GRPP, where appropriate, to enhance the effectiveness of both? 3. Oversight. To what extent is the Bank exercising effective and independent oversight of its involvement in the program, as appropriate, whether the program is housed in the Bank or externally managed? To what extent is the Bank's oversight independent of the management of the program? To what extent does the Bank's representative on the governing body have a clear terms of reference? 4. Risks and risk management. To what extent have the risks associated with the program been identified and are being effectively managed? For example, IEG identified the following risks in its global review: · Bank bears a disproportionate share of responsibility for governing and managing in-house programs? · Confusion at the country level between global program activities, Bank activities, and Borrower activities? · Representation of NGOs and the commercial private sector on program governing bodies? · Unclear role and application of Bank's safeguards? · Trust-funded consultants and seconded staff representing the Bank on some program governing bodies? 5. Disengagement strategy. To what extent is the Bank engaged at the appropriate level in relation to the Bank's new strategic framework: · Watching brief? · Research and knowledge exchange? · Policy or advocacy network? · Operational platform? To what extent is the Bank facilitating an effective, flexible, and transparent disengagement strategy for the program, in relation to the Bank's objectives for its involvement in the program: · The program declares "mission accomplished" and closes? · The program continues and the Bank withdraws from all aspects of its participation? · The program continues and the Bank remains engaged, but the degree of the Bank's engagement in some or all aspects (such as financing) declines over time? Annex A 48 Annex Table 4. Common GRPP Activities Policy and knowledge networking 1. Facilitating communica- This includes providing a central point of contact and communication among practitioners tion among practitioners who are working the sector or area of development to facilitate the sharing of analytical in the sector results. It might also include the financing of case studies and comparative studies. 2. Generating and This comprises two related activities. The first is gathering, analyzing and disseminating disseminating information, for example, on the evolving HIV/AIDS epidemic and responses to it, information and including epidemiological data collection and analysis, needs assessment, resource knowledge flows, and country readiness. The second is the systematic assembling and dissemination of knowledge (not merely information) with respect to best practices in a sector on a global/regional basis. 3. Improving donor This should be an active process, not just the side effect of other program activities. This coordination may involve resolving difficult interagency issues in order to improve alignment and efficiency in delivering development assistance. 4. Advocacy This comprises proactive interaction with policymakers and decision makers concerning approaches to development in a sector, commonly in the context of global, regional, or country-level forums. This is intended to create reform conditions in developing countries, as distinct from physical and institutional investments in public goods, and is more proactive than generating and disseminating information and knowledge. 5. Implementing Rules are generally formal. Standards can be formal or informal, and binding or conventions, rules, or nonbinding, but implementing standards involves more than simply advocating an formal and informal approach to development in a sector. In general, there should be some costs associated standards and norms with noncompliance. Costs can come in many forms, including exposure to financial contagion, bad financial ratings by the IMF and other rating agencies, with consequent impacts on access to private finance; lack of access to OECD markets for failing to meet food safety standards, or even the consequences of failing to be seen as progressive in international circles. Financing technical assistance 6. Supporting national- This is more directed to specific tasks than advocacy. This represents concrete level policy, institutional, involvement in specific and ongoing policy, institutional, and technical reform processes and technical reforms in a sector, from deciding on a reform strategy to implementation of new policies and regulations in a sector. It is more than just conducting studies unless the studies are strategic in nature and specific to the reform issue in question. 7. Capacity strengthening This refers to strengthening the capacity of human resources through proactive training and training (in courses or on-the-job), as well as collaborative work with the active involvement of developing country partners. 8. Catalyzing public or This includes improving regulatory frameworks for private investment and implementing private investments in pilot investments projects. the sector Financing investments 9. Financing country-level This refers primarily to physical and institutional investments of the type found in Bank investments to deliver loans and credits (more than the financing of studies), the benefits of which accrue national public goods primarily at the national level. 10. Financing country-level This refers primarily to physical and institutional investments of the type found in Bank investments to deliver loans and credits (more than the financing of studies) to deliver public goods such as global/regional public conserving biodiversity of global significance and reducing emissions of ozone-depleting goods substances and carbon dioxide, the benefits of which accrue globally. 11. Financing global/ This refers to financing research and development for new products and technologies. regional investments to These are generally physical products or processes -- the hardware as opposed to the deliver global/regional software of development. public goods 49 Annex B Annex B. The External Evaluation Findings External Context: Finding 1. International and national government interest in, and support for, land reform has fluctuated over time and is now on the rise as land reform becomes an increasingly important element of the global development agenda. Finding 2. In 2006, the land reform movement is characterized by a range of important actors (government, inter-government, non-government and private sector). This context fuels the need for effective mechanisms that encourage and foster multi-stakeholder dialogue about land issues. Finding 3. Donor harmonization, when effectively pursued, provides opportunities for collaboration among traditionally disparate groups at international levels, and, more occasionally, at national levels. Finding 4. While there is increased and widespread recognition of civil society's important role in land reform and protecting the rights of the poor, donor support is modest relative to civil society capacity building and program delivery needs. Finding 5. Land reform issues, needs, and possible solutions vary from country to country, and from region to region. Finding 6. The rapid expansion of affordable information technology provides increasing opportunities to share knowledge among geographically and economically disparate groups on global issues, including land reform. ILC's Institutional Context: Finding 7. While born a decade ago, ILC is in the early stages of institutional development. Finding 8. Among its stakeholders, ILC is perceived to have several competing institutional identities. (After this lesson the report explains that some stakeholders have radically different expectations of ILC performance). Finding 9. As ILC's host for the past decade, IFAD has played a critically important role in ILC's founding and continued existence. Internal changes within IFAD have significant and immediate implications for ILC's future growth and development. ILC's Relevance: Finding 10. ILC's mission and objectives are congruent with emerging global interest in land reform and with regional and local needs, and thus highly relevant. Annex B 50 Finding 11. Most current members and partners, including IFAD, see ILC objectives as highly relevant. Finding 12. Social movements are gaining importance and power in the land reform arena, but a number of important social movements involved in land issues are not part of ILC's global alliance. This could undermine ILC's future relevance if it is not addressed. Mission and Objectives: Finding 13. Most CSO members and partners interviewed indicate that the Coalition is working successfully towards its mission and objectives. IGO members and partners have more diverse opinions, ranging from positive to very critical. Strategic Thrusts: Finding 14. Of its six strategic thrusts, ILC has been most effective in creating spaces for dialogue, advocacy, and documenting and sharing knowledge; it has been least effective in coalition building. Programming Areas: Finding 15. Stakeholders broadly acknowledge the success of ILC's individual programs (in particular the CEF), but agree that the Coalition's programming should become more strategic to ensure more systematic links among its program areas and to make the most effective use of its resources. Finding 16. ILC has been only moderately successful in addressing the gender dimensions of land reform in its programming. However, the Secretariat is developing a strategy for linking experiences in gender and land issues to all programming areas. Strategic Management: Finding 17. ILC is not legally constituted as an organization. (i.e. there is no legal charter or letter of incorporation) Finding 18. While several IFSAD documents proclaim that ILC is independent from IFAD, the matter remains unresolved. This has important consequences for ILC's future growth and development. Finding 19. IFAD has multiple relationships with ILC (as host, member, donor, and partner). IFAD's role as host has overshadowed its other relationships with ILC, and may be preventing both ILC and IFAD from building on the potential benefits of IFAD as a member of the Coalition. 51 Annex B Finding 20. ILC has several unique characteristics and some important comparative advantages. Finding 21. ILC acknowledges that it needs to define a clear, distinct, recognized and appropriately resourced role. Finding 22. ILC has taken some important steps to manage itself more strategically, but will need to do more to overcome some major challenges. Finding 23. The Coalition lacks comprehensive systems for regular feedback, supervision, monitoring and evaluating ILC's progress at program and institutional levels. Membership, Governance, and Management Structures: Finding 24. ILC's Constitution and Governance Framework provides for various forms of participation in ILC. However, the purpose, benefits and expectations of each kind of participation are unclear. Finding 25. ILC is working to clarify its membership criteria and selection process. Finding 26. Given its large potential constituency and its stated intention to be a global coalition, ILC has a relatively limited number of members. ((36 check update)) Finding 27. Until now ILC has been a centralized organization, driven more by the ILC Secretariat and IFAD than by its members. ILC members now see the need for this to change. Finding 28. Governance and management roles and responsibilities within ILC are evolving and becoming clearer over time. However, some inconsistencies between what is stated in the ILC Constitution and Governance Framework document and how they are being implemented need to be addressed. Program Management: Finding 29. ILC has processes in place to plan and guide its ongoing projects and activities, but not its programs. While generally functional, existing processes differ in their degree of systematization and transparency. Initiatives and activities are monitored, but there is no joint, coherent monitoring approach. ILC-IFAD Relationship: Finding 30. While ILC and IFAD state that they have a strategic partnership, this has not been operationalized at the corporate level. Annex B 52 Inter-organizational Partnerships: Finding 31. IGO members and partners have engaged in many successful joint activities with ILC that have supported their own mandates and helped them reach out to CSOs. Several of ILC's IGO members and partners have recently established relationships with other bodies/forums that will give them access to CSOs and grassroots movements -- some stakeholders see this as a threat, while ILC considers it a demonstration of ILC's success. Resource Management and Mobilization: Finding 32. ILC Secretariat staff are highly regarded by most partners and members. Finding 33. ILC manages its human resources and finances in keeping with IFAD policies and procedures. This provides several important short-term benefits to ILC as well as some longer-term disadvantages. Finding 34. ILC Secretariat staff expertise tends to be defined in general rather than specific terms, which undermines its potential added value. Finding 35. ILC continues to be dependent on IFAD for financial support, presenting risks to both organizations. Finding 36. ILC's existing business model is not adequately developed for its long-term sustainability. Communications and Publications: Finding 37. CSO members and partners highly value the Secretariat's effective personal communications as well as its professional communications through the Web site and publications. Finding 38. ILC's draft Communications Strategy (2006­2009) outlines a systematic approach for improving the Coalition's internal and external communications, but only marginally addresses the question of how to strengthen exchange among members and partners at national or regional levels. Value for Money: Finding 39. Available data suggest that ILC provides reasonable value for money to its investors, but institutional hosting arrangements (IFAD) and organizational use of members' expertise have limited its longer term effects. 53 Annex C Annex C. Outcome Evidence Tabulation The review attempts here to assemble a selection of the main, largely anecdotal, output and outcome evidence presented in the EE classified by ILC objective. In some cases reported achievements could fit under different or multiple columns. EE page numbers are given in the left hand column. Annex Table 5. Major Findings of the External Evaluation Page No. Findings Objective #1. Enhancing Capacity of Members, Partners and Affiliates (including knowledge, networking, etc.) 43 91% partners and 58% members say in EE survey ILC provides important value-added 44, 53 239 CEF applications by 2006, 41 funded 44 4 South Africa CSOs and 3 Guatemala CSOs cited 46 "widely acknowledged to be effectively documenting and disseminating ..." 46 Web site access numbers substantial 46 Access provided to more than 450 papers, studies, etc. 47 Partners see ILC successfully assisting CSOs to strengthen capacity. Cases cited in Kenya, Uganda and Zambia Land Alliances 50 Weaknesses in use of research on networking, better synthesis of lessons, harness knowledge of larger membership. 48 e-consultation on Common Property and pastoral land "stakeholders speak positively of the process" 48 Survey of members on Donor Policy 2005; stakeholders positive about consultation process. 49 NSP helped new networks. Nepal, Cambodia, Central America and Philippines. Also Land Netet West Africa helped expand. 50­51 Performance in scaling up weak, e.g., CNOC and CODECA Guatemala legal support for landless, no scaling up resources (but the EE rightly asks here whether this is the role of ILC.) 52 With KP and NSP and link to advocacy, some stakeholders say should select topics and organize more strategically. 53­54 CEF is seen by many as flagship, but some question whether ILC suited to reaching down to community level given limited resources, hopes can be raised and dashed Objective #2: Poor Gaining Land Access 43 86 percent of members and partners agree that ILC is playing an important role in land movement 56 CALI program in Uganda, also Niger, and in some CEF grants e.g. India AP Objective #3. Poor Gaining Related Production Services Nothing evident under this objective except some small elements of a few of the CEF grants, e.g., India AP CEF support. Objective #4. Facilitating Dialogue with Decision-Makers (incl. advocacy) 45 Positive qualitative responses in survey. CSOs and some IGOs but some say need to reach out to wider array of players 45 But seen as activity driven not strategic Annex C 54 Page No. Findings 47 Members, partners, say ILC successful in facilitating dialogue but only "to some extent in decision-making processes" 47 Sponsorship of attendance at 4 big international conferences 48 Six countries cited (Guatemala, Philippines, South Africa, Zambia, Uganda, Niger) where ILC "encouraged governments to engage in dialogue." But some respondents skeptical that it has gone beyond statements of intent. 54­55 LAND program since 2003, participatory dialogue processes in Guatemala, Indonesia, Philippines, South Africa. Little evidence yet of improved policy processes or joint action to secure resources. (in SA LAND program halted due shift of policy) But limited gains must be judged against difficulty. LAND could become the country focus activity into which KP, NSP, CEF, feed. 55 Platform 2002 aimed to present shared positions but nothing further happened. 55 Annex D Annex D. ILC Members, Council, and Secretariat Annex Table 6. ILC Members, at the Time of the External Evaluation (Members hold voting rights and may be elected to the Coalition Council.) Institution Country Web sites ALOP - Asociacion Latino-Americana de Costa Rica http://www.alcop.or.cr Organizaciones e Promocion ALRD - Association for Land Reform and Bangladesh http://www.alrd@agni.com Development ANGOC - Asian NGO for Agrarian Reform Philippines http://www.angoc.ngo.ph/ and Rural Development BWI - Bread for the World Institute United States http://www.bread.org CCC - Cooperation Committee for Cambodia http://www.ccc-cambodia.org Cambodia CEPES - Centro Peruano de Estudios Peru http://www.cepes.org.pe Sociales CNIRD - Caribbean Network for Integrated Trinidad and http://www.cnird.org Rural Development Tobago COCOCH - Consejo Coordinador de Honduras http://www.twnside.org.sg Organizaciones Campesinas de Honduras CONGCOOP - Coordinacion de ONG y Guatemala http://www.congcoop.org.gt Cooperatives DWHH - Deutsche Welthungerhilfe Germany http://www.welthungerhilfe.de http://www.europa.eu.int/comm/ EC - European Commission European Union index_en.htm ELCI - Environment Liaison Centre Kenya http://www.elci.org International FAO - Food and Agriculture Organization of International http://www.fao.org the United Nations FENACOOP - Federeacion Nacional de Cooperatives Agropecuarias y Nicaragua fenacoop@turbonett.com AgroIndustriales R.L. FEPP - Fondo Ecuatoriano Populorum Ecuador proterias@fepp.org.ec Progressio Fundacion TIERRA Bolivia http://www.tierra.org Grupo ALLPA - Comunidades y Desarrollo Peru http://www.allpa.org.pe ALLPA ICRAF - International Centre for Research http://www.worldagroforestry- International in Agroforestry centre.org IDB - Inter-American Development Bank Regional http://www.iadb.org IFAD - The International Fund for International http://www.ifad.org Agricultural Development IFAP - International Federation of International http://www.ifap.org Agricultural Producers Annex D 56 Institution Country Web sites IFPRI - International Food Policy Research International http://www.ipri.org Institute IUF - International Union of Food, Agricultural, Hotel, Restaurant, Catering, Switzerland http://www.iuf.org Tobacco and Allied Workers' Association KPA - Konsorsium Pembaruan Agraria Indonesia http://www.kpa.or.id Consortium for Agrarian Reform LandNet West Africa Burkina Faso wa_landnet@yahoo.com MODE Nepal (Mobilization and Nepal http://www.modenepal.org.np Development) SDF - Social Development Foundation India http://www.tesdf.org http://www.secourspopulaire. SPF - Secours Populaire Francais France asso.fr TWO - Transborder Wildlife Association Albania spllaha@yahoo.com ULA - The Uganda Land Alliance Uganda http://www.africaonline.co.ug UNEP - United National Environment International http://www.unep.org Programme WB - The World Bank International http://www.worldbank.org WFP - The United Nations World Food International http://www.wfp.org Programme ZERO Regional Environment Organisation Zimbabwe http://www.zeroregional.com ANGOC - Asian NGO for Agrian Report and Philippines http://wwwangoc@angoc.ngo.ph Rural Development Source: ILC. Annex Table 7. ILC Coalition Council Institution Country Contact Email Address ANGOC - Asian NGO for Agrarian Reform and Rural Philippines Don Marquez angoc@angoc.ngo.ph/ Development CONGCOOP - Coordinacion de Helmer coordinadordr@congcoop. Guatemala ONG y Cooperatives Velasquez org.gt Deutsche Welthungerhilfe Germany Jochen Donner jochendonner@dwhh.de European EC - European Commission Philip Mikos philip.mikos@ec.europa.eu Union FAO - Food and Agriculture Paraviz Organization of the United International parviz.koohafkan@fao.org Koohafkan Nations Fundacion TIERRA Bolivia Miguel Urioste fundaciontierra@tierra.org IFAD - The International Fund for International Kevin Cleaver k.cleaver@ifad.org Agricultural Development IFAP - International Federation of International David King ifap@ifap.org Agricultural Producers 57 Annex D Institution Country Contact Email Address IFPRI - International Food Policy Ruth Meinzen- International ifpri@cgiar.org Research Institute Dick KPA - Konsorsium Pembaruan Agraria Consortium for Agrarian Indonesia Erpan Faryadi kpa@kpa.or.id Reform Hubert LandNet West Africa Burkina Faso wa_landnet@yahoo.com Ouedraogo Malcolm The World Bank International mchildress@worldbank.org Childress WFP - The United Nations World International Allan Jury wfpinfo@wfp.org Food Programme Source: ILC. Annex Table 8. ILC Secretariat, 2008 Title Name Director Bruce Moore Programme Manager Land Policy and Africa Region Michael Taylor Programme Manager Capacity Building and Latin America Annalisa Mauro Policy Officer & Asia Region Andrew Fuys Associate Professional Officer Sabine Pallas Associate Professional Officer Barbara Codispoti Junior Programme Officer Stefano Di Gessa Fellow Peter Giampaoli Programme Assistant Lucia Angelucci Administrative clerk Alessandra Goverti Administrative clerk Hedwige Croquette Administration and Communication clerk Norma Peverell Source: ILC. Annex E 58 Annex E. ILC Finances Annex Table 9. ILC Statement of Revenues and Expenses (U.S. dollars, for the years ended December 31, 2005 and 2006.) 2005 2006 Revenues Contributions revenues 1,418,437 2,047,132 Interest income 105,695 285,816 Total Revenues 1,524,132 2,332,948 Operating Expenses Staff salaries and benefits (499,433) (860,514) Consultants and other non staff costs (265,420) (256,993) Office and general expenses (215,337) (290,648) Grant expenses (543,942) (924,793) Total Operating Expenses (1,524,132) (2,332,948) Total Revenues less Operating Expenses ­ ­ Adjustment for changes in fair value 4,215 18,829 Total Revenues less Expenses 4,215 18,829 Total foreign exchange rate movements 64,603 (43,101) Transfer to Retained Earnings 68,818 (24,272) Source: ILC/IFAD. Note: Revenues are only recognized when the related costs occur. 59 Annex E Annex Table 10. ILC Balance Sheet at Nominal Value (U.S. dollars, for the years ended December 31, 2005 and 2006.) 2005 2006 Assets Cash 3,964,623 2,879,070 Contributions receivable ­ 2,058,456 Interfund receivables * 97,800 115,619 Other receivables 2,479 31,901 Total Assets 4,064,902 5,085,046 Liabilities, Deferred Revenues, and Retained Earnings Payables and liabilities 262,682 410,260 Undisbursed grants 404,224 593,659 Interfund payables 573,616 436,640 Deferred contribution revenues 2,739,765 3,675,474 Deferred interest income 99,385 26,884 Retained earnings (14,770) (57,871) Total Liabilities, Deferred Revenues, and Retained Earnings 4,064,902 5,085,046 Source: ILC. Note: Interfund receivables and payables are movements between ILC and IFAD based on administrative services provided by IFAD and ILC. Annex F 60 Annex F. Persons Consulted In a few cases, where people could not be spoken to, emails were exchanged. This does not include those who responded to ILC questions in the IEG email consultation. Name Position Date Klaus Deininger WB Lead Econ. 12/05/07 Malcolm Childress WB Sr. Land Admin. Spec 12/09/07 Christine Cornelius WB Prog. Coord. 11/25/07 Keith Bell WB Sr. Land Admin 03/03/08 Frank Byamugisha WB 12/19/07 Jonathan Lindsay WB 03/04/08 Bruce Moore ILC 02/05/08 Michael Taylor ILC 02/05/08 Barbara Codispoti Assoc. Program Off. ILC 02/05/08 Annalisa Mauro Prog. Off. ILC 02/05/08 Andrew Fuys ILC 02/13/08 Richard Trenchard FAO Emergency Program 02/13/08 Jane Kirby-Zaki WB GPP 01/29/08 Isabel Lavandenz Paccieri WB 01/03/08 Ruth Meinzen-Dick IFPRI 01/15/08 Parviz Koohafkan Chief NRLD, FAO 02/06/08 Jessica Mott WB 11/28/07 Paul Mathieu FAO 02/06/08 Paul Munro-Faure FAO 02/19/08 Harry Liversage IFAD 02/07/08 Kevin Cleaver IFAD 02/07/08 Sappho Haralambous IFAD 02/07/08 Jean-Pierre Audinet IFAD 02/07/08 Sally Bunning FAO NRLD 01/10/08 Maria Theresa Quinones WB (Manila) 01/15/08 61 Annex G Annex G. Response of the ILC Secretariat to IEG's Global Program Review The ILC welcomes the opportunity provided by the Global Program Review (GPR) of the Independent Evaluation Group (IEG) of the World Bank to consider the effectiveness of the International Land Coalition (ILC) and the involvement of the World Bank as a member of the Coalition. Being commissioned by a founding and current member of ILC, the World Bank, it offers to the overall membership a further opportunity to consider the effectiveness of the Coalition. This is particularly salient in the light of the institutional changes that have been set in motion over the last two years, emanating from the External Evaluation (EE) of ILC. The ILC Secretariat considers this review as an opportunity to learn and refocus efforts towards becoming an increasingly meaningful and recognized global actor in land related issues. General Comments The GPR provides some insightful and valuable comments on the ILC. It confirms the overall findings of the 2006 EE, that while the potential relevance of an ILC-type of alliance is substantial, the impact so far has been modest, reflecting to some extent the complexity of land issues that make this by definition a long-term endeavor. The GPR provides useful insights on how ILC might sharpen its focus, skills and efforts in order to fully build upon its comparative advantages and demonstrate a meaningful impact on land-related poverty. The emphasis on integrating effective monitoring and evaluation into ILC's operations highlights the need for ILC to implement its recently approved M&E Framework. The GPR's insights on legitimacy and efficiency in ILC's governance and operations are useful to decisions on the hosting of the Secretariat. The ILC Secretariat also welcomes the constructive lessons and suggestions on how the World Bank may more effectively engage as a member with the wider coalition. Without detracting from valuable lessons that the review contains, the ILC Secretariat considers the methodological approach of the review to be a limiting factor in its ability to contribute to a multi-dimensional analysis of the ILC. This limitation does not invalidate the findings of the GPR, but in the opinion of the Secretariat they represent a missed opportunity to include the full spectrum and diversity of the Coalition in the issues raised by the review. The constrained view of the ILC by the GPR has institutional, geographical, and temporal dimensions. Institutionally, the GPR appears to have reviewed ILC as if it is a program, seemingly following an approach that the World Bank would apply to reviewing its own programs. This was an issue raised by the ILC Secretariat with the IEG in the initial consultations with stakeholders. If the review had been designed in the frame of ILC being a network institution the methodology and the outcomes may have been quite different, given the decision- making, policy and operational processes needed to build common approaches, joint planning and sharing of responsibilities. Furthermore, efficiency and effectiveness indicators in a network go beyond what are the outcomes and results with the ultimate beneficiaries, to Annex G 62 include; inter alia, how the members learn from on another, influence the policies and practices of the others, and build relationships for future joint actions. ILC members own ILC institutional and strategic documents, spaces and opportunities in a different way than that of a program belonging to one institution. The participation of each member in ILC activities, regionalization processes and global or national events is determined by different needs, interests and context. The choice, for example, to interview only inter-governmental members and rely on Web-based surveys for civil society members, provides a uni-dimensional view of ILC that, while not necessarily inaccurate, fails to capture many more important dimensions. Geographically, consultations for the review were primarily focused on inter-governmental members in the north. Furthermore, concerns were raised by the ILC Secretariat during the review process about the capacity of reviewers to accommodate non-English consultations and documents. This bias is evident in the review report, with very little evidence, for example, of input from members in Latin America. The review of the regionalization process would have been enhanced by inclusion of Latin America, where many of the issues addressed by the reviewer are being actively addressed by members. Once more, the methodology employed in the review has resulted in attribution of one facet of the Coalition to the Coalition as a whole, neglecting the diversity inherent in ILC as a global coalition. Temporally, the GPR draws heavily from the EE of August 2006, while also questioning both its methodology and findings. While the GPR acknowledges some of the many changes that ILC members have acted upon since the EE of 2006, its comments on the institutional change of ILC would have been more relevant had it been able to engage more deeply with the direction and substance of the change processes that the ILC has implemented since 2006. It will help readers of the GPR to fully appreciate the commitment by ILC to improve its future performance by visiting the ILC Web site to obtain a current view of the Coalition and its progress. Specific Comments Within the context of these general comments on the methodological limitations of the GPR, there are a number of specific comments to bring to the attention of IEG. EVIDENCE BASE FOR ASSERTIONS It would be useful for greater evidence to be given to some of the assertions of the review. In part this would involve explaining the process through which the review was undertaken. It is not clear how long the process lasted, how information was collected, packaged and validated. It is not clear what people were asked in semi-structured interviews, if they were semi-structured interviews, and how they were selected; equally it is not clear what the content was of the online survey, how many responded and from which countries. We propose inserting as annexes: the TORs of the IEG group; the methodology followed in collection/collation of data; the outreach of the same (how many members were involved out of the current 65 members [57 at the time of the review], how many were not from English language countries; and the methodology by which multiple evidence was brought together 63 Annex G to be considered as findings and lessons. This would allow a deeper level of engagement with the lessons and analysis of the review. Related to this is the issue of providing a clearer evidence base for general statements, which would allow ILC to be more focused in acting on the lessons. Assertions such as: "a number of grants focused on peripheral issues such as technology and marketing"; "voluminous paper work was observed"; "ILC support for land reform has yielded only patchy and opportunistic achievements"; "governance should benefit from a less paternalistic system", and; "the member CSOs surveyed had some motive to report favorably since they were actual or potential recipients of ILC funds" are vague and open to contestation unless better supported by evidence and figures. We suggest citing examples of reported findings, including a numerical proxy so that they are evidence-based. CLEARER DISTINCTION BETWEEN ILC SECRETARIAT AND THE WIDER COALITION The review does not distinguish adequately between the roles of the ILC Secretariat and those of the ILC as a coalition of members. The roles of the Secretariat are defined in the Governance and the Strategic Framework 2007-2011, which were adopted by all members, including the World Bank. The ILC Secretariat is not tasked to be a technical land unit in support of membership. However, it is supposed to facilitate a networking role, and it is on this that Task Managers of the Bank may legitimately assess the value to their work in given countries. The assertion that ILC is weak on land policy/administration should be examined in relation to the capacities of its members, or the capacity of the Secretariat to leverage these skills from within the membership. We suggest clarifying this apparent confusion of roles. RECOGNIZE ILC AS A GLOBAL KNOWLEDGE BASE In noting that the technical input to land issues comes from the experience of the members, the review incorrectly uses this as the basis to conclude that ILC is therefore not providing a global knowledge base. ILC has always been a knowledge network and is expanding this domain. The Knowledge for Change program, launched in early 2007, is one of the additional means by which ILC has expanded the opportunities for member-to-member learning and public sharing of knowledge on land issues. Members and partners often refer to the information available on the ILC Web site as a way in which they access information for use in their own policy and advocacy work. Conclusion In conclusion, the World Bank in its role as a member of ILC has played a valuable and active role in ILC's institutional life, including participation in the Council, Assemblies of Members, and the regionalization process of ILC. Despite the methodological limitations of the review, the ILC Secretariat recognizes important and valuable lessons for both ILC and the World Bank in strengthening the value that ILC brings to its members, including the World Bank, and vice-versa. The ILC Secretariat will use these lessons to strengthen and, where necessary redirect, its ongoing efforts to reshape the way in which it works. We look forward specifically to building from this review a wider-based, more clearly defined, and more dynamic relationship with the World Bank as a valued member of ILC. The Global Program Review Series The following reviews are available from IEG. Volume #1, Issue #1: ProVention Consortium Issue #2: Medicines for Malaria Venture Issue #3: Development Gateway Foundation Issue #4: Cities Alliance Volume #2, Issue #1: Critical Ecosystem Partnership Fund Issue #2: Association for the Development of Education in Africa Issue #3: Population and Reproductive Health Capacity Building Program Issue #4: International Land Coalition Volume #3, Issue #1: Consultative Group to Assist the Poor GPR The organization that is now called the International Land Coalition (ILC) was founded in 1996 following the Conference on Hunger and Poverty in 1995 in Brussels. Located in the International Fund for Agricultural Development in Rome, it is a global alliance of civil society and intergovernmental organizations set up to promote secure and equitable access to and control over land for poor women and men through advocacy, dialogue, and capacity building. Making concrete progress on an issue such as land--with its enormous political hurdles--has proven particularly challenging. Nonetheless, the ILC has made a moderate contribution, with limited resources, to land reform and has improved administration in some countries. Following an external evaluation in 2006, the ILC embarked on a proactive change process that has met milestones such as expanded membership, greater regionalization, and increased donor funding. IEG's review concludes that the ILC should revisit its nexus of membership and advocacy, improve its monitoring and evaluation, and better utilize its global and regional meetings to agree on specific actions rather than to restate principles. The World Bank's task team leaders could also have been more proactive in some areas; they would have benefited from strategically focused terms of reference that had given appropriate direction and continuity of purpose.