39187 Global Environment Facility Evaluation Office STAKEHOLDER SURVEY Technical Paper 5 Part of the Joint Evaluation of the GEF Activity Cycle and Modalities, conducted by the GEF Evaluation Office. Evaluation Component 5: Stakeholder Survey Not edited November 2006 By: Andre Aquino, GEF Evaluation Office Survey design by: Josh Brann, GEF Evaluation Office, and Michael Spilsbury, UNEP Evaluation and Oversight Unit Task Manager for the Joint Evaluation: Siv Tokle, GEF Evaluation Office Table of Contents Summary of Key Findings .............................................................................................................. 2 1. Introduction............................................................................................................................. 4 1.1 Background ..................................................................................................................... 4 1.2 Component Eight: Stakeholder Survey........................................................................... 4 1.3 Survey Respondents' Profile........................................................................................... 5 1.4 Stakeholder Groups......................................................................................................... 8 1.5 Reasons for not working with the GEF........................................................................... 9 2. The Activity Cycle................................................................................................................ 10 2.1 Direct Questions............................................................................................................ 10 2.1.1 What step of the activity cycle requires greatest resources?................................. 10 2.1.2 What step of the activity cycle requires least resources?...................................... 11 2.1.3 What are the most important activity cycle steps for improving the focus on global environmental benefits? ............................................................................................. 13 2.1.4 What are the least important activity cycle steps for improving the focus on environmental benefits? ........................................................................................................ 14 2.2 Open Questions on the Project Cycle ........................................................................... 15 2.2.1 What is the greatest strength of the GEF activity cycle? ............................................. 15 2.2.2 What is the greatest weakness of the GEF activity cycle?.................................... 17 2.2.3 How could the GEF activity cycle be improved? ................................................. 19 3. GEF Modalities..................................................................................................................... 20 3.1 Modality relevance to GEF's mandate and countries' needs........................................ 20 3.2 Medium-Size Projects: Strengths and Weaknesses ...................................................... 24 3.3 Recommendations of other possible modalities ........................................................... 27 3.4 Modalities and Activity Cycle: Strengths and Weaknesses.......................................... 27 4. GEF Operational Principles .................................................................................................. 30 4.1 Operational Principle requiring least resources ............................................................ 30 4.2 Operational Principle requiring most resources............................................................ 30 4.3 ­ Operational Principles contributing MOST to the potential success of GEF projects .... 31 4.4 Operational Principles contributing LEAST to the potential success of GEF projects 31 5. GEF in Comparison to other International Agencies............................................................ 32 5.1 Timeframe of GEF Projects .......................................................................................... 33 5.2 Resources necessary to prepare and approve GEF Projects ......................................... 33 5.3 Implementation Period of GEF Projects ....................................................................... 34 5.4 Budget of GEF Projects ................................................................................................ 34 5.5 Time and effort required for Monitoring and Evaluation............................................. 35 5.6 Preparedness of GEF projects before implementation.................................................. 35 5.7 Chances of having a significant impact ........................................................................ 36 5.8 Level of overall success ................................................................................................ 37 Annex A - Survey Methodology and Questions ........................................................................... 38 Survey Methodology................................................................................................................. 38 Survey Original questions ......................................................................................................... 39 1 Summary of Key Findings Background of the Survey. The Joint Evaluation Component 8 concerned a survey of country clients, GEF Focal Points, Agency partner HQ staff and field project staff, and other stakeholders to elicit views on experiences and challenges with regard to efficiency and modalities, and on the effectiveness of GEF's operational programming approach and strategy. The responses were analyzed in an aggregated and disaggregated fashion to demonstrate differences of opinions among stakeholder groups. This highlighted the difference of perception on the GEF's Activity Cycle and Modalities among stakeholder groups, including questions related to their own roles. Respondents' background and experience with GEF. 660 respondents gave their feedback to from early April to early June 2006, which represents a substantial response rate of 32%. 60% of total responses came from implementing agencies, followed by 16% from national government respondents. Overall, the number of respondents who had experience with Implementing Agencies is almost three times higher than those who worked with Executing Agencies. 93% of respondents have had experience with the GEF during its third Replenishment Period. The experience with world regions was equally distributed among those who worked with projects in Asia, Africa, Latin America and the Caribbean, Europe and Central Asia and global projects. Most respondents had experience with Full-Size Projects, while least with National Capacity Self-Assessments (NCSA). Reasons for not working with the GEF. 91% of respondents had experience with the GEF. Among those who had the chance to work with the GEF and opted not to, the most cited reason for that choice are the `levels of co-financing required', `national or local circumstances' and `amount of time for project design and approval', in that order. Activity Cycle · `Project Preparation' stands out as the Activity Cycle step requiring greatest amount of time and effort. `Review by GEF Secretariat' follows. · `GEF Focal Point Endorsement' and `Project Identification' are regarded as the steps requiring least amount of time and effort. · `Project Preparation' is deemed to be the activity cycle step most helpful for improving the focus on global environmental benefits, followed by `Project Identification'. · `Government Endorsement' and ``GEF Focal Point Endorsement' were regarded as the least helpful activity cycle steps for improving the focus on global environmental benefits. · Among the open-ended suggestions to improve the GEF activity cycle are the need to `trim the bureaucracy', the adoption of a `shortened approval process of projects' and the need of further harmonization among the GEF and Implementing and Executing Agencies. · In a direct question about the strength of GEF's activity cycle and modalities, most participants selected `Good capacity levels for implementation' and `Appropriate scope of project goals and objectives'. These are directly linked to project preparation. So, even though it is the most resource- intensive step, `Project Preparation' is important for achieving well prepared projects. Modalities · All modalities were considered by most respondents as `highly relevant' to both GEF's mandate and countries' needs, except NCSA. · Over 70% of respondents deem Full-Size Projects to be `highly relevant'. 2 · A correlation of 0.99 between the responses to `Relevance of Modalities to GEF's Mandate' and `Relevance of Modalities to Countries' needs' clearly indicates that respondents perceive GEF's mandate as consistent to countries' needs. · The open-ended suggestions on new modalities included Programmatic Approaches, Technical Assistance, a decentralized MSP modality (with different financial range). · Enabling Activities and National Capacity Self-Assessment are favored more by National Governments than by the average across stakeholders. Medium-Size Projects · There is high recognition that MSPs pose the same implementing challenges as FSPs and require just as long to be approved as FSPs. · The vast majority of respondents think MSPs are an initial step of a broader process. · MSPs are favored most by National Governments and NGOs respondents. GEF Operational Principles · Full Disclosure is considered the Operational Principle that requires least resources, but at the same time contributes the least to the success of GEF projects. · Even through Incremental Cost is considered the most resource-intensive Principle, it ranks second as Principle contributing the least to GEF projects' success, with 27% of total responses. That indicates it is considered an inefficient Operational Principle. · Country Ownership is considered the principle that contributes the most to the potential success of GEF projects. GEF in Comparison to other International Agencies · Most respondents deem GEF projects to require more resources to prepare and approve than other international agencies, but have similar budget sizes and implementation periods. This finding is line with other questions where respondents showed concern with the amount of resources for project preparation and approval. · 56% of respondents think GEF projects are better prepared than those of other internationalagencies. · Even though the higher level of time and effort that is required to prepare a GEF project may lead to better prepared projects, 57% of respondents think GEF projects have the same level of success as projects by other international agencies. · There does not seem to be a direct relation between project preparedness and its likelihood of achieving significant impact, or its level of success. This raises the question about how much time and effort should be spent on project preparation in order to maximize efficiency, without losing the quality that directly influences project likelihood of success. 3 1. Introduction 1.1 Background The Global Environment Facility (GEF) provides new and additional funding to meet the agreed incremental costs of measures to achieve agreed global environmental benefits in six focal areas. The GEF functions as a mechanism for international cooperation, with three Implementing Agencies (UNEP, UNDP, World Bank), seven Executing Agencies (AfDB, ADB, EBRD, IADB, IFAD, FAO, UNIDO), and a Scientific and Technical Advisory Panel (STAP). In June 2005, the GEF council approved, as a special initiative, a Joint Evaluation of the GEF Activity cycle and Modalities. The need for this joint evaluation was identified during a process of consultation with GEF partners on monitoring and evaluation within the GEF. The objective of the evaluation was to review the experience in the programming and management of GEF activities and recommend improvements, with a view to supporting further simplification and standardization in GEF operations. Specifically, the evaluation aimed to: (a) Demonstrate the strengths and weaknesses in the GEF activity cycle and modalities and identify the contributing factors; (b) Identify and analyze the constraints which need to be addressed to improve the efficiency in GEF operations, including possible changes in procedures and systems; and (c) Provide recommendations to increase the efficiency and effectiveness of GEF operations and modalities. As per the overall Terms of Reference (TOR) and Evaluation Matrix, the evaluation addressed the following five key questions: (a) Is the GEF activity cycle efficient?; (b) Is the GEF activity cycle effective?; (c) Are the GEF modalities efficient?; (d) Are the GEF modalities effective?; and (e) Are the GEF modalities relevant? The evaluation was undertaken through eight components. 1.2 Component Eight: Stakeholder Survey Component 8 concerned a survey of country clients, GEF Focal Points, Agency partner HQ staff and field project staff, and other stakeholders to elicit views on experiences and challenges with regard to efficiency and modalities, and on the effectiveness of GEF's operational programming approach and strategy. The survey was designed in collaboration between the GEF Evaluation Office and the Evaluation and Oversight Unit of the United Nations Environmental Program (UNEP). The survey questions were based on the main issues regarding GEF Activity Cycle and Modalities that had already arisen in the analysis undertaken for Component 2 ­ Meta Analysis Evaluation and from Component 7 ­ Field Visits. The Joint Evaluation Management Group provided valuable comments on the design of survey, which were fully taken into account. Consultations were also held with experts on Survey from the World Bank and lessons learned from previous GEF surveys were considered. Preliminary results of the survey were presented in the May 2006 Joint Evaluation Meeting in Vienna. The methodology used in processing the data from the survey and the original questions can be found in Annex A. The survey was based on a web tool, Survey Monkey (www.surveymonkey.com), which allowed responses to be collected online and transferred to Excel. It encompassed both direct and open questions, which were grouped in five main sections: (a) activity cycle; (b) modalities; (c) operationalprinciples; (d) comparisons to other international agencies and (e) respondent's profile. Respondents were allowed to skip any question, and that is the reason why the number of total responses differs among questions. The Survey was broadly disseminated among GEF stakeholders electronically. The Joint Evaluation Management Group elaborated a strategy for distributing the survey, using the available GEF's and Implementing and Executing Agencies' e-mail lists, in order to broadly reach targeted stakeholders. The 4 survey was sent to approximately 2075 stakeholders and was responded by 660 of them. That is a response rate of 32%. This paper presents and analyzes the results of this survey. The responses are analyzed in an aggregated and disaggregated fashion. Sometimes, responses are broken down to highlight differences of opinions across stakeholder groups on views of their own and others' role. Section 1 presents the profile of the respondents, the grouping methodology used in this paper and the first question on `factor influencing decision not to work with the GEF'. Section 2, 3 and 4 present the main data and findings on the project cycle, modalities and operational principles, respectively. Section 5 presents data on `comparisons with other international agencies'. 1.3 Survey Respondents' Profile As mentioned above, the survey was completed by 660 respondents. The stakeholder group most represented in the survey responses is the Implementing Agencies: 60% of total responses (196 responses) came from this group. This is followed by National Governments (including State/Local Governments) that account for 16% of responses. These were the main target groups, since they are involved with the whole activity cycle. The Table 1 presents the stakeholder groups and the number of responses from each. Table 1: Survey Respondents' Profile Group Nš of Answers % of total Implementing Agency Staff 196 60% National Government 52 16% Others 36 11% STAP Roster Expert 28 9% Executing Agency Staff 27 8% GEF Operational Focal Point 26 8% International NGO 25 8% National/Local NGO 23 7% Convention National Focal Point 16 5% Private Sector 15 4% GEF Political FP 13 4% GEF Council Member 12 4% STAP Member 7 2% State/Local Government 4 1% GEF Secretariat 4 1% GEF Evaluation Office 3 1% Convention Secretariat 1 0.3% Total 327 100% However, around 50% of the total respondents skipped this question. The high number of skipped answers may be explained by the fact that this question was at the very end of the survey. 333 of respondents (around 55% of total respondents) indicated which Implementing or Executing Agencies they had experience with. For this question, respondents were allowed to choose more than one answer, since many respondents have had experience with different Agencies1. Over 75% of total respondents had experience with the United Nations Development Programme (UNDP) as the Implementing Agency, followed by the World Bank, with which over half of the respondents had had experience. Around 40% of respondents had experience with the United Nations Environment 5 Programme (UNEP). The most cited Executing Agencies with Expanded Opportunities was the United Nations Industrial Development Organization (UNIDO), with around 10% of the total responses, whilst the corresponding figure for the United National Food and Agriculture Organization (FAO) and the Asian Development Bank (ADB) was 8% of the total responses. Both the Inter-American Bank for Development (IADB) and the International Fund for Agricultural Development (IFAD) represented 5% of total responses. The African Development Bank (AfDB) was the Executing Agency with which respondents had least experience, having been cited in only 2% of total responses. Overall, the number of respondents who had experience with Implementing Agencies is almost three times higher than those who worked with Executing Agencies. See Figure 1. Figure 1: Respondents having experience with GEF projects by IA or ExA GEF Experience: Implementing or Executing Agencies 300 250 200 150 100 50 0 UNDP WB UNEP UNIDO FAO ADB IABD IFAD EBRD AfDB % 77% 52% 39% 10% 8% 8% 5% 5% 3% 2% Nš Respon 257 174 131 32 27 25 18 16 9 6 327 respondents (around 50% of total respondents) indicated what region they had experience with2. Once again, more than one alternative could be chosen. All world regions with which the GEF works were represented in the survey. As Figure 2 shows, 38% of respondents had experience with projects in Asia, 34% in Africa, 29% in Latin America and the Caribbean (LAC), 27% in Europe and Central Asia (ECA), and 19% with global projects. Figure 2: Regions with which respondents had experience GEF Experience: Region 150 100 50 0 Asia AFR LAC ECA Global % 38% 34% 29% 27% 19% Nš Respon 125 112 94 89 63 6 321 respondents (48% of total respondents) indicated what GEF Replenishment Period they had experience with. The vast majority of respondents, 93%, had experience with GEF-3 (2003-2006), which captures well the current views on the GEF activity cycle and modalities. 64% of respondents had experience with GEF-2 (1999-2002), while 33% with GEF-1 (1995-1998). 17% of respondents had experience with the Pilot Phase (1991-1994). The total number of responses is more than 100% because many respondents have had experience with more than one GEF phase. See Figure 3. Figure 3: GEF Phases with which respondents had experience Respondents' Experience: GEF Phases 400 300 200 100 0 GEF Phase 3 GEF Phase 2 GEF Phase 1 Pilot Phase (2003-2006) (1999-2002) (1995-1998) (1991-1994) % 93% 64% 33% 17% Nš Respon 300 204 107 53 333 of respondents (around 55% of total respondents) indicated what GEF modality they had experience with. Project Development Facilities, Full-Size Projects and Medium-Size Projects are the GEF Modalities with which respondents had most experience (72%, 71% and 70% of total respondents respectively). Around half of respondents had experience with Enabling Activities, while 42% worked with the Small Grants Program and 41% with the National Capacity Self-Assessment. 10% of respondents worked with other GEF Modalities. See Figure 4. Figure 4: Type of GEF Modalities with which respondents had experience Respondents' Experience: Type of GEF modalities 250 200 150 100 50 0 Full-sized Medium- Enabling Small Nat'l PDFs Others projects sized Activities Grants Capacity % 72% 71% 70% 49% 42% 41% 10% Nš Respon 240 238 234 164 140 136 34 7 1.4 Stakeholder Groups For the purpose of analysis, the 17 stakeholder groups have been condensed into 9 stakeholder groups, according to Table 2. The Figure 5 shows the number of responses per stakeholder group, according to this new regrouping. Before aggregating the data, all relevant differences in responses within these groups have been analyzed and are presented below. Table 2: Stakeholder Grouping Present Stakeholders Groups Subgroups aggregated NGO National and International NGOs National Government National, regional and local government Focal Points Political, Operational and Convention Focal Points STAP STAP Members and STAP Roster Experts GEF GEF Secretariat, GEF Evaluation Office, GEF Council members and Convention Secretariat Implementing Agency Implementing Agencies Executing Agency Executing Agencies Private Sector Private Sector Other Consultants, Academics, other donors, etc. Figure 5: Stakeholder Groups How do you identify yourself? 200 150 100 50 0 Implem. GEF/ Executing GEF(SEC, Private Agency Govern. Conv. NGO Others STAP Agency EO and Sector Staff Focal Staff Council) % 60% 17% 17% 15% 11% 11% 8% 6% 5% Nš Respon 196 56 55 48 36 35 27 20 15 Relevant differences in responses within aggregated stakeholder groups: · National and International NGOs ­ shows a difference in what step of the activity cycle help most in focusing projects on the delivery of Global Environmental Benefits (GEB) in projects: whereas International NGOs indicated project preparation, local NGOs selected project identification (which might be explained by the proximity of local NGOs to the projects beneficiaries and the perception that local benefits are intrinsically linked to GEB). · National NGOs also pointed out that they have to spend fewest resources on Identification step of the activity cycle. 8 · The Convention National Focal Points stated that the two least important activities for helping project focus on Global Environmental Benefits are: Review by other Implementing / Executing Agencies (26%) and Review by Convention Secretariat 13%. · 35% of respondents from the STAP Roster Experts contended that the activity where fewest resources are spent is on the STAP Review. · GEF SEC indicated that the fewest resources are spent on Convention Reviews. They also indicated that the least important activity to Global Environmental Benefits is `Project Start-up'. It should be noted, however, that this data consists of only 4 responses. 1.5 Reasons for not working with the GEF From the 660 respondents to this survey, 603 (91%) had already had some sort of experience with the GEF, while only 57 (9%) have not. 45 respondents (almost 80% of total respondents who did not have experience with the GEF) indicated that they did not have the chance to work with the GEF, either. Only 12 respondents (around 20% of the respondents who did not have experience with the GEF) stated that they had the chance to work with the GEF but opted not to. The table below shows the factors that influenced this decision of not working with the GEF: Table 3: Reasons for opting not to work with the GEF If you have had the opportunity to be involved with the development or implementation of GEF projects but have not done so what are the factors that influenced this decision? Rank Influencing Factors Nš Resp % total 1 Levels of co-financing required 6 35% 2 National or local circumstances 5 29% 3 Amount of time required for project design and approval 4 24% 4 Others 4 24% 5 Adaptive management 3 18% 6 Financial sustainability of project results 3 18% 7 Capacity levels for implementation 3 18% 8 Complexity or simplicity of project design 2 12% 9 Scope of project goals 2 12% 10 Timeframe needed successfully address environmental challenges 2 12% 11 Data availability for monitoring and evaluation 2 12% 12 Promotion and documentation of replication 2 12% 9 2. The Activity Cycle Regarding the Activity Cycle, respondents were asked to give feedback on (a) the steps3 in which most and least resources are spent; and (b) the most and least important steps for the delivery of global environmental benefits and the greatest strength and weakness of the GEF activity cycle (open question). For each direct question some comments are presented, followed by the general results table. The open questions are discussed in 2.2. 2.1 Direct Questions 2.1.1 What Step Of The Activity Cycle Requires Greatest Resources? 465 respondents indicated what activity cycle steps were most resource-intensive. They were allowed to choose more than one answer. `Project Preparation' stands out as the most resource-intensive step in the activity cycle step, with 302 (65%) of total responses for this question. This view is strongest among NGOs and the STAP, with 73% (35 responses) and 71% (20 responses) respectively4, whereas weakest amongst the GEF, with 47% (8 responses). The `amount of time required for project design and approval' also ranked third as an influencing factor for not working with the GEF, as shown in Section 1.3 above. See Box 1. The second most resource-intensive step is the `Review by the GEF Secretariat', with 111 valid responses, accounting for 24% of total responses. This second position, however, varies considerably among stakeholder groups. Among the Implementing Agencies and Focal Points, the `GEF pipeline entry' step ranked second, while among the National Government the `Implementing/ Executing Agency internal Box 1: Comments on Project Preparation: approval' step held this position. Several respondents aired their views and discontent about the Preparation step of the project cycle. Some of these comments were: `GEF Pipeline entry' (23%, 105 responses), `Project start- · Too complex format of proposals. up' (20%, 94 responses) and `Implementation Supervision' · Excessive documentation (20%, 93 responses) ranked third, fourth, fifth respectively, · Too many 'new terms' among most categories of respondents. · Complicated rules · Standardized and simplified presentation 10% of STAP members (3 responses) regard the 'Review by (template) requirements would avoid a lot a Scientific and Technical Advisory Panel (STAP) roster of disagreement over the 'readiness' of a member' as the most resource-intensive step, slightly less draft. than the aggregated responses across stakeholder groups · Too much paper generated. (12%). · Too much bureaucratic issues and work, for too little physical impact on the While no respondent from the GEF perceived either the ground 'Review by GEFSec' or 'CEO Endorsement/ Approval' as the steps requiring most resources, on average, 24% of respondents across the stakeholder groups regarded the 'Review by GEFSec', and 9% regarded the 'CEO Endorsement/ Approval' as the most resource- intensive steps. 9% of respondents from Implementing Agencies judge the 'Review by IA' to be the most resource- intensive step, whereas the average across stakeholder groups is 12%. Interestingly, a higher percentage of respondents from IAs (18%) deem the 'IA's internal approval process' to be the most resource-intensive than the average (16%). The same pattern is shown for 'Project Start-up' and 'Implementation Supervision' steps, where 22% of respondents from IAs deem it to be the most resource-intensive step, whereas the average is 20%. 10 11% of respondents from Executing Agencies, compared to an average across stakeholder groups of 12%, deem the 'Review by other IA/ExA' as the most resource-intensive step. Interestingly, almost twice as many ExA respondents (26% of total responses from ExA respondents) regard the 'IA/ExA internal approval' as the most resource-intensive step than the average (16%). However, only 11% of EA respondents consider 'Project Start-up' to require greatest resources, as opposed to an average of 20% across stakeholder groups. Whereas the average of respondents across stakeholders who consider the `GEF Focal Point endorsement' step to require greatest resource is 11%, only 6% of Focal Point respondents chose this option. The percentage of Focal Point respondents who consider 'Government Endorsement' as the most resource- intensive step was half of the average across stakeholder groups (18%). Table 4: Activity Cycle steps requiring greatest resources Which part of the process requires the GREATEST amount of time and resources? Rank Activity Cycle Step Nš Resp5 % of total 1 Project preparation 302 65% 2 Review by GEF Secretariat 111 24% 3 GEF pipeline entry 105 23% 4 Project start-up 94 20% 5 Implementation supervision 93 20% 6 Government endorsement 85 18% 7 Project identification 83 18% 8 Implementing Agency / Executing Agency internal approval 74 16% 9 Submission to GEF Work Program 60 13% 10 Review by a STAP roster member 58 12% 11 Review by other Implementing Agencies/Executing Agencies 55 12% 12 Work Program approval 55 12% 13 GEF Focal Point endorsement 52 11% 14 CEO Endorsement / CEO Approval (for MSPs) 43 9% 15 Review by Convention Secretariats 31 7% 2.1.2 What Step Of The Activity Cycle Requires Least Resources? 441 respondents indicated what activity cycle steps were most resource-intensive. They were allowed to choose more than one answer. `GEF Focal Point Endorsement' is regarded by 24% of total respondents (105 responses) as the activity cycle phase requiring least resources. As expected, this Box 2: Comments on STAP: view is widely held among Focal Points respondents, with 24 Even though the review by the STAP ranked responses or 46% of total FP members, and National Government third as the least resource-intensive project officials, with 22 responses or 43% of total national government cycle stage, it raised some very positive members. comments from respondents: · Review processes with people who are The `Project Identification' step ranks second, with 94 responses, experienced in the topic and have knowledge of other experiences or 21% of total. 21% (38 responses) of Implementing Agencies worldwide and are therefore able to guide respondents and 23% (11 responses) of NGOs respondents the project elaboration & work planning consider this step to be the least resource-intensive. No GEF process. respondent, however, regarded this step as the least resource- · The STAP review gives perspectives from intensive. many angles. 11 31% of STAP respondents (10 responses) regard the 'Review by a Scientific and Technical Advisory Panel (STAP) roster member' as the least resource-intensive step, a percentage about three times higher than the average across stakeholder groups (11%). See Box 2. While no GEFSec respondent regarded the 'Review by GEFSec' nor 'CEO Endorsement/ Approval' as the steps requiring least resources, on average across the stakeholder groups, 5% of respondents pointed the 'Review by GEFSec', and 4% pointed the 'CEO Endorsement/ Approval' as the least resource-intensive steps. 3% of IAs respondents, compared to an average across stakeholder groups of 2%, consider the 'Review by IA' the least resource-intensive step. As to the 'IA's internal approval' and 'Project Start-up' steps, there is no difference between the average across stakeholder groups and IAs respondents, both with 5% and 4% respectively. Twice as many IA respondents (5% of total IAs respondents) than the average (2.5%) consider 'Implementation Supervision' to be the least resource-intensive step than the average across stakeholder groups. No ExA respondent chose the 'Review by other IA/ExA', 'IA/ ExA internal approval', nor 'Project Start- up' to be the least resource-intensive step. However, an expressive 8% of ExA respondents (2 responses) deem 'Project Implementation Supervision' to require least resources. The 'GEF FP Endorsement' step was chosen by an average of 24% respondents as the least resource- intensive step (105 responses), whereas among Focal Points it was almost the double, 46% (24 responses). However, regarding the 'Government Endorsement' step, the situation is different. While the average was 10% (43 responses), only 6% of Focal Point respondents selected this step as the least resource-intensive (3 responses). Table 5: Activity Cycle steps requiring least resources Which part of the process requires the LEAST amount of time and resources? Rank Activity Cycle Step Nš Resp6 % of total 1 GEF Focal Point endorsement 105 24% 2 Project identification 94 21% 3 Review by a STAP roster member 50 11% 4 Government endorsement 43 10% 5 Implementing Agency / Executing Agency internal approval 23 5% 6 Review by GEF Secretariat 20 5% 7 CEO Endorsement / CEO Approval (for MSPs) 19 4% 8 Project start-up 17 4% 9 GEF pipeline entry 15 3% 10 Review by Convention Secretariats 13 3% 11 Implementation supervision 11 3% 12 Review by other Implementing Agencies/Executing Agencies 10 2% 13 Submission to GEF Work Program 9 2% 14 Work Program approval 9 2% 15 Project preparation 3 1% The results demonstrate the need to look at the activity cycle phases in a disaggregated manner, taking into account the steps that make up a given Phase. Activity Cycle Phase 4 ­ Project Approval and Start- up, for instance, is made up of various steps, such as Focal Point Endorsement, Work Program Approval and Government Endorsement, some of which are regarded as not resource-intensive, such as Focal Point Endorsement. Project Preparation, on the other hand, is considered a phase on its own. 12 2.1.3 What Are The Most Important Activity Cycle Steps For Improving The Focus On Global Environmental Benefits? For this question, respondents were asked to choose a first and second option. An average of 415 respondents7 indicated what activity cycle step is the most helpful for improving the focus on global environmental benefits. The `Project Preparation' step, with 232 responses8, or 28% of total responses across stakeholder groups, and the `Project Identification' step, with 188 total responses, or 23% of total, were regarded as the two most important activity cycle steps for improving the focus on global environmental benefits, according to most respondents. These were the two steps singled Box 3: Further Comments on Project Preparation: The rigorous project preparation was pointed out by several out by most stakeholders groups, except for the respondents as the greatest strength of the GEF. Executing agencies and the STAP, who suggest · Projects have a high quality once they have been through the Review by STAP to be the most important the whole project cycle step in this regard. See Box 3. · High standards for GEF Projects preparation ensures technical and operational quality entry and during The `Review by GEF Secretariat' step ranks implementation. third among total respondents, having 90 · Long process ensures that project is formulated according responses, or 11% of the total. The to sound and solid procedures Implementation Supervision and Review by · It provides a mechanism for a thorough process of STAP followed, with 7% (60 responses) and 6% preparing projects (50 responses) respectively. · Design process is participatory and 'Community-driven' The `Project Preparation' step ranks first both as the most resource-intensive step and the most important for improving the focus on global environmental benefits. The `Project Identification' step, in turn, ranks as the second most important step for improving the focus on global environmental benefits, but 7th as the most resource-intensive activity. 29% of STAP Respondents (19 responses) regard the 'Review by STAP' as the most helpful step for improving the focus on global benefits, whereas the general average was 6%, more than five times smaller. While the average percentage of respondents across stakeholder groups who deem the `Review by GEFSec' as the most important step for improving the focus on global environmental benefits was 11%, among GEF respondent only one person (3% of totals GEF respondents) regards it so. Only 1% of IAs respondents consider the 'Review by other IA's' to be the most helpful step for improving the focus on global benefits, whereas the average across stakeholder groups was 3%. Regarding 'IA Approval', 'Project Start-up' and 'Implementation Supervision', there was no difference between the average values and the perception of the IAs respondents. 4% of ExA respondents regard the 'Review by other IA/ExA' to be the most helpful step to improve the focus on global benefits, as opposed to an average across stakeholder groups of 3%. Interestingly, no ExA respondent chose the IA/ExA approval step as the most helpful for improving the focus on global benefits. Table 6: Activity Cycle Steps most helpful for improving the focus on global environmental benefits Activity cycle steps ­ Most helpful for improving focus on global environmental benefits? Rank Activity cycle steps Nš Resp9 % of total 1 Project preparation 232 28% 13 2 Project identification 188 23% 3 Review by GEF Secretariat 90 11% 4 Implementation Supervision 60 7% 5 Review by STAP 50 6% 6 GEF pipeline entry 49 6% 7 Government Endorsement 29 3% 8 GEF Focal Point endorsement 29 3% 9 Work Program Approval 23 3% 10 Review by other IAs/ EAs 21 3% 11 Review by Convention Secretariats 20 2% 12 Project start-up 14 2% 13 Submission to GEF Work Program 13 2% 14 CEO Endorsement / CEO Approval (for MSPs) 6 1% 15 IA/EA Approval 6 1% 2.1.4 What Are The Least Important Activity Cycle Steps For Improving The Focus On Environmental Benefits? For this question, respondents had to choose a first and second option. An average of 385 respondents10 indicated what activity cycle step is the least helpful for improving the focus on global environmental benefits. 151 respondents, 20% of total responses, indicated `Government Endorsement' as the least important activity cycle step for improving the focus on global environmental benefits. The second least important step is the `GEF Focal Point Endorsement', selected by 107 respondents, 14% of total Box 4: Comments on the Review Process: responses across stakeholder groups. Both responses One of the most criticized aspects of the GEF's indicate concern regarding the role the Focal Points project cycle is the high quantity of reviews a project plays in contributing to the quality of the project. has to go through. · Too many bodies to review The `Review by other IA's/ ExA's' step ranked third, · A project can end up being reviewed and amended by different stakeholders, agencies, with 88 responses, 11% of the total responses, while STAP, GEF SEC and then Council, to the point `GEF Pipeline Entry' came in fourth place, with 54 where it needs to be re-written 6 or 7 times or responses, 7% of the total. even more (often with contradictory reviews). · Sometimes projects are forced to accommodate Twice as many STAP Respondents (6% of total or 4 an exhaustive number of parameters to satisfy all responses) regard the 'Review by STAP' as the least concerns leading to unwieldy or unrealistic goals helpful step for improving the focus on of global and project monitoring. benefits than the average (3%). · GEF SEC is a bottleneck due to limited technical staff availability While no GEF respondent selected the 'Review by GEFSec' nor 'CEO Endorsement/ Approval' as the least helpful step to for improving the focus on global benefits, in average, 4% of respondents pointed the 'Review by GEFSec', and 5% pointed the 'CEO Endorsement/ Approval' as the least helpful step to the delivery of global benefits. See Box 4. 17% of ExAs respondents consider the 'Review by other IA/ExA' the least helpful for delivery of global benefits, while the average across stakeholder groups is 11%. Interestingly, a higher percentage of ExA members, 11%, consider the IA/ExA approval to be the least helpful step, while the average is 7% across stakeholder groups. 4% of ExA members deem 'Project Start-up' and 'implementation supervision' to be the least helpful for delivery of global benefits, whereas the average is 8% and 4% respectively. 14 While the average across stakeholder groups considering the 'Government Endorsement' step as the least helpful for improving the focus on global environmental benefits was 20%, the rate was only 13% among ExAs respondents. Table 7: Activity Cycle Steps least helpful for improving the focus on global environmental benefits Activity cycle steps - LEAST helpful for improving focus of global environmental benefits? Rank Activity cycle steps Nš Resp % of total 1 Government Endorsement 151 20% 2 GEF Focal Point endorsement 107 14% 3 Review by other IAs/EAs 88 11% 4 Project start-up 61 8% 5 GEF pipeline entry 54 7% 6 IA/EA Approval 54 7% 7 Review by Convention Secretariats 49 6% 8 Submission to GEF Work Program 39 5% 9 CEO Endorsement / CEO Approval (for MSPs) 35 5% 10 Implementation Supervision 28 4% 11 Review by GEF Secretariat 27 4% 12 Work Program Approval 27 4% 13 Review by STAP 24 3% 14 Project identification 19 2% 15 Project preparation 6 1% 2.2 Open Questions on the Project Cycle Respondents were asked to answer three open questions: (a) What is the greatest strength of the GEF activity cycle and; (b) What is the greatest weakness of the GEF activity cycle?; and (c) Do you have any suggestions on how the GEF project cycle could be improved? These answers provide a rich and useful insight into the wide range of opinions about GEF's project cycle. These insights have been aggregated into common categories and are presented below. 2.2.1 What Is The Greatest Strength Of The GEF Activity Cycle? The responses to this question were grouped according to the operational principle or activity cycle step the response best fits into. 303 respondents gave their open feedback to this question. The operational principle or activity cycle step cited is presented in order of frequency of responses, followed by quotations made by respondents. Some responses also mentioned some specific tools used by the GEF as strength, while others emphasized the partnership nature of the GEF as one of its strengths. Table 8: Greatest strengths in the GEF Activity Cycle Steps and Operational Principles Rank Responses (Operational Principles and Activity Cycle Steps) Nš Resp11 Phase 3 ­ Appraisal 1 · Rigorous application of eligibility criteria · 40 Reviews by the GEF SEC, STAP, other IAs/ EAs and Convention Secretariats · Highly qualified knowledge from GEF Incremental costs for global environmental benefits 2 · Focus on global benefits · 29 Funding of increment / co-financing Requirements · It comprises all the environmental areas (water, biodiversity, etc.) 15 Rank Responses (Operational Principles and Activity Cycle Steps) Nš Resp11 Country ownership · High degree of country ownership 3 · 26 National/ Local technical support · Government endorsement Public participation 4 · 24 Stakeholder inclusion in project identification and preparation Cost-effectiveness 5 · Rigor on technical quality assurance 24 · Similarity to IA's project cycle (which reduce costs) Phase 2 ­ Preparation 6 · 17 Availability of PDF resources Catalytic role · Catalytic Aspect and leverage 7 · Sustainability and continuity 16 · Focus on impact, leverage and replication · Synergy and reliability Tools · Clear guidance in steps and project documents with examples (content, format, 8 etc.) 14 · Incremental cost analysis · sharp log frame analysis Phase 1 ­ Concept 9 · Project Identification 9 · Good, prompt, practical GEFSEC reviews at pipeline entry Flexibility · Quick response to project idea 10 · 7 Control of the allocated funds · Quick delivery of funds as compared to other donor Phase 5 ­ Implementation 11 · 6 Implementation supervision Coordination / Partnership 12 · Coordination among IA's and EA's dealing with environmental projects 6 · `The fact that the GEF is a partnership' Full disclosure 13 · Raises awareness about the GEF · 3 Transparency · Well-designed website Monitoring and evaluation 14 · Result oriented management of project/ Focus on outcomes 5 · Monitoring and evaluation tools in all GEF eligible countries Phase 4 - Approval/ Start 15 · 1 Work Program Approval 16 Box 5: Strengths of the GEF Activity Cycle · The GEF projects create a framework for cooperation in the country between national and international institutions. The GEF funds work as seed money for leveraging funds for addressing emerging challenges. The projects initiated by GEF have high rating and have strong ownership background. The funds provided by GEF increase the position of environmental benefits in shaping national development strategies. · The PDF phase provides an important opportunity for developing partnerships and identifying strengths and weaknesses of potential partner agencies. The identification Phase is also an important point in the cycle which has been strengthened over the years. · So-called 'project briefs' in fact turn out to be detailed project documents. Very little project preparation work is required after the 'project brief' is prepared. · Review processes with people who are experienced in the topic and have knowledge of other experiences worldwide and are therefore able to guide the project elaboration & work planning process. The GEF Regional coordinators are usually extraordinarily helpful and supportive and help in problem solving and technical guidance and act as good sounding boards. · The greatest strength lays in project identification and preparation. Though that is a very complicated and lengthy process, it allows to discuss the idea in detail, from all perspectives, consider the initiative taking into account potential risks identify and set out milestones, indicators of success, etc. · In the project preparation, the following elements are crucial for the success and the impact of the initiative: sense of ownership, identification of the real problems, mobilization and alignment of the required resources, partnerships and capacities for the implementation of the initiative. There lies the greatest strength of the GEF activity cycle. · The activity cycle encourages the proponent to carry out a systematic baseline analysis and to identify synergistic relationship with other relevant ongoing initiatives. 2.2.2 What Is The Greatest Weakness Of The GEF Activity Cycle? 305 respondents gave their open feedback on the weaknesses of the GEF activity cycle. The responses are aggregated in order of frequency, along with some quotations from respondents. Table 9: Greatest weaknesses in the GEF Activity Cycle Steps and Operational Principles Rank Responses (Operational Principles and Activity Cycle Steps) Nš Resp12 Tools · The length and excessive number of procedures/steps · Too complex format of proposals and excessive documentation 1 · 96 GEF Bureaucracy · Changes in GEF guidelines and formats · Too much reporting Phase 3 ­ Appraisal · Excessive reviews 2 · 27 GEF SEC poor reviews of projects · Lack of attention to other IA's reviews Cost-effectiveness · Lack of harmonization with the cycles of IA and ExA · High administrative costs for project preparation and lack of compensation for IA's and ExA's 3 · More focus on process than result 27 · Too much attention to single projects (instead of strategic advice to IA's ExA's) · Gaps between the projects' cycle phases · The limited amounts that are approved for projects · Limited timeframe of projects 17 Rank Responses (Operational Principles and Activity Cycle Steps) Nš Resp12 Full disclosure · Lack of transparency in GEF SEC decision making 4 · 24 Communications among all stakeholders is fragile · Lack of transparency in allocation of resources Phase 4 - Approval/ Start · Too many approvals needed 5 · 21 Work Program Approval · Convention secretariat endorsement Coordination / Partnership · Competition and lack of coordination between the different GEF implementers at country level · Uncertainties about replenishment · Too many people involved · Lack of competition between IAs /ExAs 6 · Pipeline Entry and Work Program Inclusion can become politicized 16 · IA's and ExA's dependency on GEF budget · Unclear council structure and government representation · Too many vested interests at all levels: IAs, GEF Sec staff, government focal points, etc. · Too much 'turf protection' · Too restricted access to GEF Incremental costs for global environmental benefits 7 · Co-finance requirements 11 · Project template does not contribute to the global environmental benefits Country ownership · Difficulty to get focal point endorsement and lack of clarity as to its usefulness · Exclusion of National benefits and impacts on poverty 8 · 8 The distance between the Regional Focal Point with the Countries Offices · The projects are not adapted to the reality of the country areas they intend to develop. Phase 5 ­ Implementation 9 · Implementation Supervision 8 · Insufficient resources for Project Implementation and Evaluation Monitoring and evaluation 10 · Inadequate Monitoring and impact evaluation, including ex-post. 8 · Lessons learned are not considered Phase 1 ­ Concept 11 · 5 Project Identification - too ad hoc; too open to personal judgment Catalytic role 12 · 3 Inhibits innovation Flexibility 13 · Limitations on delegation of authority 3 · Rigid management procedures for complex consultative processes Phase 2 ­ Preparation 14 · Too rigorous project preparation · 2 Inability to assure that all IAs/ ExAs have carried out necessary due diligence in project preparation and implementation, Public participation 15 · 1 Lack of stakeholders participation 18 Box 6: Weaknesses of GEF Activity Cycle · The system very often fails to recognize priorities giving more importance to a 'perceived' sense of political fairness or justice under which every country or client should get a little bit. It would be more transparent to focus on what the impact can be the highest even if this means putting all resources in one area or country. · The duplication in terminology (is a PDF B documents the same as a Concept Note? Is a GEF Project Brief the same as a PAD? Is work program entry the same as effectiveness? Is a Facility the same as a Fund? Is a Council the same as a Board?). In addition, there should be a simplified order of activities, in the format: - Write Concept Note - Submit to Regional Coordinator (RC) - Make changes per RC's suggestions - Make final submission (for pipeline entry / preparation) - etc. As an example of where it gets complicated, the STAP review directions, including TORs and questionnaire, span 11 pages!! It could be simplified to: - locate an expert on the roster - using available sample TORs, prepare TORs for your reviewer - contact reviewer and determine his/her availability, and contractual requirements. · Coordination with other IAs and EAs as UNEP, UNDP, ADB and others which rely on this money to pay for their internal staff and consultant. They may be too dependant on GEF budget. There may be a conflict of interest as are they doing these projects for GEF or to keep their staff employed? · So many steps that teams often get frustrated and want to give up. · The length and excessive number of procedures/steps within the project cycle is increasingly making GEF financing irrelevant. It takes so long to move from identification to implementation that project design can be outdated when it finally starts or the proponents have succeeded in finding an alternative and speedier source of funding. · Too many review points. A project can end up being reviewed and amended by different stakeholders, agencies, STAP, GEFSec and then Council to the point where it needs to be re- written 6 or 7 times or even more (often with contradictory reviews). More on-going contact with GEFSec 'Gate-keepers' during the development process would help to ensure strong eligibility by the time of submission. Standardized and simplified presentation (template) requirements would also avoid a lot of disagreement over the 'readiness' of a draft. 2.2.3 How could the GEF activity cycle be improved? Respondent's comments ranges from improving the tools used in the Project Cycle to the role of parties involved in the activity cycle. Below is a summary of the main suggestions. · More value added from GEF staff. Cut the paper work and transaction costs. · More flexibility in defining incrementality. · Greater attention to framework conditions (damaging subsidies, securing local support (not just formal national), securing income flows and local financial markets). · More attention to leveraging (local governments, private sector, not just IBRD & ODA). · Stricter application of 'global benefits' & incremental costs criteria by IAs/ ExAs before projects enter the pipeline, so as to reduce the need for later screening. · Streamline pipeline entry and discussions with GEF Sec and put a penalty on any GEF Sec person who acts in the interest of his/her original organization to the detriment of another IA. · Go to a shortened approval process and drop the Council approval and drop the STAP review - useful but if Bank staff and management are doing their job, not needed. · The role and effectiveness of the three IA needs an in-depth review. There is too much waste of resources in the present implementation/execution modality. · Trim back the bureaucracy. I was a team leader for one of the first GEF projects in the early 90s. Compared to that early experience, I hardly recognize the GEF today -- it's excessively bureaucratic, pseudo-scientific, and opaque. While other international organizations are taking steps to simplify and harmonize their assistance, the GEF is moving in the opposite direction. 19 · To be most effective, the idea of the concept in setting forth the entire outline of a project from inception to formulation and preparation to execution (all of which are separate steps in the GEF process) should be more strictly adopted, and not replicated at each stage. Before the GEF funds a project, knowledge of the entire scheme being proposed is of great benefit in scheduling the flow of funds. Repeated updating or refinement of the concept papers serves little purpose other than to replicate requirements that are already included in the PDF and project documents. · Quicker public dissemination of GEF supported opportunities to Parties. Each Party has special circumstances and needs that should be taken into consideration by the GEF by supporting greater flexibility to Parties to customize activities and budgets within projects during the project development stage. · GEF council should not comment on each and every project, this does not add value and is a source for huge delays. GEFSEC should rule on eligibility and not on general project design. · Executive Summaries (project briefs) should be shortened, and be more in line with adaptive management (using a result based management approach). 3. GEF Modalities The second part of the survey aimed at obtaining perception in relation to the different GEF modalities, more specifically on Full-Size Projects (FSP), Medium-Size Projects (MSP), Small Grants Program (SGP), Enabling Activities (EA), National Capacity Self-Assessment (NCSA) and Project Development Facilities (PDFs). Around 60% of total survey respondents (400 responses) of total respondents answered 13 questions concerning GEF's modalities. 3.1 Modality Relevance To GEF's Mandate And Countries' Needs The first and second questions, which refer to the relevance of each GEF modality, need to be analyzed together. While the first questions inquiries into the relevance of each modality to `GEF's mandate and objective of funding global environmental benefits and acting as the financial mechanisms for multi- lateral environmental agreements', the second one addresses the relevance of modalities to `the needs and priorities of GEF partner countries'. A correlation of 0.99 between the responses to these two questions, indicates that respondents perceive GEF's mandate as consistent to countries' needs. Figure 6 shows the 381 responses to the first question: `How relevant is each modality to the GEF's mandate and objective of funding global environmental benefits and acting as the financial mechanisms for multi-lateral environmental agreements?' 20 Figure 6: Modalities' relevance to GEF's mandate Modalities' relevance to GEF's mandate 80% 70% 60% Highly relevant 50% Moderately 40% relevant Not relevant 30% 20% Not sure 10% 0% FSP MSP Small Enabling NCSA PDFs Grants Activities Respondents judged most modalities to be `highly relevant'. Full-Size projects, Medium-Size Projects and PDFs received most `highly relevant' ratings. The Small Grants Program and the Enabling Activities were also mostly ranked as `highly relevant', but with a smaller difference than the former categories. The National Capacity Self-Assessment was deemed mostly as `moderately relevant'. Lack of awareness was greatest in regard to National Capacity Self-Assessment, followed by Enabling Activities and Small Grants. Full-size projects received most ratings as `highly relevant': over 70% (268 responses) of total respondents across stakeholder groups regard FSP as highly relevant, while 22% (84 responses) see them as `moderately relevant' and 2% (8 responses) as `not relevant'. The Medium-Size Projects were rated as `highly relevant' by over 60% (229 responses) of respondents; while 32% (122 responses) judge them to be `moderately relevant' and only 3% (11 responses) regard them as `not relevant'. PDFs, in turn, were regarded as `highly relevant' by 56% (213 responses) of total respondents, `moderately relevant' by 33% (126 responses) and not relevant by 5% of total respondents (19 responses). FSPs rank as 'Highly Relevant' among 73% of Implementing Agencies respondents and among 75% of National Governments respondents, above the 72% average across stakeholder groups. National Government members attach more importance to MSPs: they were ranked as 'Highly Relevant' by 58% of average respondents across stakeholder groups and by 68% of National Governments respondents. No National Government respondent selected MSPs as 'Not relevant', while the average was 2% across stakeholder groups. In regard to PDFs, there are not major differences across stakeholder groups, with the vast majority of respondents regarding this modality as moderately or highly relevant. The Small Grants Program was deemed `highly relevant' by 46% of respondents across stakeholder groups, `moderately relevant' by 35% and not relevant by 10%. The Enabling Activities were ranked as `highly relevant' by 45% of respondents, `moderately relevant' by 38% and `not relevant' by 7%. The Enabling Activities were considered 'Highly Relevant' by a substantially larger share of National Government members: 66%, while the average across all stakeholder groups was 45%. At the same time, 21 far fewer national government respondents considered Enabling Activities to be 'not relevant', when compared to the average across all stakeholder groups. The only modality for which the number of respondents selecting `moderately relevant' outnumbered the number selecting `highly relevant', by 4%, was the National Capacity Self-Assessment. This was also the modality that received the greatest number of `not relevant' ratings and about which the greatest lack of awareness was shown. 17% of respondents across all stakeholder groups deemed this modality not to be important, and 11% were not sure what it is. Like the SGP, this modality was also more valued by National Governments respondents: while an average of 39% across al stakeholder groups consider it to be 'highly relevant', 46% of National Government members attached 'high relevance' to it. Figure 7 presents the 381 answers to the second question, which reads: `How relevant is each modality to the needs and priorities of GEF partner countries?' Figure 7: Modalities' relevance to GEF's partner countries Modalities' relevance to GEF partner countries 80% 70% 60% Highly relevant 50% Moderately relevant 40% Not relevant 30% 20% Not sure 10% 0% FSP MSP Small Enabling NCSA PDFs Grants Activities As noticed above, there is a high correlation between this and the previous graph. Some points worth noting are: · All categories were mostly rated as `highly relevant', which may show that the National Capacity Self-Assessment Modality is indeed an important modality for GEF partner countries' needs. · Lack of awareness or familiarity still persists with regard to NCSA and Enabling Activities, with 11% of total responses for both. · Regarding the Small Grants Program, a larger share of respondents deem it be `high relevant' for the GEF partner countries than to the GEF mandate, which may reflect the higher importance of the SGP for countries. 22 Box 7: Strengths of GEF Modalities · Not a one-size fits all approach - it can be tailored to the needs. · Flexibility in size of operations, from small grants to full-sized projects. · The fact that the funds are free so countries have an incentive to invest in global environmental benefits although these may not appear to be priorities on their short-term national agendas. · The full and medium sized projects are highly relevant where they enable national agencies and local communities to play a key role. This supports national capacity building efforts. · They allow for a variety of proponents to be able to participate and co-finance projects, and PDFs contribute to better preparation of projects. If it were only FSPs, participation would be reduced and only for governmental agencies. · Small Grants Program allows the GEF to reach a multitude of community groups and NGOs, which would otherwise not be touched by the GEF. The full sized projects, especially when linked to investment projects, can generate both local and global benefits. · Independent funding of activities that can address global environmental issues gives a large leverage to address overlooked environmental concerns and can look at long-term issues. In blended projects, it provides large scope of integrating environmental issues within the development agenda, which is great! · The project preparation funding is much more valuable than the project grant itself, since this will influence the project design in a phase where funding is hard to get. · Enabling activities and National Capacity Self-Assessment can help governments to identify their priorities with regard to the environment sector; not only the global environment and their Convention obligations, but the overall links with development. · PDFs are excellent to bring stakeholders on board and develop good proposals. · Medium and small projects that can be implemented with minimal bureaucracy and through NGOs and other smaller, directly responsible agencies closer to the 'field'. Quick decision making and a transfer of responsibility for assessment and accountability eases the burdens on GEF core staff. · Enabling Activities: These have provided the country with an opportunity to ensure that it fulfills its obligations under the conventions to which it is signatory. Small Grants Program: This provides UNDP with the only avenue through which it can support environmental initiatives at the community / grassroots level. Project Development Facility (PDFs): This modality is essential to develop proposals that meet all the GEF requirements. These have made it possible for the country to systematically identify critical issues to be addressed by the project in a systematic and highly consultative manner. NCSA: The less stringent co-financing requirement helps the country make timely proposal submissions. MSPs: The expedited process that this modality offers enables the country to act on its global environmental management problems in a timely manner. FSPs: These offer the time and resources required to address the root causes of the environmental problems that the country contends with. In addition, FSPs more effectively support policy and institutional interventions. 23 Box 8: Weaknesses of GEF Modalities · Excessive emphasis on free-standing capacity building projects. Limited options and too concentrated on the project idea. The GEF needs to think outside the box, longer term approaches and more strategic. · GEF is too `project focused'. The current piloting of Country Partnership Programs (and other longer termed programmatic approaches) is a step in the right direction in adding another interesting modality to the range of options available. · GEF doesn't respond to private sector needs. · It is as complicated and time consuming to process small things as big ones. Many project managers now do not want to bother with MSPs. The PDF or preparation money is often very problematic as it is often hard to internalize it in various countries. Getting matching funds or trying to blend is also sometimes very problematic. Another big headache with smaller projects like MSPs is that one receives very small supervision budgets yet the supervision demands may be equal to those of larger projects. · Too many financial and human resources are devoted to enabling activities and capacity self-assessments that have little or no impact on countries' development strategies and action plans. · Inadequate resources to fund projects and the time-lag between completion of PDF-B and start-up of the full project. · This range of modality options is not really available to World Bank staff, since MSPs are discouraged, SGPs are non-existent, enabling activities quite hap-hazard, and full-size projects are almost as hap-hazard under the current changes (RAF). · PDF provides a good basis for sound project preparation. Nevertheless we can observe a lack of ownership during the preparatory phase: international and local consultants are fully in charge of the project, lack of government contribution. In most cases the international consultant is the key person for the project document. · PDF resources are usually spent in things that are needed to develop a project, but that not necessarily are useful should the project not take off. It would be very nice if each component of a PDF could also become a stand alone result. · Full-sized projects are sometimes too ambitious in scope. Follow up on implementation is spotty. National communications are many times too oriented towards the mobilization of specialized consultants and not enough capacity building for sustainability of efforts. 3.2 Medium-Size Projects: Strengths and Weaknesses Figure 8 shows the position of 320 respondents in regard to seven statements about MSP's strengths. Most agreement was expressed for the statements `MSPs can serve as an initial step in a longer process', with 85% (272 respondents) agreeing with it; and `MSPs are effective in building capacity among project partners', with almost 80% (250 respondents) of total respondents' agreement. There was also agreement with most other statements, but with varying degrees. The only exception is the statement `MSPs require less capacity to implement than FSPs', with which most respondents, almost 55% (169 respondents), disagree, showing that there is high recognition that MSPs pose the same implementing challenges as FSPs. The greatest uncertainty was shown over the statement `MSPs are most cost-effective', with 18% of total responses (57 respondents), followed by MSPs are more flexible than FSPs, with 15% (47 respondents). The proportion of National Government respondents who agree that `MSPs are more flexible than FSPs', at 49% (26 respondents), was almost 20% lower than the average across all stakeholder groups, at 58% (185 respondents). However, among NGOs respondents, there was an expressive 68% (32 respondents) of respondents in agreement with the statement that MSPs are more flexible than FSPs. 24 Figure 8: Strengths of Medium Size Projects Strengths: Medium-sized projects. 90% 80% 70% Agree 60% 50% Disagree 40% Don't know 30% 20% 10% 0% MSPs are MSPs MSPs as an Broader MSPs brings MSPs are MSPs builds more flexible require less initial step stakeholders co-financing more cost- capacity than FSPs capacity to participation effective effecitvely implement than FSPs On the statement that `MSPs require less capacity to implement than FSPs', there was almost 20% more Implementing Agencies respondents in disagreement than the average members than the average across all stakeholder groups. However, among National Governments respondents there was over 25% more in agreement with the statement than the average. Regarding the statement that `MSPs are an initial step of a broader process', opinions were similar among most stakeholders, with a high degree (85% in the average ­ 272 respondents) of agreement. In regard to the statement that `MSPs broaden stakeholder participation' there was no notable difference of opinion across stakeholder groups. As to the statement that `MSPs are capable of bringing more co-financing', there was 25% more agreement among NGOs respondents (74% - 35 respondents) than in the average across stakeholder groups (59% - 189 respondents). And in regard to the statement that `MSPs are more cost-effective', there were different opinions among stakeholders. While among National Government respondents the agreement rate to this statement was 38% (20 respondents), among NGOs members it was 57% (27 respondents). The average across all stakeholder groups fell in between, at 45% (144 respondents). There was a high rate of agreement to the statement that `MSPs can build capacity effectively', 78% (250 respondents) in the average As a conclusion, it can be observed that MSPs are better favored by NGOs, while Implementing Agencies and National Governments show mixed opinions about different aspects of MSPs. Most respondents, 75% of total (238 respondents), agree that `MSPs require just as much supervision as FSPs', even though projects are smaller. Most respondents, around 60% (190 respondents), also agree that `MSPs take just as long to be approved as FSPs'. On the other hand, most respondents, about 60% (190 respondents), disagree that `MSPs are not as sustainable as FSPs', while 65% (206 respondents) of total respondents do not agree that `It is difficult to obtain endorsement for MSPs'. See Figure 9and Box 9. 25 Box 9: Strengths and Weaknesses of MSPs Strengths: · MSPs do not have to be approved by GEF Council. · MSPs can be more innovative and experimental. · MSPs allow faster response to private sector interested parties. · Less time for project design and implementation. · MSPs require less counterpart investment, which is often hard to deliver. · More rapid and more manageable with fewer opportunities for corruption. · MSPs can be (and have been) better suited to NGO involvement. · MSPs allows a wider spectrum of geographic and activity coverage. Weaknesses: · MSPs have too high of transaction costs to be cost-effective. · $1 million becomes the budget regardless of objectives and activities. · Administrative burden. · Lack of sufficient preparation and supervision budget for MSPs. · Some UN agencies view MSPs are opportunities for budget support. · Can be difficult to scale up. · Processing time for approval is the same as for FSPs. · The original 'fast-track' concept has been lost as bureaucracy has seeped in and the MSP process is now equally as torturous and demanding as the FSP. · Often times, MSPs are as labor-intensive as FSP, but less attractive to Government because of small amount of resources. Figure 9: Weaknesses of Medium Size Projects Weakness: Medium-sized projects 80% 70% 60% 50% Agree Disagree 40% Don't 30% know 20% 10% 0% MSPs are not as MSPs take just as long MSPs require just as MSPs endorsement is sustainable as FSPs to be approved as FSPs much supervision as difficult FSPs 26 3.3 Recommendations Of Other Possible Modalities 191 respondents provided open feedback on possible new modalities to be adopted by the GEF. Most responses suggested the need to adopt more programmatic approaches, followed by a need for Micro- project Approaches, which may suggest changes in the Small Grants Program. The aggregated responses can be seen in the Table 10. Table 10: Other modalities the GEF could make use of Category Number of Res. Programmatic and Sector Wide Approaches 20 Micro-project Approach (changes in the SGP) 9 Technical Assistance and Capacity Building 7 Non-grant financing modalities, particularly to engage private sector. PPPs 6 Budgetary support 3 Projects with larger co-financing from national partners 2 Closer work with Implementing Agencies (basket funding) 3 Analytical work 2 Less stringency over Global Environmental Benefits 2 Grant for Local Governments 1 Loan guarantees 1 Out-sourcing 1 Debt swaps 1 Direct Execution Project (DEX) 1 Emergency funds for natural disasters 1 A common recommendation among participants was that the projects should become more decentralized and there should be more delegation of responsibilities over the use of GEF Funds to Implementing and Executing Agencies. There were some complaints about the present financial range of MSPs. According to some respondents, this range should be increased. Furthermore, there were comments on the need for MSPs to become more decentralized. Support to South-South cooperation also emerged as a possible area of engagement for the GEF. 3.4 Modalities and Activity Cycle: Strengths and Weaknesses In the last section of this part of the survey, respondents were asked to provide their perception of the strengths and weaknesses of the GEF Activity Cycle and Modalities. They were given both direct questions and open questions. Table 11 and 12 aggregate the responses to the questions on strengths and weaknesses, each of which were answered by approximately 305 respondents. Box 10 and 11 present some of the main observations made by the respondents in the open questions. Table 11: Activity Cycle and Modality Strengths Categories Nš of Total Responses13 % Good capacity levels for implementation 112 18% Appropriate scope of project goals and objectives 108 18% National or local circumstances particularly appropriate for project goals and objectives 99 16% Good use of adaptive management 73 12% Good relationship between complexity of project design and scope of project 56 9% 27 Categories Nš of Total Responses13 % goals Appropriate amount of time used for project design and approval 40 7% Appropriate timeframe to successfully address environmental challenges 37 6% Sufficient financial sustainability of project results 36 6% Positive promotion and documentation of replication 25 4% Sufficient data availability for monitoring and evaluation 21 3% TOTAL 607 100% Table 12: Activity Cycle and Modality Weaknesses Categories Nš of Total Responses % Too much time required for project design and approval 120 19% Overly complex project design in relation to scope of project goals 98 16% Too large of scope of project goals and objectives 78 12% Lack of financial sustainability of project results 70 11% Insufficient capacity levels for implementation 68 11% Too long of timeframe needed to successfully address environmental challenges 62 10% Insufficient adaptive management 44 7% Lack of data availability for monitoring and evaluation 39 6% Unanticipated national or local circumstances 32 5% Insufficient promotion and documentation of replication 16 3% TOTAL 607 100% 28 Box 10: Factors that contribute to the failure of GEF Projects · Too many fingers in the pie means the project has to satisfy everyone, and so it succeeds at nothing. · Conflicting interest between environment and national development (example: building a road). · Unnecessary intervention of IAs to hinder project preparation by other IAs due to competition over scarce resources. · The emphasis on incremental benefits rather than acknowledging the need to also invest in local benefits. · (1) Insufficient attention to capacity building; (2) Insufficient resources for strong supervision, especially at start-up stage when it is most vital; (3) Forced marriages with other financiers or projects to make matches but may not really work well together; (4) Sometimes co-financers do not act like there is one project. · The 'Christmas tree' design characterizes many of the GEF projects. In large part this is caused by the many GEF-mandated control steps which are part of the GEF project cycle. If there is greater simplification in the project cycle, the 'adding-on' tendencies which plague the GEF project cycle would be cut. · The interventions are too small and diffuse to make a strategic impact. GEF needs to have a more focused and selective approach to guide its operations in each country, in other words, GEF needs country strategies that limit the areas in which it will be involved in each country. · Inadequate understanding by governmental authorities (other than the GEF focal point) of the uses and rules for GEF funding. · Too ambitious - too many promises made to try to get funding. · Lack of clarity of strategic direction in some focal areas such as land degradation and associated projects. · Lack of government ownership, almost every step of the way is controlled by outsiders. · GEF insistence on global environmental benefits limits the potential for mainstreaming GEF issues into national development. A balance approach to generate both national and global benefits needed. · Application of the incremental costs concept constitutes an obstacles to cover all costs for the project completion and implementation and some time it results in the impossibility to realize a good project idea with good global and national environmental benefits. Box 11: Factors that contribute to the success of GEF Projects · Capacity to raise additional financial supports. · Appropriate project design. GEF stays at the cutting edge of innovation, technology. Rigorous implementation. · The key factor to success is government ownership. This may mean greater involvement with the planning commission and ministries of finance since the decisions of these agencies, rather than the environmental agencies, often determine issues like provision of counterpart financing, government priorities, replicability, etc. · Better utilization of the technical capacities of the EAs. · Allowing sufficient time and involving the appropriate stakeholders make for successful projects, wherein local people become advocates for change in their communities, with their governments, and within their regions. Building in and on an involvement citizenry is an important if time-consuming step, for success. This can be done in partnership with governmental agencies, local universities (extension services) and/or NGOs. The key is application of appropriate levels of funding for the activities, not necessarily for the convenience of the IAs/ EAs, which need to devote an extra effort in tracking and monitoring small expenditures in localized areas. · Projects should be managed by the nationals in the region where the project would be implemented. · While sufficient promotion and documentation of replication is a must, the key factor for success is a good adaptation of such replication to local circumstances (no copy-paste!) 29 4. GEF Operational Principles This section focuses on respondent's perceptions about eight operational principles of GEF operations: Incremental Costs for Global Environmental Benefits, Country Ownership, Cost-Effectiveness, Flexibility, Full Disclosure, Public Participation, Catalytic Role and Monitoring and Evaluation. This section of the survey encompasses four direct and three open questions. Tables below present the overall responses to the direct questions, answered by 335 respondents, followed by a box with some quotations about each of the Operational Principles, provided through the open questions. 4.1 Operational Principle Requiring Least Resources Table 13: Operational Principles that requires least resources In your opinion which of the GEF Operational Principles requires the LEAST amount of time and resources to incorporate? Rank Principles Nš of % Respondents 1 Full disclosure 94 28% 2 Country ownership 91 27% 3 Flexibility 43 13% 4 Monitoring and evaluation 27 8% 5 Catalytic role 25 7% 6 Public participation 22 7% 7 Cost-effectiveness 17 5% 8 Incremental costs for global environmental benefits 16 5% TOTAL 335 100% Most respondents deem to spend least resources on the principle of Full Disclosure and Country Ownership. At the same time, Incremental Costs and Cost-Effectiveness ranked lowest in the least resource-requiring category. 4.2 Operational Principle requiring mostresources Table 14: Operational Principles that require most resources In your opinion which of the GEF Operational Principles requires the MOST amount of time and resources to incorporate? Rank Principles Nš of Respondents % 1 Incremental costs for global environmental benefits 108 32% 2 Monitoring and evaluation 68 20% 3 Public participation 62 19% 4 Country ownership 28 8% 5 Cost-effectiveness 26 8% 6 Catalytic role 20 6% 7 Flexibility 12 4% 8 Full disclosure 11 3% TOTAL 335 100% Around one third of respondents deem the principle of Incremental Cost ranked to be the most resource- requiring. Monitoring and Evaluation comes next, with 20%, followed by Public Participation, by 19%. In line with the last table, Full Disclosure ranked last in this question. 30 4.3. Operational Principles Contributing MOST To The Potential Success Of GEF Projects Table 15: Operational Principles contributing most to the potential success of GEF projects In your opinion which of the GEF Operational Principles contributes MOST to the potential success of GEF projects? Rank Principles Nš of % Respondents 1 Country ownership 106 32% 2 Public participation 83 25% 3 Catalytic role 41 12% 4 Monitoring and evaluation 33 10% 5 Flexibility 32 10% 6 Incremental costs for global environmental benefits 19 6% 7 Cost-effectiveness 15 4% 8 Full disclosure 6 2% TOTAL 335 100% According 32% of respondents, the principle that contributes most to the success of GEF's projects is Country Ownership, followed by Public Participation. It should be noted that Country Ownership was deemed to be the second least resource-requiring principle, which indicates its efficiency. 4.4 Operational Principles Contributing LEAST To The Potential Success Of GEF Projects Table 16: Operational Principles contributing least to the potential success of GEF projects In your opinion which of the GEF Operational Principles contributes LEAST to the potential success of GEF projects? Rank Principles Nš of Respondents % 1 Full disclosure 101 30% 2 Incremental costs for global environmental benefits 92 27% 3 Catalytic role 37 11% 4 Cost-effectiveness 36 11% 5 Flexibility 27 8% 6 Public participation 16 5% 7 Country ownership 13 4% 8 Monitoring and evaluation 13 4% TOTAL 335 100% 30% of respondents think the principle of Full Disclosure contribute the least to the success of GEF projects. Incremental Cost comes second, with 27% of total responses. It should be noted that Full Disclosure is deemed not to be relevant to the success of GEF projects, but it does not require much resource either. On the other hand, incremental cost ranks second as the least contributing to GEF projects' success, and at the same time as the most resource-intensive principle, meaning it is considered to be an inefficient operational principle. 31 Box 12: Comments on the Operational Principles 1- Incremental Costs for Global Environmental Benefits · The often artificial distinction between 'incremental' and 'local' benefits leads to suboptimal project design, sometimes of little relevance to local actors and therefore to little impact. It would be better to acknowledged that incremental and local benefits need to be combined to get people interested. 2- Country Ownership · Lack of government ownership -- almost every step of the way is controlled by outsiders. · GEF projects have strong government ownership. 3- Cost-Effectiveness · The project 'culture' interferes negatively with modern ideas of sectoral approach and support. It sustains a parallel money and capacity consuming aid industry. 4- Flexibility · Medium and small projects that can be implemented with minimal bureaucracy and through NGOs and other smaller, directly responsible agencies closer to the 'field'. Quick decision making and a transfer of responsibility for assessment and accountability eases the burdens on GEF core staff. · Not enough guidelines for project preparation, in reviews too much emphasis is given to 'write in right language', which takes away from creativity, heavy procedures take time away from spending it on content, and have to be spent to fulfill all requirements of the process. 5- Full Disclosure · The GEF faces a lack of transparency about the decision making process. 6- Public Participation · GEF projects have a high degree of prior stakeholder consultation. · GEF projects have an emphasis on stakeholder involvement and participatory approaches. 7- Catalytic Role · GEF encourages and facilitates partnerships with other donors and has the ability to mobilize domestic resources for achieving sustainable development in the country and global benefits. GEF fund acts as catalyst to attract other funds. 8- Monitoring and Evaluation. · GEF has an inadequate Monitoring and impact evaluation, including ex-post. · A factor that contributes to the failure of GEF projects is the overly complicated M&E, so that monitoring exercises become so large they are like another project activity. 5. GEF In Comparison To Other International Agencies This section of the survey aims at comparing different features of GEF projects and the results of GEF activities with comparable projects and activities of other international agencies. The features surveyed were timeframe, amount of resources required to prepare and approve the project, implementation period, budgets, time and resources required for Monitoring and Evaluation. The results surveyed were the preparedness of projects before being implemented, chance of having a significant impact and overall success. A graph and a comment for each question are presented below. 32 5.1 Timeframe of GEF Projects Figure 10: Timeframe of GEF projects GEF projects - Timeframe 5% Take longer to prepare and 21% approve Take about the same amount of time to prepare and approve Take less time to prepare and 74% approve 289 respondents provided feedback to this question. Around three quarters of respondents (213 responses) consider that GEF Projects take more time to prepare and approve than that of other international agencies. This finding is in line with Table 15 above where most respondents point the length of projects timeframe to be the greatest weakness of GEF activity cycle. Around one fifth of respondents think it takes as much times as other agencies, while only 5% of respondents judge it to require less time to prepare and approve. 5.2 Resources Necessary To Prepare And Approve GEF Projects Figure 11: Resources necessary to prepare and approve GEF projects GEF projects - Resources 6% Require more resources to prepare and approve 28% Require about the same amount of resources to prepare and approve Require less 66% resources to prepare and approve Around 66% of total respondents deem GEF projects to require more resources to prepare and approve than other international agencies. 28% think it takes the same amount of resources, while 6% judge it to be less resource-intensive. Again, this finding is line with previous questions where respondents showed concern with the amount of resources for project preparation and approval. 33 5.3 Implementation Period of GEF Projects Figure 12: Implementation Period of GEF Projects GEF projects - Implementation Period 8% Have similar length implementation periods Have longer 32% implementation periods 60% Have shorter implementation periods 60% of total respondents think that the Implementation Period of GEF project has similar length as other agencies. 32% think they have longer implementation periods, while 8% deem GEF projects implementation period to be shorter. 5.4 Budget of GEF Projects Figure 13: Budget of GEF Projects GEF projects - Budgets 24% Have similar budgets 43% Have larger budgets Have smaller budgets 33% Responses for this question showed less agreement than for the previous ones. There is only a 10-per cent difference between those who think GEF projects have similar budgets and those who think they have larger budgets. At the same time, almost one fourth of respondents think GEF projects have smaller budgets. 34 5.5 Time And Effort Required For Monitoring And Evaluation Figure 14: Effort required by GEF projects for Monitoring and Evaluation Effort required for Monitoring and Reporting 8% Require more time and effort for monitoring and reporting Require about the same amount of time and 52% effort for monitoring and 40% reporting Require less time and effort for monitoring and reporting Just over half of total respondents think that GEF projects require more time and resources for monitoring and reporting. 40% think monitoring and reporting activities in GEF projects require the same amount of time and resources as other international agencies' projects. Only 8% judge them to require less time and effort for these activities. These figures represent an acknowledgement of GEF attention to Monitoring and Reporting. The next three questions focus on the results of GEF activities in comparison to other agencies. 5.6 Preparedness Of GEF Projects Before Implementation Figure 15: Preparedness of GEF projects GEF projects are: 9% Are more well- prepared before beginning implementation Have about the same level of preparation 35% before beginning 56% implementation Are less well- prepared before beginning implementation 35 56% of people think GEF projects are better prepared before implementation than those of other international agencies. This is probably a direct consequence of the time and effort required for project preparation. 35% of respondents think they have the same level of preparation as projects by other agencies, while only 9% judge them to be less well-prepared. The percentage of respondents who judge GEF projects to be better prepared is highest among Implementation Agencies, 61%. It is, however, lower among Executing Agencies (50%), among the GEF members (44%) and lowest among NGOs, with 42%. 5.7 Chances Of Having A Significant Impact Figure 16: Chances of GEF projects having a significant impact GEF projects have: 11% Have about the same chance of having a significant impact Have a better chance 49% of having a significant impact 40% Have less chance of having a significant impact Almost half of respondents think GEF projects have the same chances of having a significant impact as those by other agencies. At the same time, 40% judge them to be more likely to have significant impacts, while 11% of respondents judge them to be less likely to have significant impact. Among Implementation and Executing Agencies, the percentage of respondents who think GEF Projects have a better chance of having a significant impact is higher than the average. However, this percentage is lower than the average among National Government and GEF, with 33%, and lowest among NGOs, with 28%. It should also be noted that the view that GEF projects have less chance of having a significant impact is shared by twice as many GEF members (22%) as the average (11%). 36 5.8 Level Of Overall Success Figure 17: Level of overall success of GEF projects Success Level 15% Have about the same level of success Are generally more successful 28% 57% Are generally less successful In line with the previous question, 57% of respondents think GEF projects have the same level of success as projects by other international agencies. 28% of respondents think they generally are more successful, while 15% think they are less successful. Among GEF members, this percentage of members holding the view that GEF projects are generally more useful rises significantly to 67%. However, among all other stakeholder groups, it is lower: Implementing Agencies, with 27%, Executing Agencies, with 25%, National Government, with 24% and lowest among NGOs, with 20%. There does not seem to be a direct relation between project preparedness and its likelihood of achieving significant impact, or its level of success. The ensuing question would then be what the perfect level of `preparedness' is, so that the GEF does not `overdesign' their projects. That is, how much time and effort should be spent on project preparation in order to maximize efficiency, without losing the quality that directly influences projects likelihood of success? This finding is directly linked to the overall Joint Evaluation questions: `Is the project cycle efficient?' and `Are modalities efficient'? 37 Annex A - Survey Methodology and Questions This Annex presents a brief explanation of the methodology used in the Survey and the data processing. It also presents the survey questions in their original format. Survey Methodology The survey was designed in collaboration between the GEF Evaluation Office and the Evaluation and Oversight Unit of the United Nations Environmental Programme (UNEP). The survey questions were based on the main issues regarding GEF Activity Cycle and Modalities that had already arisen in the analysis undertaken for Component 2 ­ Meta Analysis Evaluation and from Component 7 ­ Field Visits. The Joint Evaluation Management Group provided valuable comments on the design of survey, which were fully taken into account. Consultations were also held with experts on Survey from the World Bank and lessons learned from previous GEF surveys were considered. Preliminary results of the survey were presented in the May 2006 Joint Evaluation Meeting in Vienna. The survey was based on a web tool, Survey Monkey (www.surveymonkey.com), which allowed responses to be collected online and transferred to Excel. The responses were confidential and it was not required that participants identify themselves, but only provide some data on their background and experience with the GEF. The Survey was tested online by the JE Management Group before being available to the broad array of respondents. The survey addressed questions on perceptions of time, efficiency, effectiveness, GEF contributions, contributing factors and comparison to other relevant donors. It encompassed both direct and open questions, which were grouped in five main sections: i) activity cycle; ii) modalities; iii) operations principles; iv) comparisons to other international agencies and v) respondent's profile. Respondents were allowed to skip any question, and that is the reason why the number of total responses differs among questions. Some verifications questions were introduced, which could indicate the consistency of responses. The questions on the `most' and `least' helpful activity cycle phase for improving the focus on global benefits are an example of verification questions and the responses to those indicated a high degree of consistency. The Survey was broadly disseminated among GEF stakeholders electronically. The Joint Evaluation Management Group elaborated a strategy for distributing the survey, using the available GEF's and Implementing and Executing Agencies' e-mail lists, in order to broadly reach all targeted stakeholders. The survey was sent to approximately 2075 stakeholders and was responded by 660 of them. That is an expressive response rate of 32%. The table below shows an approximation of response rates according to stakeholder groups. These figures are based on the number of respondents reached directly through the e-mail lists used. It was also assumed that the GEF Focal Points forwarded the survey invitation further to other respondents in the national governments. It must be noted that over 50% of respondents (333 responses) preferred not to identify themselves. Table 17: Survey Response Rate by Stakeholder Category Stakeholder Group Response rate STAP Member 47% NGOs 25% Executing Agency Staff 25% GEF/ Conventions Focal Points 22% 38 Implementing Agency 22% GEF EO 20% National Government 19% STAP Roster 12% GEF SEC 9% Convention Secretariat 5% The responses were analyzed in an aggregated and disaggregated fashion. Sometimes, responses were broken down to highlight differences of opinions across stakeholder groups on views of their own and others' role. Survey Original questions The questions below are presented as they were in Survey Monkey. 1. Knowledge of the Global Environment Facility 1. Have you had experience or involvement with GEF-funded projects? (Yes or No question) If Not: 2. Have you had the opportunity to be involved in the development or implementation of GEF-funded projects? (Yes or No Question) If yes: 3. If you have had the opportunity to be involved with the development or implementation of GEF projects but have not done so, what are the factors that influenced this decision? Choices: Levels of co-financing required National or local circumstances Amount of time required for project design and approval Other (please specify) Adaptive management Financial sustainability of project results Capacity levels for implementation Complexity or simplicity of project design Scope of project goals Timeframe needed successfully address environmental challenges Data availability for monitoring and evaluation Promotion and documentation of replication 2. GEF Activity Cycle 4. Based on your experience, in the GEF activity cycle, which part of the process requires the GREATEST amount of time and resources? Choices: Project identification GEF Focal Point endorsement GEF pipeline entry 39 Project preparation Government endorsement Review by GEF Secretariat Review by a Scientific and Technical Advisory Panel (STAP) roster member Review by other Implementing Agencies/Executing Agencies Review by Convention Secretariats Submission to GEF Work Program Work Program approval CEO Endorsement / CEO Approval (for MSPs) Implementing Agency / Executing Agency internal approval Project start-up Implementation supervision 5. And, which part of the process requires the LEAST amount of time and resources? Project identification GEF Focal Point endorsement GEF pipeline entry Project preparation Government endorsement Review by GEF Secretariat Review by a Scientific and Technical Advisory Panel (STAP) roster member Review by other Implementing Agencies/Executing Agencies Review by Convention Secretariats Submission to GEF Work Program Work Program approval CEO Endorsement / CEO Approval (for MSPs) Implementing Agency / Executing Agency internal approval Project start-up Implementation supervision 6. Which two stages of the activity cycle are MOST helpful in focusing projects on the delivery of global environmental benefits? First Selection: Project identification GEF Focal Point endorsement GEF pipeline entry Project preparation Government Endorsement Review by GEF Secretariat Review by STAP Review by other IAs/EAs Review by Convention Secretariats Submission to GEF Work Program Work Program Approval CEO Endorsement / CEO Approval (for MSPs) IA/EA Approval Project start-up Implementation Supervision 40 7. Second Selection: Project identification GEF Focal Point endorsement GEF pipeline entry Project preparation Government Endorsement Review by GEF Secretariat Review by STAP Review by other IAs/EAs Review by Convention Secretariats Submission to GEF Work Program Work Program Approval CEO Endorsement / CEO Approval (for MSPs) IA/EA Approval Project start-up Implementation Supervision 8. Which two stages of the activity cycle are LEAST helpful in focusing projects on delivering global environmental benefits? First Selection: Project identification GEF Focal Point endorsement GEF pipeline entry Project preparation Government Endorsement Review by GEF Secretariat Review by STAP Review by other IAs/EAs Review by Convention Secretariats Submission to GEF Work Program Work Program Approval CEO Endorsement / CEO Approval (for MSPs) IA/EA Approval Project start-up Implementation Supervision 9. Second Selection: Project identification GEF Focal Point endorsement GEF pipeline entry Project preparation Government Endorsement Review by GEF Secretariat Review by STAP Review by other IAs/EAs Review by Convention Secretariats 41 Submission to GEF Work Program Work Program Approval CEO Endorsement / CEO Approval (for MSPs) IA/EA Approval Project start-up Implementation Supervision 3. Opinions on Strengths and Weaknesses of GEF Activity Cycle 10. Based on your experience, what is the greatest strength of the GEF activity cycle? (Open Question) 11. And, based on your experience, what is the greatest weakness of the GEF activity cycle? (Open Question) 4. Relevance of GEF Modalities 12. In your opinion, how relevant is each modality to the GEF's mandate and objective of funding global environmental benefits and acting as the financial mechanism for multi-lateral environmental agreements? Choices: Not Highly Moderately Not sure/Don't Response relevant relevant relevant know Average Full-size Projects Medium-size Projects Small Grants Program Enabling Activities National Capacity Self- Assessment Project Development Facility (PDFs) 13. And, in your opinion, how relevant is each modality to the needs and priorities of GEF partner countries? Not Highly Moderately Not sure/Don't Response relevant relevant relevant know Average Full-size Projects Medium-size Projects Small Grants Programme Enabling Activities National Capacity Self- Assessment Project Development Facility (PDFs) 5. Opinions on Strengths and Weaknesses of GEF Modalities 42 14. Based on your experience, what is the greatest strength of the current GEF modalities? (Open Question) 15. And, based on your experience, what is the greatest weakness of the current GEF modalities? (Open Question) 16. Are there other modalities the GEF could make use of? (Open Question) 6. Medium-size Projects Strengths and Weaknesses 17. STRENGTHS: Please indicate the degree to which you agree or disagree with the following statements about Medium-size projects. Strongly Strongly Don't Response agree Agree Disagree disagree know Average MSPs are more flexible than FSPs MSPs require less capacity to implement than FSPs MSPs can serve as an initial step in a longer process A broader range of stakeholders can more effectively participate in MSPs MSPs are effective in attracting co- financing MSPs are more cost-effective investments MSPs are effective in building capacity among project partners 18. Based on your experience, are there other important strengths that you would like to identify? (Open Question) 19. WEAKNESSES: Please indicate the degree to which you agree or disagree with the following statements about Medium-size projects. Strongly Strongly Don't Response agree Agree Disagree disagree know Average MSPs are not as sustainable as FSPs MSPs take just as long to be approved as FSPs MSPs require just as much supervision as FSPs It is difficult to obtain country endorsement for MSPs 20. Based on your experience, are there other important weaknesses that you would like to identify? (Open Question) 7. Weaknesses of GEF Activity Cycle and Modalities 43 21. First Choice: i. Too large of scope of project goals and objectives ii. Overly complex project design in relation to scope of project goals iii. Too long of timeframe needed to successfully address environmental challenges iv. Too much time required for project design and approval v. Lack of data availability for monitoring and evaluation vi. Insufficient promotion and documentation of replication vii. Insufficient adaptive management viii. Lack of financial sustainability of project results ix. Unanticipated national or local circumstances x. Insufficient capacity levels for implementation 22. Second Choice: i. Too large of scope of project goals and objectives ii. Overly complex project design in relation to scope of project goals iii. Too long of timeframe needed to successfully address environmental challenges iv. Too much time required for project design and approval v. Lack of data availability for monitoring and evaluation vi. Insufficient promotion and documentation of replication vii. Insufficient adaptive management viii. Lack of financial sustainability of project results ix. Unanticipated national or local circumstances x. Insufficient capacity levels for implementation 23. Based on your experience, are there other important factors you would like to identify that may contribute to the potential failure of GEF projects? (Open Question) 8. Strengths of the GEF Activity Cycle and Modalities 24. First choice: i. Appropriate scope of project goals and objectives ii. Good relationship between complexity of project design and scope of project goals iii. Appropriate timeframe to successfully address environmental challenges iv. Appropriate amount of time used for project design and approval v. Sufficient data availability for monitoring and evaluation vi. Positive promotion and documentation of replication vii. Good use of adaptive management viii. Sufficient financial sustainability of project results ix. National or local circumstances particularly appropriate for project goals and objectives x. Good capacity levels for implementation 25. Second choice: i. Appropriate scope of project goals and objectives 44 ii. Good relationship between complexity of project design and scope of project goals iii. Appropriate timeframe to successfully address environmental challenges iv. Appropriate amount of time used for project design and approval v. Sufficient data availability for monitoring and evaluation vi. Positive promotion and documentation of replication vii. Good use of adaptive management viii. Sufficient financial sustainability of project results ix. National or local circumstances particularly appropriate for project goals and objectives x. Good capacity levels for implementation 26. Based on your experience, are there other important factors you would like to identify that may contribute to the potential success of GEF projects? (Open question) 9 GEF in Comparison to other International Agencies 27. Timeframe - GEF projects Choices: Take longer to prepare and approve Take less time to prepare and approve Take about the same amount of time to prepare and approve 28. Resources - GEF projects: Choices: Require more resources to prepare and approve Require less resources to prepare and approve Require about the same amount of resources to prepare and approve 29. Implementation period - GEF projects Choices: Have longer implementation periods Have shorter implementation periods Have similar length implementation periods 30. Budgets - GEF projects Choices: Have larger budgets Have smaller budgets Have similar budgets 31. Time and effort required for monitoring and reporting - GEF projects: Choices: 45 Require more time and effort for monitoring and reporting Require less time and effort for monitoring and reporting Require about the same amount of time and effort for monitoring and reporting 10 GEF Results in Comparison to other International Agencies 32. Based on your experience, in relation to projects funded by other donors, would you say that GEF projects: Choices: Are more well-prepared before beginning implementation Are less well-prepared before beginning implementation Have about the same level of preparation before beginning implementation 33. And, would you say that GEF projects: Choices: Have a better chance of having a significant impact Have less chance of having a significant impact Have about the same chance of having a significant impact 34. And, would you say that GEF projects: Choices: Are generally more successful Are generally less successful Have about the same level of success 11 Operational Principles of the GEF 35. In your opinion, which of the GEF Operational Principles requires the LEAST amount of time and resources to incorporate into the preparation and implementation of GEF projects? Choices: Incremental costs for global environmental benefits Country ownership Cost-effectiveness Flexibility Full disclosure Public participation Catalytic role Monitoring and evaluation 36. In your opinion, which of the GEF Operational Principles requires the MOST amount of time and resources to incorporate? 46 Choices: Incremental costs for global environmental benefits Country ownership Cost-effectiveness Flexibility Full disclosure Public participation Catalytic role Monitoring and evaluation 37. In your opinion, which of the GEF Operational Principles contributes MOST to the potential success of GEF projects? Choices: Incremental costs for global environmental benefits Country ownership Cost-effectiveness Flexibility Full disclosure Public participation Catalytic role Monitoring and evaluation 38. In your opinion, which of the GEF Operational Principles contributes LEAST to the potential success of GEF projects? Choices: Incremental costs for global environmental benefits Country ownership Cost-effectiveness Flexibility Full disclosure Public participation Catalytic role Monitoring and evaluation 12. Open Input 39. Do you have any suggestions about how the GEF activity cycle could be improved? (Open Question) 40. Do you have any suggestions about how the GEF could improve the range of its modalities? (Open Question) 41. Is there anything else you would like to comment on? (Open Question) 13. Experience with the GEF Activity Cycle and Modalities 47 42. Please identify the periods during which you have had experience with the GEF: Choices: GEF Pilot Phase (1991-1994) GEF Phase 1 (1995-1998) GEF Phase 2 (1999-2002) GEF Phase 3 (2003-2006) 43. Type of GEF modalities with which you have had experience: Choices: Full-size projects Medium-size projects Small Grants Programme Enabling Activities National Capacity Self-Assessment Project Development Facility (PDFs) Other (please specify) 44. With which Implementing or Executing Agencies have you had experience working on GEF-funded projects? Choices: United Nations Development Programme United Nations Environment Programme World Bank African Development Bank Asian Development Bank European Bank for Reconstruction and Development Inter-American Development Bank International Fund for Agricultural Development United Nations Food and Agriculture Organization United Nations Industrial Development Organization 14. Background Information 45. From the following categories, how would you identify yourself? You may select more than one category Choices: GEF Political Focal Point (current or former) GEF Operational Focal Point (current or former) GEF Council Member (current or former) Implementing Agency Staff Executing Agency Staff National Government State/Local Government Convention National Focal Point 48 GEF Secretariat GEF Evaluation Office STAP Member STAP Roster Expert Convention Secretariat International NGO National/Local NGO Private Sector Other (please specify) 46. In which region or regions are you involved with GEF activities? Choices: Africa (including northern Africa) Asia (including Western Asia and Pacific Islands) Eastern Europe/Central Asia Latin America & Caribbean Global 1This is the reason why the total number of responses to this question does not add up to 660. Moreover, 327 people skipped this question. 2The Survey followed the GEF operational classification of regions: Africa, Asia, Europe and Central Asia (ECA) and Latin America and the Caribbean (LAC). 3Each Activity Cycle Phase is made up of different steps, which are specific activities that happen in a given phase. The term `stage' is sometimes used to refer to an Activity Cycle `step'. 4It should be noted that the total number of answers may vary from question to question, since respondents could skip any question. This is the reason why 20 responses mean 71% of STAP member responses. 28 STAP members responded this question, even though .35 STAP members participated in the survey. 5The total number of respondents to this question was 465. The % of total adds up to more than 100% because respondents were allowed to choose more than one answer. 6The total number of respondents to this question was 441. The % of total adds up to more than 100% because respondents were allowed to choose more than one answer. 7An average number is used here because some respondents selected their first option, but not the second one. 8It should be noted that the total number of responses encompasses both the first and second options. 9It should be noted that the total number of responses encompasses both the first and second options. 10An average number is used here because some respondents selected their first option, but not the second one. 11The number of responses represent how many times a certain activity cycle or operational principle was cited as a GEF strength by the survey respondents. 12The number of responses represent how many times a certain activity cycle or operational principle was cited as a GEF strength. 13The total number of responses aggregates the first and second choices that respondents could choose from. 49