REPUBLIC OF CROATIA Agriculture Pollution Control Project Report No. 115691 JUNE 6, 2017 © 2017 International Bank for Reconstruction This work is a product of the staff of The World RIGHTS AND PERMISSIONS and Development / The World Bank Bank with external contributions. The findings, The material in this work is subject to copyright. 1818 H Street NW interpretations, and conclusions expressed in Because The World Bank encourages Washington DC 20433 this work do not necessarily reflect the views of dissemination of its knowledge, this work may be Telephone: 202-473-1000 The World Bank, its Board of Executive reproduced, in whole or in part, for Internet: www.worldbank.org Directors, or the governments they represent. noncommercial purposes as long as full attribution to this work is given. Attribution—Please cite the work as follows: The World Bank does not guarantee the World Bank. 2017. Project Performance accuracy of the data included in this work. The Any queries on rights and licenses, including Assessment Report: Republic of Croatia: boundaries, colors, denominations, and other subsidiary rights, should be addressed to Agriculture Pollution Control Project. information shown on any map in this work do World Bank Publications, The World Bank Washington, DC: World Bank. not imply any judgment on the part of The Group, 1818 H Street NW, Washington, DC World Bank concerning the legal status of any 20433, USA; fax: 202-522-2625; e-mail: territory or the endorsement or acceptance of pubrights@worldbank.org. such boundaries. tabZ Report No.: 115691 PROJECT PERFORMANCE ASSESSMENT REPORT REPUBLIC OF CROATIA AGRICULTURE POLLUTION CONTROL PROJECT (TF090845-HR) June 6, 2017 Financial, Private Sector, and Sustainable Development Independent Evaluation Group ii Currency Equivalents (annual averages) Currency unit = Croatian kuna (HRK) 2007 $1.00 HRK 5.36 2008 $1.00 HRK 4.94 2009 $1.00 HRK 5.28 2010 $1.00 HRK 5.50 2011 $1.00 HRK 5.34 2012 $1.00 HRK 5.85 Source: IMF International Financial Statistics Database. All dollar amounts are U.S. dollars unless otherwise indicated. Abbreviations and Acronyms AACP Agricultural Acquis Cohesion Project APCP Agricultural Pollution Control Project CAEI Croatian Agricultural Extension Service Institute CAS Country Assistance Strategy CW Croatian Waters EU European Union FY fiscal year GDP gross domestic product GEF Global Environment Facility GEO global environment objective HRK Croatian kuna IBRD International Bank for Reconstruction and Development ICR Implementation Completion and Results Report IEG Independent Evaluation Group LU livestock unit MAFWM Ministry of Agriculture, Forestry, and Water Management M&E monitoring and evaluation NVZ Nitrate Vulnerable Zone PAD project appraisal document PDO project development objective PIU project implementation unit PPAR Project Performance Assessment Report TTL task team leader Fiscal Year Government: January 1 - December 31 Director-General, Independent Evaluation: Ms. Caroline Heider Director, IEG Financial, Private Sector Sustainable Development: Mr. José Cándido Carbajo Martínez Manager, IEG Sustainable Development: Ms. Midori Makino Task Manager: Ms. Alexandra C. Horst iii Contents Principal Ratings ............................................................................................................................. v Key Staff Responsible..................................................................................................................... v Preface........................................................................................................................................... vii Summary ...................................................................................................................................... viii 1. Background and Context............................................................................................................. 1 Agricultural Pollution Control Project ........................................................................................ 2 2. Project Objective and Relevance ................................................................................................ 2 Project Design and Relevance .................................................................................................... 4 3. Implementation ........................................................................................................................... 7 Financial Management and Procurement.................................................................................... 8 Safeguards Compliance .............................................................................................................. 9 Monitoring and Evaluation ......................................................................................................... 9 4. Achievement of the Objectives (Efficacy) ................................................................................ 10 Objective 1: Increased Use of Environmentally Friendly Practices ......................................... 10 Objective 2: Reduced Nutrient Discharge from Agricultural Sources ..................................... 13 5. Efficiency .................................................................................................................................. 16 6. Ratings ...................................................................................................................................... 17 Outcome .................................................................................................................................... 17 Risk to Development Outcome ................................................................................................. 18 World Bank Performance ......................................................................................................... 18 Quality at Entry ..................................................................................................................... 18 Quality of Supervision .......................................................................................................... 19 Borrower Performance .............................................................................................................. 20 Government Performance ..................................................................................................... 20 Implementing Agency Performance ..................................................................................... 21 Monitoring and Evaluation ....................................................................................................... 22 7. Lessons ...................................................................................................................................... 23 References ..................................................................................................................................... 24 This report was prepared by Alexandra C. Horst, who assessed the project in November 2016. The report was peer reviewed by Pierre Gerber and panel reviewed by Christopher Nelson. Vibhuti Narang Khanna provided administrative support. iv Table Table 4.1. Project Manure Storage Investments and IEG Sample................................................ 11 Appendixes Appendix A. Basic Data Sheet...................................................................................................... 27 Appendix B. Fieldwork Methodology .......................................................................................... 30 Appendix C. Interviews with Direct Beneficiaries ....................................................................... 34 Appendix D. Stakeholder Interviews ............................................................................................ 36 Appendix E. List of Persons Consulted ........................................................................................ 37 Appendix G. Example of Green Manure ...................................................................................... 48 Appendix H. APCP Brochures ..................................................................................................... 49 Appendix I. Summary of Studies Commissioned by APCP ......................................................... 50 Appendix J. Croatia River Basin Management Plan 2016–21 ..................................................... 56 v Principal Ratings ICRa ICR Reviewa PPAR Outcome Moderately satisfactory Moderately unsatisfactory Moderately satisfactory Risk to Moderate Moderate Moderate Development Outcome Bank Moderately satisfactory Moderately unsatisfactory Moderately satisfactory Performance Borrower Moderately satisfactory Moderately satisfactory Moderately satisfactory Performance a. The Implementation Completion and Results Report (ICR) is a self-evaluation by the responsible World Bank global practice. The ICR Review is an intermediate IEG product that seeks to independently validate the findings of the ICR. Key Staff Responsible Project Task Manager or Leader Sector or Senior Country Director Director Appraisal Aleksandar Nacev Juergen Voegele Anand K. Seth Completion Vera Dugandzic Juergen Voegele Kyle Peters vi IEG Mission: Improving World Bank Group development results through excellence in independent evaluation. About this Report The Independent Evaluation Group assesses the programs and activities of the World Bank for two purposes: first, to ensure the integrity of the World Bank’s self-evaluation process and to verify that the World Bank’s work is producing the expected results, and second, to help develop improved directions, policies, and procedures through the dissemination of lessons drawn from experience. As part of this work, IEG annually assesses 20–25 percent of the World Bank’s lending operations through fieldwork. In selecting operations for assessment, preference is given to those that are innovative, large, or complex; those that are relevant to upcoming studies or country evaluations; those for which Executive Directors or World Bank management have requested assessments; and those that are likely to generate important lessons. To prepare a Project Performance Assessment Report (PPAR), IEG staff examine project files and other documents, visit the borrowing country to discuss the operation with the government and other in-country stakeholders, interview World Bank staff and other donor agency staff both at headquarters and in local offices as appropriate, and apply other evaluative methods as needed. Each PPAR is subject to technical peer review, internal IEG Panel review, and management approval. Once cleared internally, the PPAR is commented on by the responsible World Bank country management unit. The PPAR is also sent to the borrower for review. IEG incorporates both World Bank and borrower comments as appropriate, and the borrowers’ comments are attached to the document that is sent to the World Bank’s Board of Executive Directors. After an assessment report has been sent to the Board, it is disclosed to the public. About the IEG Rating System for Public Sector Evaluations IEG’s use of multiple evaluation methods offers both rigor and a necessary level of flexibility to adapt to lending instrument, project design, or sectoral approach. IEG evaluators all apply the same basic method to arrive at their project ratings. Following is the definition and rating scale used for each evaluation criterion (additional information is available on the IEG website: http://ieg.worldbankgroup.org). Outcome: The extent to which the operation’s major relevant objectives were achieved or are expected to be achieved efficiently. The rating has three dimensions: relevance, efficacy, and efficiency. Relevance includes relevance of objectives and relevance of design. Relevance of objectives is the extent to which the project’s objectives are consistent with the country’s current development priorities and with current World Bank country and sectoral assistance strategies and corporate goals (expressed in Poverty Reduction Strategy Papers, country assistance strategies, sector strategy papers, and operational policies). Relevance of design is the extent to which the project’s design is consistent with the stated objectives. Efficacy is the extent to which the project’s objectives were achieved or are expected to be achieved, taking into account their relative importance. Efficiency is the extent to which the project achieved or is expected to achieve a return higher than the opportunity cost of capital and benefits at least cost compared with alternatives. The efficiency dimension is not applied to development policy operations, which provide general budget support. Possible ratings for outcome: highly satisfactory, satisfactory, moderately satisfactory, moderately unsatisfactory, unsatisfactory, and highly unsatisfactory. Risk to Development Outcome: The risk, at the time of evaluation, that development outcomes (or expected outcomes) will not be maintained (or realized). Possible ratings for risk to development outcome: high, significant, moderate, negligible to low, and not evaluable. World Bank Performance: The extent to which services provided by the World Bank ensured quality at entry of the operation and supported effective implementation through appropriate supervision, including ensuring adequate transition arrangements for regular operation of supported activities after loan/credit closing, toward the achievement of development outcomes. The rating has two dimensions: quality at entry and quality of supervision. Possible ratings for World Bank performance: highly satisfactory, satisfactory, moderately satisfactory, moderately unsatisfactory, unsatisfactory, and highly unsatisfactory. Borrower Performance: The extent to which the borrower (including the government and implementing agency or agencies) ensured quality of preparation and implementation and complied with covenants and agreements toward the achievement of development outcomes. The rating has two dimensions: government performance and implementing agency(ies) performance. Possible ratings for borrower performance: highly satisfactory, satisfactory, moderately satisfactory, moderately unsatisfactory, unsatisfactory, and highly unsatisfactory. vii Preface This is a Project Performance Assessment Report by the Independent Evaluation Group (IEG) of the World Bank Group on the Agricultural Pollution Control Project (APCP) in Croatia. In addition to reviewing the project documents and conducting fieldwork in Croatia, the mission sought to investigate the links between this project and IEG’s Evaluation of the World Bank Group Support for Pollution Management. IEG met with different stakeholders linked to the APCP, including the project implementation unit staff, government counterparts, nongovernment technical experts, project beneficiaries, World Bank staff, and Global Environment Facility staff. Following standard IEG procedures, a copy of the draft report was sent to the relevant government officials and agencies for their review and feedback. The Borrower did not have any comments on the report. viii Summary The Danube River Basin is one of Europe’s largest cross-boundary river catchment areas, draining large parts of central and southeastern Europe before flowing into the Black Sea. Pollution of the Danube River and the Black Sea is a serious problem given the high volume of nutrients in these water bodies. More than half of the total nutrient loads into the Danube River are estimated to originate from agriculture, a result of intensive fertilizer application and poor manure management, particularly in areas next to surface watercourses. The resulting water pollution and biodiversity loss leads to substantial negative consequences for agricultural productivity, soil fertility, and conservation of the biological ecosystem. In Croatia, the Danube River and its tributaries drain about 60 percent of the country’s territory, covering large parts of its land for agricultural production. About 40 percent of registered agricultural holdings in Croatia raise livestock. In the early 2000s, poor manure management practices were prevalent in the agricultural sector, and it lacked awareness of sustainable agricultural practices to reduce nutrient loads to water bodies. Before joining the European Union (EU) on July 1, 2013, Croatia committed to comply with EU principles, priorities, and requirements for accession, including the EU Nitrates and Water Directives. To address pollution of the Danube River Basin and support Croatia’s preparation for EU accession, the Global Environment Facility provided a $5 million grant fund to the government of Croatia for the Agricultural Pollution Control Project (APCP), implemented with the World Bank. The APCP’s objective was “to significantly increase the use of environmentally friendly agricultural practices by farmers in the Recipient’s Danube River basin in order to reduce nutrient discharge from agricultural sources to surface and groundwater bodies.” The World Bank Board approved the project in December 2007, and the project completed as planned in July 2012. The project design had four interlinked components: 1. Mitigation of nutrient loads to water bodies from point-source pollution to establish an investment fund to finance grants for 75 percent of the cost of manure storage and management infrastructure for medium-scale livestock farmers in the three agriculture- intense counties of Osiječko-Baranjska, Vukovarsko-Srijemska, and Varaždinska. This component was designed to also support Croatia’s water monitoring and impact analysis efforts in collaboration with Croatian Waters, the state entity for water management. 2. Development and promotion of agri-environment measures to develop and disseminate a Code of Good Agricultural Practices and implement a field demonstration program for farmers on sustainable, cost-effective manure management. 3. Public awareness and replication strategy to disseminate knowledge on a national scale about the activities that Components 1 and 2 supported, and thus stimulate widespread replication. 4. Project management to ensure effective and efficient use of resources and monitoring of project activities. The Independent Evaluation Group (IEG) assessed the project’s performance with the following project ratings: ix  Relevance of the APCP’s objectives is rated high given its previous and continued importance in the context of Croatia’s environmental priorities in the agricultural sector and its consistency with former and current World Bank country strategies. The relevance of design is rated substantial based on the assessment that the APCP design was appropriate given its underlying theory of change to address Croatia’s development needs in the agricultural sector, with only minor shortcomings related to the results framework and the Nitrates Mitigation Fund design.  Efficacy of the APCP is considered in two parts: The project substantially achieved an increase in the use of environmentally friendly agricultural practices in the three priority counties (Objective 1). However, because of insufficient outcome evidence, the project only modestly achieved a reduction in nutrient discharge in water bodies from agricultural sources (Objective 2).  Efficiency of the APCP is rated modest. Most project activities were implemented in the range of acceptable costs while contributing to the assumed project benefits. However, at project closure, IEG did not receive any new economic or financial information on the actual take-up of improved infrastructure or practices. Furthermore, the project assessment revealed operational inefficiencies related to significant project implementation delays and that all APCP-financed water analysis stations were idle. Overall, the ratings for relevance, efficacy, and efficiency result in an outcome rating of moderately satisfactory. Given that Croatia’s EU membership since 2013 requires compliance with the Nitrates and Water Directives and the continued funding for APCP-like environmental measures in the agriculture sector, IEG rated the risk to development outcome as modest. The World Bank’s performance in preparing the APCP was adequate, with reasonable technical design of the project’s components and integration of lessons from previous operations in the region. However, there were significant shortcomings in quality given the overambitious design of the APCP results framework, with unrealistic project development objective indicators (on a quantifiable reduction in nutrient loads and for national awareness raising), and weaknesses in the financing mechanism design for the Nitrates Mitigation Investment Fund. Therefore, IEG rated quality at entry as moderately satisfactory. Regarding World Bank supervision, quality varied significantly throughout project implementation, with frequent changes of task team leaders, poor handovers that disrupted the continuity and consistency of supervision, and a recurrent lack of attention and responsiveness, which significantly hindered the flow of project implementation and client satisfaction. Consequently, IEG rated quality of World Bank supervision as moderately unsatisfactory. IEG gave an overall moderately satisfactory rating for World Bank performance. The government of Croatia was mostly consistent in its support of the APCP throughout project implementation, but the project received minor attention considering its small size and grant- financing nature. Government commitment varied substantially between different agencies depending on their link to the agricultural sector, general concern about agricultural pollution, and direct involvement with farmers. Considering significant shortcomings in the performance of key central government agencies in the APCP implementation, IEG rated government performance as moderately unsatisfactory. The implementing agency, which was the project implementation unit (PIU) at the Ministry of Agriculture, Forestry, and Water Management, x consisted of qualified personnel who were highly engaged and motivated throughout implementation. The PIU conducted its activities with great dedication and was responsive to challenging implementation issues by reaching out to different stakeholders, including farmers, local municipalities, nongovernmental organizations, and academia. Therefore, IEG rated the implementing agency performance as satisfactory, leading to an overall borrower performance rating of moderately satisfactory. The design of the APCP’s monitoring and evaluation (M&E) system was generally reasonable and appropriate for the project’s size. As part of its implementation, a permanent M&E specialist was hired who designed and oversaw the implementation of three farm-level surveys. Monitoring efforts went beyond the original design, but with mixed results in generating adequate results evidence, particularly on project outcomes. Furthermore, there is little evidence regarding the use of monitoring data by project stakeholders. Overall, IEG rated M&E quality as modest. Finally, IEG’s performance assessment of the APCP experience suggests the following lessons, grouped under three interrelated categories. Lessons ACHIEVING LONG-TERM BEHAVIOR CHANGE  Local stakeholder involvement: Innovative projects require strong involvement of local stakeholders to generate advocates of behavior change. The involvement of farmers, government institutions, academia, and nongovernmental organizations is highly important to secure interest at the local level and to stimulate the sustainability of new practices.  More than infrastructure: Combining support for infrastructure and activities to promote behavior change can be effective. Addressing constraints of a physical and social nature at the same time through complementary activities is essential to incentivize and enable beneficiaries to adopt new practices.  Realistic objectives: Objectives of pilot interventions need to be realistic, especially when aiming at longer term goals that require behavior change. The environmental objective of the APCP was unattainable in the given project time frame and scale. Mid- level outcomes consistent with a longer time frame are more useful in illustrating progress against the objectives. ADEQUATE IMPLEMENTATION  Enabling environment: Early analysis and addressing constraining factors in a project’s enabling environment can reduce implementation delays. The APCP struggled with significant delays because of land titling issues. Tackling this issue early in the project allowed the team to reduce the effects on implementation.  Effective pilots: Pilot interventions are most effective when they can take a practical approach to implementation considering future scale-up. The ability of the APCP implementing agencies to test processes and refine them was crucial for the smooth transition to managing EU funding activities. xi  World Bank involvement: The World Bank team leadership’s consistent presence and responsiveness is crucial to ensuring the success of innovative project approaches. The APCP project planning and execution was smoother when the TTL was easily accessible, especially during innovation launches or when intense support was required. CLIENT COMMITMENT AND INCENTIVES  Lack of client incentives: Small, stand-alone grant investments without government contribution risk not obtaining a sufficient level of attention. Both the World Bank and the client had little concern for the APCP implementation because of its small scale and because it was not combined with a loan.  Effective communication: A successful project requires effective communication beyond the technical level to secure political buy-in and future funding. The APCP did not emphasize reaching out to key decision makers, and the project might not have secured current funding for APCP-like activities without the EU requirements related to agricultural pollution. José Cándido Carbajo Martínez Director Financial, Private Sector, and Sustainable Development Department Independent Evaluation Group 1 1. Background and Context 1.1 This is a Project Performance Assessment Report (PPAR) for the Agricultural Pollution Control Project (APCP) in Croatia, whose objective was to promote environmentally friendly agricultural production practices to reduce nutrient discharge in Danube Basin water bodies. The World Bank Board approved the APCP in December 2007 as a $5.0 million grant from the Global Environment Facility (GEF) Trust Fund. It was designed as a five-year project considering the country’s aspiration for accession to the European Union (EU). Croatia’s EU candidacy was confirmed in April 2004 conditional on principles, priorities, and requirements for accession, including the EU Nitrates and Water Directives. Croatia joined the EU as a member state on July 1, 2013, one year after closure of the APCP. 1.2 The APCP was prepared to be complementary to the Croatia Agricultural Acquis Cohesion Project (AACP) P091715, approved in February 2006 as a $30 million loan from the International Bank for Reconstruction and Development (IBRD). The AACP objective as stated in the grant agreement was “to develop sustainable systems and capacities within the Ministry of Agriculture, Forestry, and Water Management and other public institutions to ensure timely compliance with EU acquis conditions in the rural sector” (World Bank 2008). Although the AACP supported institutional capacity development for meeting EU accession requirements in agriculture, the APCP financed pilot investment and capacity building primarily at the farm level in priority areas in the Danube Basin and its tributaries in Croatia.1 1.3 At the time of APCP appraisal, more than half of the total nutrient loads into the Danube River originated from agriculture. In Croatia, the Danube River and its tributaries, the Sava and Drava rivers, drain 60 percent of the country’s territory and flow through the Pannonian plain, which covers nearly half of the country’s agricultural production land. This highlights the influence and significance of Croatia’s agricultural sector on the waters of the Danube River and consequently the Black Sea, and underscores the importance of the APCP objective to improve these practices and mitigate pollution from agricultural sources. 1.4 Agriculture has been an important contributor to Croatia’s economic growth and employment, though its value added has been declining during the last decade. In 2005, agriculture contributed 5 percent to Croatia’s gross domestic product (GDP) and represented 14.3 percent of total employment; however, in 2014–15 these figures fell to 4.3 percent of GDP and 8.7 percent of total employment per the World Development Indicators (World Bank 2017). Despite these downturns, agriculture continues to have an important role in the Croatian economy and, next to tourism, is a strategic sector for rural development.2 1.5 According to a study commissioned by the APCP on the pressure on Croatian water resources caused by nitrates and phosphorous of agricultural origin, agriculture is responsible for 97 percent of the anthropogenic nitrates and phosphorous pressure on water resources (appendix I summarizes the study). Mineral fertilizers are a source of 69 percent of nitrogen and 52 percent of phosphate applied to Croatian agricultural soils, which makes Croatia one of the most intensive EU consumers of nitrates and phosphorous from mineral fertilizers on a per-hectare basis. The study also reports an input of 52 kilotons nitrates from livestock manure. In the Danube River Basin, agriculture contributes to approximately 50 percent of the total nitrates and 2 phosphorous load (Liska 2015), suggesting that the pressure from agriculture is more than 50 percent because only a portion of nitrates eventually ends up in water. Agricultural Pollution Control Project 1.6 Project cost and financing. The GEF approved the grant contribution of $5.0 million to the APCP on December 6, 2007. The government of Croatia did not commit to a direct monetary contribution to the APCP, but the complementary AACP (which was IBRD-financed) was estimated to contribute $13.9 million. Local communities were expected to contribute $1.1 million in financing (partially in-kind). Overall, the APCP’s total, original project cost estimate at appraisal was $20.0 million. At completion, the total actual project cost was $19.8 million, including $4.99 million from GEF (appendix A). 1.7 Dates. The World Bank Board approved the APCP in December 2007, and the project became effective in July 2008 and closed in July 2012. There were no restructurings during project implementation. 2. Project Objective and Relevance 2.1 Project development objective (PDO) and indicators. The APCP’s PDO, as stated in the grant agreement, was “to significantly increase the use of environmentally friendly agricultural practices by farmers in the Recipient’s Danube River basin in order to reduce nutrient discharge from agricultural sources to surface and groundwater bodies.” The two PDO indicators were to achieve the following: “at least 40 percent of the farming population in the project areas adopting preventive and remedial measures to reduce nutrient discharges” (from an assumed baseline of 0 percent), and “increased national awareness of links between local actions and impact on water nutrient load” (with an end-of-project target of 25 percent from an assumed non-numeric low baseline to be determined by a social assessment early in the project implementation). 2.2 The project aimed at nationwide targeting of its awareness-raising activities, but it concentrated its pilot investment activities to promote the adoption of preventive farming measures on three counties in Northern and Eastern Croatia: Osiječko-Baranjska, Vukovarsko- Srijemska, and Varaždinska. The project appraisal document (PAD) explains that the selection of these counties was due to their relatively high livestock density and concentration of medium- scale livestock farms (mostly cattle and milk and pig production in Osiječko-Baranjska and Vukovarsko-Srijemska, and poultry farms in Varaždinska). It also cited prevailing poor manure management practices, lack of awareness of sustainable agricultural practices to reduce nutrient loads to water bodies, and inadequate water monitoring infrastructure. About 2,000 private farms were a likely fit for project eligibility requirements. 2.3 Global environment objective (GEO) and indicator. The GEO of the APCP was much broader and at a higher results level than the PDO, specifically “to reduce the discharge of nutrients into waters draining into the Danube River and Black Sea.” The PAD explains, “The project will provide an opportunity for the GEF to be a catalyst for actions to bring about the successful introduction and widespread adoption of integrated improved land and water resource management practices. GEF support was to help reduce costs and barriers to farmers adopting 3 improved and sustainable agricultural practices.” The PAD lists a key project outcome indicator linked to the GEO objective as part of the overall project results framework, which aimed at “a 10 percent reduction in discharge of nutrients into surface and groundwater in the three project regions” (that is, the three counties in which the APCP’s farm-level investments were concentrated). 2.4 Relevance of the PDO. The relevance of the APCP development objective was high and remains high. At appraisal, the PDO was highly relevant in the context of country priorities and was consistent with the World Bank’s FY05–08 Country Assistance Strategy (CAS) for Croatia. By assisting with the implementation of the EU Nitrates and Water Directives and pilot demonstrations of manure management systems and agro-environmental farming practices, the APCP activities were directly linked to the main CAS objective of supporting Croatia’s “growth and reform strategy for a successful EU accession, while also ensuring broad participation in…sustainable natural resource management.” The APCP activities remained highly relevant for the FY09–12 CAS because it addressed Pillar 2 (“strengthening private sector–led growth and accelerating European Union convergence”) and Pillar 4 (“increasing the sustainability of long-term development, and strengthening environment management and nature protection”). 2.5 Furthermore, considering the APCP’s broader global environmental relevance, the project objective of reducing nonpoint source nutrient pollution aligned strongly with the GEF-3 International Waters focal area’s operational program at project preparation. The Waterbody- Based Operational Program focused on seriously threatened water bodies. One of those was the Black Sea, which suffered a massive loss of aquatic life in the 1990s caused by increasing nutrient levels and organic pollution from agriculture fertilizer, livestock waste, and human sewage discharged into the Danube River Basin. In response, GEF -in partnership with the World Bank Group, the United Nations Development Programme, and other development agencies- supported a series of investments in 16 Danube Basin countries to “sustain regional collaboration and undertake a series of cross-sectoral nutrient pilot demonstrations,” one of which was the Croatia APCP. 2.6 More recently, the current Croatia Country Partnership Framework for FY14–17 does not specifically mention the agricultural sector because its main objectives focus on fiscal reforms, innovation and trade competitiveness, and maximizing economic benefits as an EU member. However, this shift is not surprising because activities related to agro-environmental measures and compliance with the EU Nitrates and Water Directives have been implemented through the 2014–20 Rural Development Program (supported by EU funding) since Croatia’s EU accession in 2013. The APCP was a pilot project by design that would use EU financing to expand after completion if the government considered it successful. To this end, the 2014–20 Rural Development Program led by the Ministry of Agriculture, Forestry, and Water Management (MAFWM) includes submeasure 4.1 on “Restructuring, modernization, and increasing the competitiveness of agricultural holdings” based on the identified weakness of “insufficient capacity for adequate disposal of manure on agricultural holdings” (MAWFM 2015). 2.7 Of the Rural Development Program’s €2.4 billion total estimated budget, about €20 million are earmarked for submeasure 4.1, which largely engages in farm-level financing for manure disposal, handling, and use—greatly mirroring the APCP’s investment activities. These activities are organized through three public tenders. According to the MAFWM, as of 4 November 2016, the first tender led to 82 applications from farms, of which 47 have been evaluated, and some have already received financing. The second tender led to 221 applications and was under evaluation during the PPAR field mission. Expert interviews during the field mission confirmed that the experience with the APCP had a pivotal role in the development of submeasure 4.1, which uses the same subsidy level for manure storages as the APCP. The continuation of measures similar to the APCP’s key component shows the project’s sustained relevance, facilitated through the implementation requirements of the EU Nitrates and Water Directives. Project Design and Relevance 2.8 Project design concept. The APCP design was based on the concept of interlinked components, which were expected to jointly lead to the overall PDO of increased use of environmentally friendly agricultural practices and eventual nitrate reduction in water bodies. Specifically, the project consisted of four components: mitigation of nutrient loads to water bodies from point source pollution; development and promotion of agri-environment measures; public awareness and replication strategy; and project management. The components designs were mutually reinforcing to address gaps in Croatia’s agricultural sector at the time of project preparation. Component 1 focused on supporting hard on-farm infrastructure investments for manure platforms and management equipment, and Component 2 complemented this by promoting improved practices and behavior change in manure management among farmers. Component 3’s purpose was to disseminate knowledge related to the activities supported by Components 1 and 2 at a national scale to more widely simulate replication. Component 4 was to ensure overall effective and efficient project management. The following describes each component:  Component 1: Mitigation of nutrient loads to water bodies from point source pollution (appraisal estimate: $14.61million; actual: $14.5 million, which is 73 percent of the actual total). This key component of the APCP aimed to establish a Nitrates Mitigation Investment Fund within the Paying Agency for Agriculture, Fisheries, and Rural Development to finance grants for 75 percent of the cost of manure storage and management infrastructure for medium-scale livestock farmers in the three priority counties of Osiječko-Baranjska, Vukovarsko-Srijemska and Varaždinska (appendix B contains a map). These grants, which beneficiary farmers matched, contributed to the construction of farm-level manure platforms and pits (and to the acquisition of collection and spreading equipment where the selection committee found it necessary). Furthermore, the component aimed to support Croatia’s water and soil monitoring and impact analysis efforts in collaboration with Croatian Waters (CW) - the state entity for water management - through farm-level well water testing and building on-farm water analysis stations, which were planned to become part of Croatia’s national water quality monitoring network managed by CW. These analysis stations were piezometer installments at selected manure investment farm sites (a piezometer is an instrument typically placed in boreholes to record water levels and quality).  Component 2: Development and promotion of agri-environment measures (appraisal estimate: $3.79 million; actual: $3.77 million, which is 19 percent of the actual total). This component aimed to assist the development and dissemination of a Code of Good Agricultural Practices. One of the main activities was publishing user-friendly guidelines 5 to educate Croatian farmers to implement the relevant provisions of good agricultural practices, focusing particularly on manure storage and application of organic fertilizer based on a healthy soil nitrogen balance. The other main activity was implementing a demonstration program through the Croatian Agricultural Extension Service Institute (CAEI) to train the livestock community in the three priority counties on sustainable, cost-effective manure management.  Component 3: Public awareness and replication strategy (appraisal estimate: $0.71 million; actual: $0.71 million, which is 4 percent of the actual total). The aim of this component was to familiarize the population at the national level with the APCP’s activities and thus raise the interest of potential clients. An example of an activity is assisting the CAEI to develop and maintain a website with detailed information on project activities and on technologies and land management systems appropriate for reducing point and nonpoint nutrient loads from agriculture to surface and groundwater bodies. The component also aimed to support knowledge sharing among key stakeholders, and technical staff training and participation in conferences on nitrates pollution management.  Component 4: Project management (Appraisal estimate: $0.59 million; actual: $0.58 million, which is 3 percent of the actual total). This component provided financing for project management, coordination, and M&E activities. A project coordinator managed the APCP with support from central and regional technical personnel (nitrates management specialists and regional nitrates management specialists, for example). The project implementation unit (PIU) of the closely related Agricultural Acquis Cohesion Project (AACP) established within the MAFWM managed administrative processes (procurement and financial management, for example). 2.9 Implementation arrangements. The project design selected the Ministry of Agriculture, Forestry, and Water Management (MAFWM) as the APCP’s implementing agency. MAFWM is the main government institution for integrated water management and for the protection of agricultural land from pollution by harmful substances. The APCP’s PIU was established within the MAFWM Department for Policy, EU, and International Relations and included staff from the parallel AACP on a cost-sharing basis. However, the APCP had a separate project coordinator for its own activities and technical personnel. 2.10 For Component 1, the project was designed to establish a strong relationship with the Paying Agency, which disburses all government (and now EU) market and structural payments to farmers. Throughout its implementation, the project financed two technical staff in the Paying Agency to manage and monitor the Nitrates Mitigation Investment Fund. Furthermore, the APCP was designed to be implemented in close collaboration with Croatian Waters (CW) to establish a network of piezometer water analysis stations on APCP beneficiary farms, which were planned to become part of CW’s national water quality monitoring network. 2.11 For Components 2 and 3, a strong collaboration with the CAEI was envisioned at design and fostered during implementation. For example, one regional staff member in each of the three APCP priority counties was financed as nitrates management specialists, whose main tasks were to promote the Code of Good Agricultural Practices and support field demonstrations and nitrate management among farmers. 6 2.12 Relevance of design. The APCP’s relevance of design is rated substantial given that the complementary design of the project’s components aimed to address Croatia’s insufficient coverage of manure storage and management tools, develop and disseminate guidelines on good practices to reduce water pollution from agriculture, and strengthen the capacity of national systems and promote transparent information sharing. However, this design also led to weaknesses in the results framework and shortcomings in the financing mechanism design of the Nitrates Mitigation Investment Fund (discussed in more detail later in this chapter). 2.13 Addressing of infrastructure insufficiencies in priority areas. The APCP was designed as a pilot project targeting Croatia’s three most important livestock producing counties along the Danube River and its tributaries, the Sava and Drava rivers. At appraisal, water pollution and high nitrate levels in those counties were the main issues. However, the government focused much of its attention in this area on wastewater treatment despite increasingly realizing the relevance of pollution from agriculture. According to the PAD, coverage of (adequate) manure storages was low and manure management practices outdated. The APCP was expected to expand the government’s focus in water pollution to agriculture and emphasize the need for interventions, especially in the context of the aspired EU accession and related directives requirements. The project channeled most of its financial resources through matching grants (75 percent by the APCP and 25 percent by farmers) for the construction of farm-level manure storages and the provision of modern manure management equipment (Component 1). This was a reasonable investment given the low coverage of manure storages, which can reduce leakage to groundwater and surface water if built and handled properly. 2.14 Development and promotion of innovative good agricultural practices guidelines and field demonstrations. By design, capacity-building activities for farmers complemented the infrastructure investments in manure storage infrastructure. The APCP developed a guidance book on good agricultural practices (the first of its kind in Croatia) and adapted it to the farmers’ practical needs. The project also distributed brochures nationally on specific topics (such as EU cross-compliance and biogas) in coordination with the CAEI, which still uses them today in trainings (appendix H). Another key capacity-building activity was field demonstrations of good agriculture-environmental practices. These soft measures embedded in the project design aimed to increase knowledge among recipient and neighboring farmers about proper manure handling and application. 2.15 Strengthening of national systems and transparency. The APCP’s implementation design involved key national institutions of the agricultural sector that the World Bank accredited through former interventions or capacity assessments. The practical application of APCP procedures (for example, clear definition of the beneficiary eligibility criteria, selection process of manure investments, payment recording and processing, and so on) allowed for institutional strengthening and prepared the Paying Agency in particular to test and adopt processes that are now applied for EU disbursements. Similarly, the CAEI was, by design, an important collaboration partner during project implementation. CAEI personnel continues to use APCP information materials in its service provisions to farmers and benefits from the capacity built during the APCP. Finally, the APCP was designed to promote information transparency for its stakeholder, which led to a project website and nationwide media involvement for public awareness building. 7 2.16 Weaknesses in the results framework. Although the APCP followed a logical underlying theory of change (matching grant investments in manure storages and equipment complemented by building farmers’ capacity will lead to better production practices and manure management, thus reducing nitrate leakage and non-environmentally friendly, traditional practices), the project’s results framework had weaknesses. For example, one of the two PDO indicators aimed at “increased national awareness of links between local actions and impact on water nutrient load.” This assumes that a project largely focused on three priority areas could have national influence with an arbitrarily selected target of 25 percent from an unknown baseline, and collecting survey data from a small sample of farmers that were not nationally representative. Furthermore, the project had a key outcome indicator that aimed at “a 10 percent reduction in discharge of nutrients into surface and groundwater in the three project regions.” This is an unrealistic outcome for a project as small as the APCP with effectively four years of implementation to attain, and without collecting evidence that could show even partial attribution to this objective. 2.17 Shortcomings in the financing mechanism design of the Nitrates Mitigation Investment Fund. The initial design of the financing mechanism used for the Nitrates Mitigation Investment Fund (Component 1) to support the construction of farm-level manure storages required that applicant farmers would need to pay the full amount of the manure investments upfront and be reimbursed by the APCP only after the construction was completed and verified. Given that the anticipated total investment per manure storage was about $60,000 to $80,000, the demand and take-up among eligible farmers was very low and challenging in the beginning because of budget and credit access constraints. This inhibiting factor was foreseeable during the project design phase. Chapter 3 discusses this in more detail. 3. Implementation 3.1 The APCP implementation experienced significant initial delays and uneven commitment among national project stakeholders, but a proactive PIU adopted activities beyond the original project design. 3.2 Three main factors led to the initial implementation delays. First, the project became effective in July 2008, but procurement of the complete PIU team of six people was not finalized until spring 2009. Second, the implementation challenge of the Nitrates Mitigation Investment Fund financing mechanism significantly delayed the implementation of Component 1. Consequently, the project’s regional coordinators increased their direct engagement with potential beneficiary farmers to motivate participation and, most important, the APCP adapted its financing mechanism relatively quickly during implementation by requiring only the farmers’ contribution of 25 percent of the total investment. These measures resulted in an increase in demand and investments in manure storages, but they came late in the project’s implementation. Approval of the first five manure storage investments occurred in mid-2010. Most storages were approved and built in late 2011 and the last seven were in early 2012. Third, farmers who were interested in investing in a manure storage platform and pit on their land faced major challenges in obtaining a building permit because of the often-needed legalization of their farms. Obtaining an official land title was extremely difficult, time consuming (administrative processing took about 12 months), and costly for many farmers (the average expenses for title documentation 8 were about $10,000 per farmer) given the outdated, often presocialist land registers at municipalities. The APCP engaged proactively and coordinated with the municipalities on a case-by-case basis to address this challenge, reducing processing time by 41 percent to seven months, which led to an increase in the applications received and contracts concluded. However, this adjustment process took considerable time and was unanticipated at project design. 3.3 The commitment of the national institutions involved in the APCP varied during implementation. The Paying Agency was a crucial part of APCP activities and hosted two of the PIU technical staff. Similarly, the Croatian Agricultural Extension Service Institute (CAEI) offered itself as a platform for dissemination of good agricultural practices promoted by the APCP through its extension services. However, the envisioned activities to be executed by Croatian Waters (CW)fell short during project implementation. Procurement of equipment for CW to conduct water and soil monitoring and analysis for the project, which was planned since the APCP design stage, did not materialize because of changes and disagreements between CW and the APCP during the bidding process on technical specifications for procurable equipment. This led to the overall cancellation of this activity late in the project implementation. Furthermore, the regular monitoring and inclusion of APCP-financed piezometers at beneficiary farm sites into CW’s national water quality monitoring network did not occur. During the PPAR mission, no one interviewed at CW knew of the existence of these piezometers, which was confirmed during the field visits (chapter 4 discusses this further). 3.4 Besides proactively addressing these challenges, the PIU also tried to go beyond the originally planned activities. For example, when the planned laboratory analyses with CW did not materialize, the PIU began collaborating with researchers from the Agricultural Faculties at the University of Oijsek and the University of Zagreb in 2011. It supported the procurement of laboratory equipment at both faculties to run field trials to provide farmers with information on required quantities of fertilizers to avoid overutilization of fertilizers. Although these activities demonstrate proactivity, their usefulness as an alternative to the originally planned water impact analyses with CW is questionable given the very short time period, small scope, and other shortcomings in the field trials design (appendix I provides a summary and assessment). However, another unplanned activity conducted by the PIU was gaining the cooperation of five local agriculture vocational schools that educate future farmers in the three project counties. The project organized several workshops and expert lectures on manure management for more than 200 students and arranged a study tour to best practice, environmentally friendly producers supported by the APCP. This is a good example of practical dissemination beyond the project’s initial target beneficiaries. Financial Management and Procurement 3.5 The APCP fiduciary and procurement personnel were the same ones who were responsible for the parallel Agricultural Acquis Cohesion Project established within the Ministry of Agriculture, Forestry, and Water Management. From the World Bank’s side, financial management specialists and procurement specialists were located in either the Croatia country office or a regional office throughout project implementation. This arrangement allowed direct interaction with the PIU. Regarding financial management, the project had unqualified and timely financial audit reports and no notable concerns during implementation. Regarding procurement, the project experienced significant challenges. 9 3.6 As previously discussed, the canceled procurement of laboratory equipment for CW involved a lengthy procurement complaint process. Eventually, the procurement of laboratory equipment for the agricultural faculties from the University of Oijsek and the University of Zagreb replaced the canceled procurement. Given the laboratory equipment’s different purposes, the change in implementation of this particular procurement decision raises concerns about the appropriateness of resource use to achieve the project development objective. The original design aimed to strengthen CW’s capacities for national water and soil testing, but the university laboratory equipment supports small-scale academic field trials partly related to the project’s good agricultural practices field demonstrations. The procurement plan’s remaining activities were implemented by project closure, with a few activities added during implementation (technical studies and portable water testing equipment, for example). 3.7 Delays in procurement, explained by recurrent slow responses from the World Bank to no objection requests, were a major concern raised during the PPAR mission. Among other cases mentioned, the procurements of manure storage investments and the piezometers for farm-level water analysis stations were noted as severely delayed because it took about six months to obtain a no objection response. The Implementation Completion and Results Report (ICR) did not mention this lack of responsiveness. However, interviews conducted during the PPAR mission found that these delays exacerbated the already delayed implementation, which makes the efficacy assessment of the project more challenging. Safeguards Compliance 3.8 The project was classified as environmental category B; the only safeguard triggered was Operational Policy 4.01: Environmental Assessment. Anticipated environmental issues related to the farm-level manure storage constructions, but these were insignificant in scope and scale, and adhering to engineering standards for construction, operation, and maintenance could address them. Potential negative environmental effects were expected to be managed effectively through an environmental management plan, which the government developed during project preparation. The World Bank’s environmental and social safeguards specialists were in the country office or the region during the entire implementation period. They collaborated with the PIU and local municipalities on assessing the farm-level manure storage constructions. Specifically, the environmental specialist conducted several site visits and surveys in all three project counties to assess the project’s effects related to noise, dust, waste, chance finds, reporting, complaints, potable water and sewage, fire protection, and familiarity with the project environmental management plan and monitoring. No safeguards-related issues or deficiencies in the plan’s implementation were reported during project implementation or the PPAR mission, so the project can be considered compliant on its safeguards commitments.3 Monitoring and Evaluation 3.9 Monitoring and evaluation (M&E) implementation of the APCP varied across the project’s two objectives. As discussed in more detail the Monitoring and Evaluation section in chapter 4 and in chapter 6, generally acceptable output and outcome evidence was collected concerning changes in farmers’ use of environmentally friendly agricultural practices. However, the collection of evidence on changes in nutrient discharge was deficient because planned analyses were not implemented and the results framework was overambitious. Despite this 10 shortcoming, the PIU tried to implement good M&E practices. For example, an M&E specialist was a permanent part of the PIU and ensured complete and timely submission of M&E progress reports. Furthermore, the project conducted three medium-scale surveys, which is commendable for such a relatively small pilot project. The survey results provided substantial information on the project results indicators and other issues relevant to the government agencies involved in the project implementation. However, questionnaires were not consistent and did not apply panel data collection, so some results are not comparable. 4. Achievement of the Objectives (Efficacy) 4.1 This PPAR uses four sources of evidence to assess the achievement of the project development objective (PDO) in outputs and outcomes: the ICR assessment (World Bank 2013), the ICR Review (IEG 2014), the IEG PPAR fieldwork conducted in November 2016, and the survey reports and studies conducted by the APCP. The IEG PPAR fieldwork applied several validation tools, including site visits, farm-level asset verification, interviews with a large sample of direct beneficiaries (both semi-structured face-to-face and by phone), and interviews with implementers and other stakeholders. appendices B through F provide a description of the fieldwork methodology and details on the different validation tools. For this assessment, IEG divided the PDO into two objectives: “to significantly increase the use of environmentally friendly agricultural practices by farmers in the Recipient’s Danube River basin” and “to reduce nutrient discharge from agricultural sources to surface and groundwater bodies.” This chapter discusses the efficacy of the respective objectives. Objective 1: Increased Use of Environmentally Friendly Practices 4.2 Manure storage investments. The APCP cofinanced 48 farm-level manure storages across the three priority counties through its Nitrates Mitigation Investment Fund (table 4.1). Thirty-seven of these beneficiaries also requested and were approved for cofinancing to buy manure and sludge management equipment, mainly manure spreaders. The project appraisal document (PAD) did not include a target value for the total number of manure storages to be built, but was based on the established monetary limit of $2.66 million supported by the APCP to the fund. 4.3 IEG interviewed 92 percent of the 48 manure storage beneficiaries. Face-to-face interviews were conducted during field site visits with 12 beneficiaries (25 percent of the total) and phone interviews were conducted with other 32 beneficiaries (67 percent of the total). Table 4.1 shows the distribution of these interviews by county. Based on these field site visits and interviews, IEG confirmed the construction of financed storages and equipment. Eighty-nine percent of the storages and 99 percent of the equipment were still in good condition, showing high quality infrastructure in line with the PAD objective to construct manure storages acceptable to EU standards. Furthermore, the same percentages of storages and equipment were still in use during the PPAR mission, and the beneficiaries were satisfied with them for their production, indicating continued usage 4.5 years after project closure. 11 Table 4.1. Project Manure Storage Investments and IEG Sample Total manure IEG field site IEG phone County storages visit sample interview sample Osiječko-Baranjska 16 6 7 Vukovarsko-Srijemska 18 3 14 Varaždinska 14 3 11 Total 48 12 32 Source: IEG. 4.4 The evaluation found that 42 percent of the beneficiaries interviewed in person did not have a manure storage before the APCP, and the 58 percent that said that it was either of lower quality (57 percent) or had less capacity (43 percent). All of the 12 beneficiaries interviewed face-to-face said that they would not have invested in manure storage on their own, most commonly because of financing constraints. When asked about the most noticeable impact of the APCP investment on their production, the 44 beneficiaries interviewed answered with the following: improved soil structure (76 percent), improved handling and more even manure spreading on fields (17 percent), and increased storage capacity leading to less frequent need to spread small volumes of manure (7 percent). Furthermore, 84 percent said that they lowered their use of mineral fertilizer since using the APCP manure storage, equipment, or both, estimating a median reduction of 30 percent. 4.5 The APCP results framework included a key indicator on “percentage of livestock farms in three participating counties that have animal waste storages” with the end-of-project target value of 25 percent (from a baseline of 6 percent). According to the ICR, the project well exceeded this target, as 88 percent of farms had “some sort of animal waste storages” at the end of the project. The source and scope of the 6 percent baseline is unclear. However, when IEG revised the APCP baseline report (produced in January 2010) and the endline report (produced in April 2012), it found that of the baseline of 327 farmers in the three priority counties, 46 percent had no manure storage at all and 54 percent had some manure storage (of which 8 percent were reportedly inadequate). Only 10 percent of farmers reported having waterproof manure platforms that prevent nutrient leaching. The 2012 endline of 242 farmers in the three priority counties reports that 60 percent had manure storage, which is higher than the average 56 percent for the nationwide endline sample of 785 farmers. Although a positive trend is evident, it is notable that the surveys were not based on panel interviews (except for APCP direct beneficiaries), so direct comparisons cannot be made. Furthermore, the endline provides no information on the storages’ quality (such as whether the platforms were waterproof). However, the field visits and interviews with beneficiaries confirmed the farmers’ appreciation and continued use of the storages constructed through the APCP and behavior change in agricultural practices through reduced mineral fertilizer use. 4.6 Promotion of good agricultural practices and capacity building among farmers. The APCP results framework had a target of reaching “at least 200 hectares of pilot good agricultural practices demonstration sites in each of the three counties.” The project largely surpassed this target by the end of the project, implementing field demonstrations at 87 farms covering 1,400 hectares between spring of 2010 and 2012. Technical assistance, individualized nutrient management planning, and demonstration of cover crop technology (green manure) were developed and provided to the participating farmers with the objective of reducing nitrogen and 12 phosphorus loss to water bodies (appendix G provides an example of a field with traditional postharvest practices and a field with green manure practices).4 Participation in the good agricultural practices demonstrations was voluntary, but farmers had to comply with clearly established selection criteria and a three-year commitment to take part. 4.7 According to the APCP 2010 baseline, only 43 percent of the 327 farmers interviewed in the three priority counties were aware of the good agricultural practices that the APCP promoted, and two-thirds expressed interest in taking part in field demonstrations. The endline reports that by April 2012, 91 percent of the 242 farmers interviewed in the three priority counties had taken part in at least one training on good agricultural practices, and 99 percent felt knowledgeable about good agricultural practices compared with 33 percent of 785 farmers in the nationwide survey sample. Ninety-four percent started to take preventive measures to protect groundwater and soil, surpassing the APCP indicator target of 10 percent -this is almost three times higher than the average response of 33 percent found in the nationwide survey of 785 farmers. However, a major shortcoming in measuring this indicator is that respondents were asked about all preventive measures they applied from a list of various measures, including organic manure usage, proper manure storage, three-year crop rotations, and the like. This multiple-option design makes it impossible to compare baseline to endline information on individual measures and, most important, mixes measures of nutrient management with measures of widely used practices like crop rotation that do not necessarily help control nutrient leaching. Despite these shortcomings, the endline results reveal that the most popular measures reported were organic manure usage (19 percent of responses), plowing harvest residues (18 percent), three-year crop rotations (13 percent), and manure storage 40 meters away from wells (12 percent), showing that most of the measures used related to improved manure management. Considering the shortcoming in measuring this indicator, 48 percent of the farmers interviewed in the three priority counties said that they apply at least one good agricultural practices measure on their whole farm surface compared with 38 percent of the nationwide sample. Furthermore, the ICR reports on the respective results indicator that 30 percent of farmers use soil analysis for nutrition planning at the end of the project compared with 5 percent at baseline. This is in line with the IEG finding that 25 percent of the 12 farmers interviewed in person confirmed that they commission regular soil analysis. 4.8 Of the 48 beneficiaries that received manure storages, 13 also took part in the field demonstrations. The IEG evaluation mission interviewed 10 of those beneficiaries (four in person and six by phone), representing 12 percent of the total 87 good agricultural practices beneficiaries. None of these farmers had applied good agricultural practices before the APCP field demonstrations, and all of them confirmed that they still applied them when the field visit took place in November 2016 (about half had begun with these practices in 2010). According to the farmers that IEG visited in person, the main reasons they continued to use the practices that the APCP introduced was that they saw results in improved soil structure, easier tillage, and improved yields. Similar to the effect stated from the manure storage investments, all field demonstration participants interviewed said that they use less mineral fertilizer because of applying good agricultural practices, with an average 25 percent reduction in fertilizer use. 4.9 Along with field demonstrations to induce behavior change in farming practices, the APCP produced several output activities to strengthen farmers’ capacities. Some of these activities interlinked closely with the APCP activities related to public awareness raising. 13 Outputs ranged from expert lectures and seminars for farmers to open field days with good agricultural practices extension service providers in nine locations in the priority counties. As previously noted, the PIU went beyond the original project plan and conducted an educational campaign and study tour with 200 students from five agriculture vocational schools to build capacities among young, prospective farmers. Furthermore, the unplanned collaboration with the agricultural faculties from universities in Oijsek and Zagreb included field trials aimed at providing farmers with information on the required quantity of fertilizer needed for most common cereal and vegetable crops to avoid fertilizer overutilization. The project provided the participating farmers with the results from trials on their fields to inform them of fertilizer over- or underutilization and the potential need for improved fertilizer management. Again, the usefulness of these studies for the involved farmers is straightforward, but the IEG evaluation questions the overall justification of these small-scale trials. 4.10 National awareness. The APCP was active in producing a number of outputs aimed at increasing national awareness of nitrate pollution and management in agricultural production. The project put much effort into developing the country’s first guidance book on good agricultural practices, disseminating it to 85,000 farmers in Croatia’s Farm Registry to inform them about measures and procedures of environmental protection within agricultural production. The project produced another 80,000 copies of guidance materials and disseminated them nationally. Based on IEG stakeholder interviews and field visits, the Croatian Agricultural Extension Service Institute and local agricultural ministries still use these materials. All materials are still publicly available on the APCP website, which provides useful information on agricultural pollution control (it was also a knowledge platform for participating farmers during implementation). Furthermore, the project conducted 390 events and awareness campaigns across Croatia and on national media (television and radio). 4.11 The ICR focused solely on the large number of output activities and did not show sufficient outcome evidence on the results indicator of “increased national awareness of links between local actions and impact on water nutrient load.” Stakeholder interviews during IEG’s field mission revealed that many Croatian farmers continue to favor nitrogen fertilizer, which is locally produced and inexpensive. Farmers perceive manure management and spreading as cumbersome and time consuming, especially those who lack the appropriate equipment (especially manure storage and spreaders) or knowledge of nutrient planning techniques. However, this perception seems to be changing, especially among larger farmers and mainly driven by the country’s aim to fully achieve EU environmental requirements for agriculture. Therefore, the APCP is considered successful in changing public awareness to some extent, given that the current Rural Development Program supports investments in manure disposal, handling, and use similar to the activities that the APCP promoted. 4.12 Overall, because the project produced most planned outputs for Objective 1 and IEG found sufficient evidence on most outcomes, IEG rated the efficacy of Objective 1 as substantially achieved. Objective 2: Reduced Nutrient Discharge from Agricultural Sources 4.13 Evidence collection for the global environment objective (GEO) indicator. The key target for the second objective was the GEO indicator of a “10 percent reduction in discharge of 14 nutrients into surface and groundwater in the three project regions.” The ICR evidence for this indicator was that 26 percent of the farms surveyed in the priority counties had “appropriate manure storage” at the end of the project. This information is weak evidence because the existence of an appropriate manure storage does not show the use of appropriate manure management or that the project had an effect on nutrient discharge and water quality. Related to manure management promotion, evidence collected during the IEG field mission interviews found that 84 percent of APCP manure storage beneficiaries reduced their use of mineral fertilizer by about 30 percent since using the manure storage, equipment, or both. This change in behavior can be assumed to lead to lower local nutrient leaching to soil and water from these farms. However, the project did not collect any evidence that showed such an effect. 4.14 Groundwater quality monitoring and analysis. The APCP design included activities to improve water monitoring and analysis at the beneficiary farm level to assess the quality of water flowing out of these farms and to expand Croatia’s national water quality monitoring network. The project planned construction of 30 water analysis stations consisting of three piezometers each. These activities were designed to be implemented in strong collaboration with Croatian Waters (CW), which would be responsible for the (post-project) piezometer monitoring and maintenance as part of its national groundwater monitoring program. The IEG field mission obtained the detailed locations of 28 of those water stations, but former PIU staff confirmed that 30 water stations had been built by the end of the project (which was also confirmed in the ICR). During the IEG site visits, eight of the 12 sample farms had at least one piezometer installed through the APCP, and phone interviews confirmed that 11 more farms had piezometers, for a total confirmed sample of 19 sites (63 percent of the 30 water analysis stations).5 However, none of the beneficiaries was aware of CW monitoring those piezometers, and no one informed them about any resulting water analysis. Most of the piezometers inspected during the IEG site visits were visible from the livestock stables or farmhouses, so farmers would have noticed regular inspection in the 4.5 years since project closure. Furthermore, during the site visits, IEG found most piezometers either locked or overgrown with vegetation and therefore unused (appendix F provides photos). Interviews with stakeholders from CW confirmed the suspicion that these water analysis stations were idle -no one interviewed knew of their existence or their integration into CW’s national water quality monitoring network. Even after IEG provided CW with the piezometers’ geographic locations, CW did not respond. It is unclear whether CW’s interest in collaborating with the APCP diminished after cancellation of the laboratory equipment procurement, but consequently the planned provision of evidence on water quality and the GEO on nutrient discharge reduction is lacking. 4.15 Well water analysis. As part of Component 1, the APCP sampled and tested more than 2,000 farm-level wells in more than 60 municipalities between 2011 and 2012. According to stakeholder interviews, this activity was well received and obtained strong support from the municipalities to demonstrate to farmers the risks of inappropriate nitrates management to rural communities. However, the project took samples only once and not systematically at different times during project implementation. Therefore, IEG considers this exercise more relevant to the APCP public awareness–raising activities instead of providing evidence on well water quality of the APCP investments. 4.16 Data availability on nutrient discharge in Croatia. It was impossible to obtain comprehensive data from CW on agricultural pollution and nutrient discharge to soil and water 15 bodies at the regional or county level. Despite several inquiries, CW did not provide the IEG mission with localized (historical) data on nitrates and phosphorus in the counties where the project was active, even though water-monitoring stations exist in those areas (besides the unused APCP piezometer stations). When asked about the scale or trends of pollution from agricultural sources or the high mineral fertilizer use in Croatia, CW consistently referenced the unavailability of reliable data on this issue, and neither the Ministry of Agriculture, Forestry, and Water Management nor CW claimed to have such information. IEG tried to obtain other data on nutrient discharge from agricultural sources in Croatia and organized meetings with the International Commission for the Protection of the Danube River and the International Sava River Basin Commission. However, neither commission had national data that was different from data that the Croatian government provided. 4.17 The most comprehensive, integrated, and updated overview of water quality in Croatia is the 2016–21 River Basin Management Plan prepared by CW (summarized in appendix J).6 The plan presents shortcomings regarding water quality monitoring, which are relevant for the assessment of the APCP’s GEO and M&E design. The time span of groundwater monitoring data (2007–12) is too short to determine valid trends. Instead of taking groundwater quality measurements at different depths in Croatia, they are measured at quite deep points. However, it is likely that nitrates from agricultural sources are abundantly present in the upper layers and might gradually move downward and reach deeper groundwater. Water soaks slowly through permeable layers of soil and rock, taking decades to reach the water table below, but still carrying nitrates. Furthermore, water quality monitoring stations for surface water are located along large rivers instead of smaller streams where agricultural pollution is more likely to occur. Therefore, a comprehensive assessment on the impact of agriculture on nutrient discharge into soil and water bodies is not feasible. 4.18 Appropriateness of Objective 2. The ICR did not provide any outcome evidence on Objective 2 and the PPAR could not obtain any. Although some evidence could have been generated if project implementation had happened as planned (especially activities related to the subproject-level water station monitoring and analysis of Component 1), IEG questions the overall appropriateness of Objective 2. The Implementation Completion and Results Report Review had already highlighted the main reason for this assessment, which the PPAR analysis confirmed: Objective 2 was too ambitious given the APCP’s scope and duration. The M&E section in chapter 6 discusses the reasons behind this argument. 4.19 Overall, despite the shortcomings described and the lack of outcome evidence on Objective 2, IEG assumes that the APCP’s pilot activities made some positive contribution to the objective and the GEO, given the EU-financed Rural Development Plan’s continued and scaled- up farm-level investments in similar preventive measures. However, considering these arguments, a longer assessment would be necessary after achieving a sufficient scale to draw conclusions that are more reliable. Consequently, IEG considers Objective 2 on the reduction of nutrient discharge modestly achieved. 16 5. Efficiency 5.1 At appraisal, the project conducted an economic and financial analysis based on theoretical assumptions on take-up of improved farm-level manure storage and handling and improved farming practices. The APCP’s assumed main benefit was a reduction in the ’s nutrient pollution of the Black Sea. The analysis estimated the annual reduction of dissolved nutrients flowing into the Black Sea to be 20 kilograms per hectare nitrate and 2.5 kilograms per hectare phosphate -about half of what was flowing to the Black Sea at appraisal. The analysis provided a cost effectiveness analysis from the farmers’ and society’s perspective. Overall, the financial cost-effectiveness calculation at the farm level led to the conclusion that the planned on-farm investments were not cost-effective for the farmer unless they were subsidized, but they were cost-effective for society overall because of negative environmental impacts. This result justified public investments and subsidies in farm-level manure storages.7 5.2 The project appraisal document also provided an incremental cost analysis baseline scenario that included the costs of various projects financed by several donors totaling $18.4 million, of which the Global Environment Facility would provide $6 million toward achieving global environmental benefits. The assumption was that the investments would have only a limited impact on water quality, but that the EU Instrument for Pre-Accession Assistance in Rural Development compliance measures developed under the APCP would become an integral part of the overall EU Common Agriculture Policy and thus have a large multiplier effect in accessing finance. 5.3 At project closure, the ICR referred to the analyses conducted at appraisal, but did not provide additional information on actual take-up of improved infrastructure or practices from the APCP interventions. IEG questions the rationality of the assumptions used in the economic and financial analysis presented for the APCP. For example, it concluded that 60 percent of farmers in the project area would need to adopt APCP-promoted practices within 10 years to reach the estimated annual reduction of pollutants flowing into the Black Sea. During the PPAR mission, IEG did not receive any evidence to support this assumed trend. Similarly, the analysis assumed that through the APCP public awareness campaign, field visits, and workshops, farmers from adjoining areas would adopt the environmentally friendly agricultural practices, resulting in a larger project impact. The ICR sees this as fulfilled based on the APCP beneficiary survey’s result showing that 94 percent of the farmers interviewed adopted a practice that led to a reduction in the nutrient loads promoted under the project. However, as discussed in the section on the efficacy of Objective 1, when revising the specific questionnaire, it becomes evident that “adopting a practice” can include widely used agricultural practices like crop rotation that do not necessarily help control nutrient leaching. Generally, the efficiency evidence would be more robust if the ICR had presented actual comparative data to underpin the theoretical calculations of the appraisal analysis. Most important, the unavailability of actual data on nutrient reduction in the ICR makes it impossible to validate the economic efficiency of the project in this aspect. 5.4 The ICR also discusses the choice of design for manure pits and platforms made entirely of concrete, noting that these may have been the most cost effective option and suggesting that although effective, the project technology was not efficient. It states that the project should have used the opportunity to test alternatives such a steel, ceramic, or fiberglass elements, “which are 17 quite readily available and manufactured in a number of the neighboring new EU member states and could have been demonstrated as cheaper alternatives” (World Bank 2013). In the project’s defense, the financial analysis at appraisal concluded, “The initial analysis indicates that concrete structures, at an estimated cost of €1,350 per livestock unit, will be the most cost-effective technology for manure storage.” The ICR also reports that the unit costs for manure construction used in the APCP were within the range considered normal in the Western Balkan region despite the substantial permit processes and quite stringent building requirements the APCP encountered. Overall, given the short project time frame, the initial difficulties with finding farmers to invest in manure storages, and the small scale of intervention, IEG considers that testing new technologies that are not known to be more cost-effective would have been another factor inhibiting implementation. 5.5 Finally, the ICR did not report on any of the operational inefficiencies that IEG encountered. As described in chapter 3, significant project implementation delays led to extremely slow disbursement up to one year before project closure. In June 2011, only 23 percent of the funds were disbursed and 52 percent were committed. Between 2011 and 2012, many of the key APCP activities were implemented, and by the last supervision mission in May 2012, disbursements had increased to 83 percent and all grant funds were committed. Overall, Component 4 on project management stayed within 98 percent of the appraised budget. Furthermore, and most important, the investment made for the construction of the farm-level water analysis stations (piezometers) were completely unused. Based on the receipt for 30 piezometers received during the IEG mission, the total cost of this investment was HRK 1,468,952, equivalent to about $240,000 (4.8 percent of the total grant amount). Besides being an example of inefficient financing resources use, IEG was surprised to learn during the field mission that Croatia had requested funding from the EU to expand its network of piezometer water analysis stations. IEG also questions the cost-effectiveness of replacing the planned but canceled investments for CW to conduct water impact analyses with small-scale fertilizer field trials. Considering all of these inefficiencies, IEG rated the project’s efficiency as modest. 6. Ratings Outcome 6.1 The APCP was an innovative pilot intervention for Croatia. IEG rated the relevance of project objectives as high because of its past and continued importance in the context of Croatia’s priorities and its consistency with World Bank strategies. The rating for relevance of design is substantial because the APCP design was appropriate given the country’s circumstances and development needs, with only minor shortcomings in the results framework and the Nitrates Mitigation Fund design. The project produced most of its planned outputs, and IEG considered the increase in the use of environmentally friendly agricultural practices of Objective 1 in terms of efficacy as substantially achieved. Regarding Objective 2 on reducing nutrient discharge in water bodies from agricultural sources, the APCP failed to provide sufficient outcome evidence and therefore IEG considered the efficacy of Objective 2 as modestly achieved. The efficiency of the project is modest because the project implemented project activities in the range of acceptable costs while contributing to the assumed project benefits. However, at project closure, no new economic or financial information was provided on 18 the actual take-up of improved infrastructure or practices. Furthermore, the project assessment revealed operational inefficiencies related to project implementation delays and that all APCP- financed water analysis stations were idle. Overall, the combination of the APCP’s ratings for relevance, efficacy, and efficiency results in an outcome rating of moderately satisfactory. Risk to Development Outcome 6.2 Agricultural pollution in soil and water bodies continues to be important to Croatia’s rural sector. The country’s EU membership since 2013 and the subsequent requirement to adopt and comply with the norms of the EU Nitrates and Water Directives have ensured continued government commitment and funding for environmental measures in the agriculture sector. The findings of this PPAR revealed a loss in institutional memory and use of APCP-financed infrastructure (current government staff are not aware of APCP piezometers, which are not used as planned as part of the national water quality monitoring network). However, Croatia’s current Rural Development Program includes a submeasure that engages in farm-level financing for manure disposal, handling, and use that is similar to the APCP investment activities, but at the national level and a much larger funding scale. Therefore, the prospects for infrastructure investments toward the development outcome are good. 6.3 However, when considering the APCP activities of incentivizing farmers to adopt good agro-environmental practices such as green manure, current institutional and funding priorities do not align. Promoting such practices and behavior change in farmers is not a focus of the Rural Development Program, which directs related funding toward physical infrastructure investments. The Croatian Agricultural Extension Service Institute and similar regional government institutions, whose leadership is interested in stronger promotion of agro-environmental practices, face underfunding challenges for such activities. The current policy environment provides weak incentives for farmers to adopt agro-environmental practices because local mineral fertilizer is available at low cost, and because of the low coverage of territory declared nitrate vulnerable (only 9 percent of Croatia’s territory is nitrate vulnerable, much below the EU average).8 Considering all of this, IEG rated the risk to development outcome as moderate. World Bank Performance QUALITY AT ENTRY 6.4 The World Bank developed an adequate technical design for the APCP investment components and ensured the integration of lessons from previous rural environmental and agricultural operations in the region. In particular, ICR lessons from the Poland Rural Environment Protection Project (1999–2004), which had implemented a similar approach to financing improved on-farm manure storages. The APCP PAD highlighted several lessons: the importance of involving local administrations, communities, and key decision makers early in project preparation; strong collaboration with the national advisory services to educate traditionally conservative farming communities to adopt innovative nutrient management technology and accept responsibility for their environmental risks; and demonstration activities to disseminate results and stimulate project intervention replication. Furthermore, the APCP design aligned strongly with the Croatian agricultural sector’s needs as stated in the World Bank’s strategic documents (country assistance strategy and Country Partnership Framework) 19 and in Croatia’s national Rural Development Program. Furthermore, the project was timely in addressing country priorities linked to the EU pre-accession process and the related requirements to prepare for compliance with the EU Nitrates and Water Directives. 6.5 Regarding implementation arrangements, the quality of preparation was reasonable. At preparation, it made sense to combine the Croatia Agricultural Acquis Cohesion Project (AACP) and the APCP’s PIUs on a cost-sharing basis, with some specialized personnel for the latter. However, this combination also brought implementation and supervision challenges, discussed later in this chapter. Given the project’s continuous, strong engagement with farmers on the local level, the design decision was to employ a separate APCP general coordinator in Zagreb supported by three county-based coordinators embedded within the local Croatian Extension Institute offices (one for each of the selected priority counties) was crucial. On the World Bank side, locating the task team leader (TTL) in the region at project preparation was highly beneficial. Based on IEG’s interviews with local stakeholders, this circumstance created an environment for fruitful communication exchange with the client, and for Croatia, one of receptiveness for the APCP’s innovative nature. 6.6 Despite these positive design features, the APCP had significant shortcomings in the quality at entry, given its design of the results framework and the Nitrates Mitigation Investment Fund. As discussed in the section Project Design and Its Relevance (chapter 2) and the monitoring and evaluation (M&E) section (chapter 3), the results framework was overambitious in its PDO indicator of increasing awareness at the national level. It was even more so in the expectation of Objective 2 to lead to a significant reduction in nutrient discharge, which was problematic given the APCP’s pilot nature. The World Bank should have addressed and corrected these issues proactively at the design stage. It missed the opportunity for candid discussion with the client and the donor (the Global Environment Facility) about how attaining nutrient reduction was an unrealistic objective for a planned five-year project of small scale, and that no adequate M&E design existed to show even partial attribution to this objective. Similarly, the quality of the Nitrates Mitigation Fund’s design had weaknesses related to the financing support required by beneficiary farmers. Considering its vast experience with rural development projects, the World Bank could have foreseen that the initial upfront payment design could lead to challenges in take-up, given farmers’ common budget or financing constraints for larger investments. 6.7 Overall, more rigorous quality checks by the World Bank on both design shortcomings would have been crucial, especially given the intention to up-scale the APCP pilot activities through either a future World Bank project or expected EU funding. Consequently, on balance IEG rated the quality at entry as moderately satisfactory. QUALITY OF SUPERVISION 6.8 The quality of World Bank supervision varied significantly throughout implementation. A major reason was the frequent changes of TTLs, as the APCP had four different TTLs within the project’s five-year duration. IEG found that the client perceived the TTL transitions as poorly managed because the World Bank’s supervision role lacked continuity and consistency, which disrupted the project implementation flow. According to interviews with different stakeholders, the World Bank’s most constructive and useful supervision was during project preparation and closing, when TTLs were located in the region. 20 6.9 Overall, the IEG mission had the impression that the APCP was treated as an add-on activity to the parallel, larger AACP loan project’s regular supervision work of the. All supervision missions, status reports, Aide Memoires, and other project documents were combined for both projects. Combining supervision activities was logical (such as organizing and executing one mission to the same country, meetings with the same high-level clients at the central level, and so on). However, combining most project documentation resulted in weak products and product management. For example, the APCP and AACP’s ICRs were combined, resulting in the loss of much of the detail needed for an assessment (the same page and content requirements for a one-project ICR were applied to the joint ICR). Furthermore, because the APCP was in effect a stand-alone grant separate from the AACP loan, a separate ICR should have been prepared. 6.10 Similarly, the World Bank’s attention to the project implementation flow and to ensuring good results was deficient during supervision. Major delays of several months in providing no objection responses to the PIU for crucial activities was a major complaint raised to IEG and illustrates undesirable World Bank supervision. Furthermore, the problems with the result framework were not resolved during supervision. When delays occurred, the PIU was led to believe that the demonstration of outcome evidence at closure should not be a major concern, leading to a vast underestimation of the results information requested for the ICR preparation and the subsequent M&E quality rating. In this instance, the World Bank failed to provide adequate advice and discuss the potential need for restructuring the results framework. Instead, the World Bank team emphasized the need for quick disbursements, which put a lot of pressure on the PIU considering the reasons for implementation delays. 6.11 Finally, from IEG’s perspective, the World Bank’s execution of its responsibility in proper project documentation filing is unsatisfactory. During preparation of this PPAR, IEG found almost no APCP-related reports on the operations portal except for key documents. Much of the filed correspondence is from the AACP (which has its own operations portal database), so IEG had to obtain specific project reports (such as survey results reports and progress reports) in country from former PIU staff and stakeholders. 6.12 Considering these issues, IEG rated the quality of World Bank supervision as moderately unsatisfactory. Consequently, the moderately satisfactory rating for quality at entry and the moderately unsatisfactory rating for quality of World Bank Supervision, and considering the moderately satisfactory outcome rating, IEG gave an overall rating for World Bank Performance of moderately satisfactory. Borrower Performance GOVERNMENT PERFORMANCE 6.13 In assessing government performance, IEG distinguishes between the central government in Zagreb and government institutions in the regions, particularly in the three priority counties of the APCP. 6.14 Changes occurred in the central government during project implementation and after project closure (elections were held in November 2007, December 2011, November 2015, and September 2016). Consequently, key staff in the institutions involved in the APCP changed, 21 which led to challenges in the continuity of interventions and awareness and the use of APCP outputs. According to the ICR, government support to the APCP was perceived as “fairly consistent” throughout implementation. However, based on the IEG mission, it seems the government considered the APCP to be a minor concern (and paid it little attention), given its small size and grant financing. Croatian Waters (CW) showed significant shortcomings in performance compared with its initially planned role, especially regarding integration of the APCP-financed water analysis stations in its national water quality monitoring system. The government did not make any counterpart contribution to the APCP, though at appraisal the expectation was that it would contribute $13.9 million from the parallel IBRD-financed AACP (at AACP closure, the government had provided only 27 percent of the originally planned $13.6 million to the AACP). 6.15 Regarding regional government institutions, the IEG mission found that the municipalities and regional offices of the Ministry of Agriculture and the Croatian Agriculture Extension Institute were strongly engaged and enthusiastic about the APCP. This was rooted in their concern about agricultural pollution in their respective regions and the potential they saw in the project to benefit their local farmers. Regional government institutions showed their ownership and engagement in project activities through their continued support during implementation. For example, municipalities were forthcoming in finding a solution to the land rights issue for farmers to obtain a building permit for the manure storages. Similarly, municipalities supported the APCP’s various dissemination activities, such as taking part in educational events, and dissemination and display of good agricultural practices brochures (appendix H) and the like. During independent interviews with different regional representatives, IEG saw the support for the APCP and the regional institutions’ continued interest in agricultural pollution. Based on these findings and the key role that the central government was expected to take for the APCP, IEG rated overall government performance as moderately unsatisfactory. IMPLEMENTING AGENCY PERFORMANCE 6.16 The APCP’s implementing agency was the Ministry of Agriculture, Forestry, and Water Management (MAFWM), and the PIU was established within the MAFWM Department for Policy, EU, and International Relations. It included staff from the AACP on a cost-sharing basis, but the APCP had a separate project coordinator and its own technical staff in Zagreb and in the three priority counties. Overall, the PIU was staffed with qualified and highly engaged professionals and performed well throughout project implementation. Based on IEG interviews with various stakeholders, IEG confirmed that the PIU conducted its activities with great dedication despite changes in the institutional environment. Furthermore, it was responsive to implementation issues, and when delays or cancellations of planned activities occurred (for example, slow up-take on manure storages because of financing requirements for farmers, or collaboration with CW on water impact analysis), PIU staff was flexible and creative in finding solutions or alternatives. The PIU’s proactivity in reaching out to different stakeholders - including farmers, local municipalities, nongovernmental organizations, academia, and so on- was exemplary and showed its dedication to the project. 6.17 ICR evidence and conversations with former task team members reveal that the PIU’s working relationship with the World Bank was generally good. However, significant implementation delays occurred, partly because of frequent changes in TTLs and the World 22 Bank’s slow responses. A more assertive attitude from the PIU toward the World Bank might have lessened these delays. Furthermore, the PIU could have managed some of the M&E implementation shortcomings better. For example, the PIU was proactive and ensured the completion of three farm-level surveys, but the questionnaire design was inconsistent and therefore the results are not directly comparable. Given these findings, IEG rated implementing agency performance as satisfactory, with an overall borrower performance rating of moderately satisfactory. Monitoring and Evaluation 6.18 Design. The APCP’s M&E design was generally reasonable. An M&E specialist was to be part of the PIU from the start of the project, and three farm-level surveys and water impact analyses through integration with the existing national CW system were planned. However, the results framework had weaknesses, particularly in the PDO indicator on “increased national awareness,” which was measured only through outputs (number of promotional materials distributed), and the global environment objective (GEO) indicator on “reduction in discharge of nutrients,” which was too ambitious and difficult to be attributed to the small-scale project activities. Specifically, the GEO’s expectation at design that an innovative pilot project with an expected five-year duration could produce a 10 percent change in nutrient discharge into water bodies was unrealistic because of two factors. First, the APCP’s duration was too short for achieving such a high-level objective -even if implementation delays had not occurred- because it takes a long time for nitrates to leak into groundwater (APCP-financed water stations were aimed at groundwater analysis). Second, the such a small-scale project (investments at 48 farms for improved manure storage and management and at 87 farms to strengthen capacity in environmentally friendly agricultural practices in three counties that together contain more than 25,000 registered agricultural holdings) cannot be expected to significantly affect nutrient levels in water bodies that multiple other agricultural and nonagricultural actors also affect. Overall, the M&E design should have considered that the APCP was an innovative pilot project of limited scope, and thus avoid unreasonable expectations regarding achievable objectives. 6.19 Implementation. The PIU was disciplined in hiring a permanent M&E specialist, establishing a simple monitoring system appropriate for the project size, and conducting all three planned farm-level surveys. The project conducted three medium-scale surveys (baseline with 327 respondents in the project’s three priority counties, follow-on with 731 nationwide respondents, and endline with 785 nationwide respondents), which went beyond the originally planned design and is rarely found in small pilot projects. However, the M&E implementation was mixed in generating sufficient, adequate results evidence. The M&E management information system collected ample information on project outputs, but data on outcomes not as well recorded. Regarding Objective 1, M&E activities presented generally acceptable output and outcome evidence because the survey results provided substantial information. However, IEG noted relevant shortcomings in the questionnaires. Some questions were incomparable across surveys, especially regarding the specific preventive measures against nutrient discharge that were adopted. The questions did not apply panel data collection, and the simplicity of analysis focused on basic descriptive statistics without any further sophisticated analysis. In retrospect, a more consistent design of the surveys’ sampling strategy and questionnaires would have been beneficial for the outcome assessment. An important drawback of the M&E implementation is that the evidence collection on Objective 2 was deficient because the planned water impact 23 analyses did not occur and the GEO indicator’s design was overambitious. Furthermore, an assessment and continued monitoring on the functionality (and use) of the farm-level infrastructure investments were impossible because manure storage construction started late in the project implementation attributable to significant delays (discussed in chapter 3). 6.20 Utilization. This PPAR found little evidence regarding M&E utilization. The ICR did not include any information on this issue, but the ICR Review states that based on discussions with the task team, “M&E findings were used to redefine planned activities and allocate available funds within each category.” IEG confirmed during the PPAR fieldwork that some of the results collected by the farm-level surveys were presented at a project closing workshop in June 2012. IEG obtained the workshop presentations, and the results presentations focused strongly on project outputs (for example, the number of manure storages built, brochures disseminated, events conducted, and so on). Furthermore, all survey results reports were published on the APCP website. Based on the evidence on M&E design, implementation, and utilization, the overall M&E quality rating is modest. 7. Lessons 7.1 IEG’s performance assessment of the APCP experience suggests the following lessons, grouped under three interrelated categories: ACHIEVING LONG-TERM BEHAVIOR CHANGE  Local stakeholder involvement: Innovative projects require strong involvement of local stakeholders to generate advocates of behavior change. The involvement of farmers, government institutions, academia, and nongovernmental organizations is highly important to secure interest at the local level and to stimulate the sustainability of new practices.  More than infrastructure: Combining support for infrastructure and activities to promote behavior change can be effective. Addressing constraints of a physical and social nature at the same time through complementary activities is essential to incentivize and enable beneficiaries to adopt new practices.  Realistic objectives: Objectives of pilot interventions need to be realistic, especially when aiming at longer term goals that require behavior change. The environmental objective of the APCP was unattainable in the given project time frame and scale. Mid- level outcomes consistent with a longer time frame are more useful in illustrating progress against the objectives. ADEQUATE IMPLEMENTATION  Enabling environment: Early analysis and addressing constraining factors in a project’s enabling environment can reduce implementation delays. The APCP struggled with significant delays due to land titling issues. Tackling this issue from early on allowed the team to diminish the effects on implementation.  Effective pilots: Pilot interventions are most effective when they can take a practical approach to implementation considering future scale-up. The ability of the APCP 24 implementing agencies to test processes and refine them was crucial for the smooth transition to managing EU funding activities.  World Bank involvement: The World Bank team leadership’s consistent presence and responsiveness is crucial to ensuring the success of innovative project approaches. The APCP project planning and execution was smoother when the TTL was easily accessible, especially during innovation launches or when intense support was required. CLIENT COMMITMENT AND INCENTIVES  Lack of client incentives: Small, stand-alone grant investments without government contribution risk not obtaining a sufficient level of attention. Both the World Bank and the client had little concern for the APCP implementation because of its small scale and because it was not combined with a loan.  Effective communication: A successful project requires effective communication beyond the technical level to secure political buy-in and future funding. The APCP did not emphasize reaching out to key decision makers, and without the EU requirements related to agricultural pollution, current funding for APCP-like activities might not have been secured. References APCP (Agricultural Pollution Control Project). 2007. APCP Operational Manual. Draft, October 2007. APCP, Zagreb. _____. 2009. APCP Survey I. Questionnaire for Farms. APCP, Zagreb. _____. 2010. Manual on Good Agricultural Practices: Demonstracijski Emonstracijski Program Mjera Dobre Poljoprivredne Praske. APCP, Zagreb. _____. 2010. APCP Survey II. APCP, Zagreb. _____. 2012. APCP Survey III. APCP, Zagreb. _____. 2012. APCP Final Report. APCP, Zagreb.. Available at: http://www.voda.hr/sites/default/files/plan_upravljanja_vodnim_podrucjima_2016._-_2021.pdf CW (Croatian Waters). (2013). River Basin Management Plan 2009-2012. Zagreb. Available at: www.voda.hr/sites/default/files/dokumenti/plan.pdf Hrabánková, A., J.V. Datel, J. Klír, and V. Kadić. 2012. Designation of Nitrate Vulnerable Zones in Croatia. T.G. Masaryk Water Research Institute and Ekotoxa, Prague. IEG (Independent Evaluation Group). 2014. “Implementation Completion Report Review, Croatia Agricultural Pollution Control Project.” Report No. ICRR 14253, World Bank, Washington, DC. _____. 2016. “Approach Paper—Toward a Clean World for All: An Evaluation of World Bank Group Support for Pollution Management.” Board Report No. 107394, World Bank, Washington, DC. IMF (International Monetary Fund). 2017. International Financial Statistics, National Currency Croatian Kuna per US Dollar, Annual Average (2007–12). Washington, DC. Kuderna, M., K. Wagner. H. Janetschek, and T. Übleis. 2012. Croatia Agri-Environmental Program Implementation Assessment for Agriculture Pollution Control Project. Vienna: WPA Beratende Ingenieure GmbH. Liska, I., ed. 2015. The Handbook of Environmental Chemistry 39: The Danube River Basin . Berlin: Springer- Verlag. 25 MAFWM (Ministry of Agriculture, Forestry, and Water Management). 2015. Rural Development Programme of the Republic of Croatia for the Period 2014-2020. Zagreb: Republic of Croatia. Republic of Croatia. 2007. Environmental Management Plan, Croatia Agricultural Pollution Control Project . Report No: E1689. Zagreb: Republic of Croatia. _____. 2011. 2011 Report: APCP Nitrates Mitigation Investment Fund . Ministry of Agriculture, Zagreb. Slrhed, Coden. 2015. Statistical Yearbook of the Republic of Croatia. Zagreb: Croatian Bureau of Statistics. Trantinová, M. 2012. Scenarios for the Implementation of Nitrates Directive: Assessment of Long-Term Costs and Benefits. Prague: Ekotoxa. University of Zagreb. 2014. Utjecaj poljoprivrede na onečišćenje površinskih i podzemnih voda u Republici Hrvatskoj. Zagreb: Agronomski fakultet Sveučilišta u Zagrebu. World Bank. 2004. “Country Assistance Strategy for the Republic of Croatia.” Report No. 30717, World Bank, Washington, DC. _____. 2004. “Country Partnership Strategy for the Republic of Croatia for the Period FY14–17.” Report No. 77630, World Bank, Washington, DC. _____. 2007a. “Integrated Safeguards Datasheet, Appraisal Stage, Croatia Agricultural Pollution Control Project.” Report No. AC3080, World Bank, Washington, DC. _____. 2007b. “Project Appraisal Document, Croatia Agricultural Pollution Control Project.” Report No. 41212. World Bank, Washington, DC. _____. 2008. “Grant Agreement Global Environmental Facility Trust Fund TF090845-HR, Croatia Agricultural Pollution Control Project.” World Bank, Washington, DC. _____. 2012. “Aide Memoire of Final Implementation Support Mission. Croatia Agricultural Pollution Control Project.” World Bank, Washington, DC. _____. 2013. “Implementation Completion and Results Report, Croatia Agricultural Acquis Cohesion Project and Agricultural Pollution Control Project.” Report No. ICR2502, World Bank, Washington, DC. _____. (2007-2012). Implementation Status Reports for Croatia Agricultural Pollution Control Project from Operations Portal. Washington, DC. ———. 2017. World Development Indicators 2017. Washington, DC: World Bank. https://openknowledge.worldbank.org/handle/10986/26447 License: CC BY 3.0 IGO. Znaor, D. 2011. “Pressure on Croatian Water Resources Caused by Nitrates and Phosphorus of Agricultural Origin.” Study commissioned by the Ministry of Agriculture, Fisheries, and Rural Development of the Republic of Croatia, Zagreb. 1 The Danube River Basin is one of Europe’s largest cross-boundary river catchment areas, spanning more than 19 different countries and draining large parts of central and southeastern Europe before flowing into the Black Sea. The 19 countries are Albania, Austria, Bosnia and Herzegovina, Bulgaria, Croatia, the Czech Republic, Germany, Hungary, Italy, the former Yugoslav Republic of Macedonia, Moldova, Montenegro, Poland, Romania, Serbia, the Slovak Republic, Slovenia, Switzerland, and Ukraine. 2 About 1.1 million hectares of agricultural land are inscribed in the Land Parcel Identification System, of which about 77 percent is arable land, 16 percent is permanent grassland, 5 percent is orchards and vineyards, and 2 percent is other permanent crops. The sector consists of mostly smallholder farmers and family farms, with 90 percent of all agricultural holdings listed in the system cultivating less than 10 hectares. Of the country’s 300,000 agricultural holdings, about half are registered in the country’s farm register administered by the Paying Agency for Agriculture, Fisheries, and Rural Development, which means that they sell their produce on the market and are eligible to receive direct payments (subsidies). However, only 66 percent of eligible holdings actually receive subsidies. As of December 2015, about 40 percent of registered agricultural holdings in Croatia reported raising 26 livestock. Of these, 48.8 percent raise cattle and produce milk, 31.7 percent raise sheep and goats, 12.7 percent raise pigs, 6.1 percent raise horses and related animals (such as donkeys), and 0.6 percent raise poultry. 3 The last supervision missions’ Aide Memoire confirmed safeguards compliance more extensively (World Bank 2012), and the ICR confirmed it briefly (World Bank 2013). 4 The APCP technicians assessed the soil structure and the participating farmers’ production needs, which led to offering more than ten different cover crop options to farmers. For each crop detailed instructions and trainings were provided to ensure adequate application and gain farmers’ interest to continue these practices after project closure. 5 Some farmers interviewed by phone were not sure if a piezometer was installed on their land because some have multiple plots. This lack of awareness seems surprising, but IEG learned during site visits that farmers generally were not told about piezometer construction on their plots. 6 The River Basin Management Plan (CW, 2013) found that 37 percent of rivers do not have a satisfactory status on nitrates content and 41 percent on phosphorous content. Agriculture is a significant source of both of these pollutants, but the plan does not include a detailed assessment on agriculture. Regarding groundwater bodies in the Danube River Basin, poor groundwater chemical status was found in Varaždinska County (Croatia’s poultry production center) based on high nitrates, which expert interviews confirmed during the IEG mission. Overall, the plan concludes that comparative figures for 2009–12 show no improvement in water quality, mainly because of the modest implementation progress of protection measures. The comparison between the 2009 and 2012 figures on nitrates and phosphorous content in rivers suggests that the chemical status has worsened, as the number of river body streams with unsatisfactory conditions for nitrates rose by nearly 70 percent and almost 20 percent for phosphorous. These pollutants have many sources (for example, households, industry, agriculture, and so on), but no detailed analysis or information related to agriculture was provided to IEG. 7 Specifically, the estimate for the average cost of building a manure storage facility was about €1,350 per livestock unit (LU). Assuming a 30-year usage period, the estimate for the average annual depreciation cost was €45 per LU. Assuming the opportunity cost of capital is 6 percent per year, the annual opportunity cost would be €81 per LU, which is an annual financial cost of about €126 per LU. By investing in manure storage systems, farmers would incur an annual charge of €126 per LU per year while generating a benefit of about €38 per LU per year. Therefore, from the farmer’s perspective, it would be more cost effective to buy nutrients as fertilizer than to invest in manure storage. A 75 percent subsidy would make manure storage costs neutral for the farmer. From the social perspective, the value of the associated environmental damage of nutrient loss and public investments would be about €244 per LU per year, which is double the annual cost of the proposed measure. 8 The implementation of the EU Nitrates Directive includes the designation of Nitrate Vulnerable Zones (NVZs), defining them either lands that drain into polluted waters or waters at risk of nitrate pollution. Alternatively, EU member states can also choose to declare their whole territory as NVZ (appendix A provides more details). 27 Appendix A. Basic Data Sheet Agricultural Pollution Control Project (TF-90845) Table A.1. Key Project Data (US$, million) Appraisal Actual or Actual estimate current estimate (% of (US$, (US$, millions) appraisal millions) estimate) Total project costs 19.99 19.79 98.9 Grant amount 5.0 4.99 99.9 Cofinancing n.a. n.a. Cancellation n.a. n.a. Sources: APCP Implementation Completion and Results Report; World Bank Operations Portal. Note: For the remaining funds, a refund application of $29,883.98 is recorded for January 31, 2013. Table A.2. Cumulative Estimated and Actual Disbursements FY08 FY09 FY10 FY11 FY12 Appraisal estimate (US$, 0.40 1.77 3.06 4.35 5.00 millions) Actual (US$, millions) 0.00 0.40 0.80 1.33 4.99 Actual (% of appraisal) 0 23 26 31 99.9 Date of final disbursement: 11/05/2012 Sources: APCP Project Appraisal Document; APCP Implementation Completion and Results Report. Table A.3. Project Dates Original Actual Concept review 09/28/2006 09/28/2006 Appraisal 07/31/2007 08/03/2007 Board approval 12/06/2007 12/06/2007 Signing 3/11/2008 3/11/2008 Effectiveness 01/20/2008 07/31/2008 Mid-term review 12/01/2009 11/15/2010 Closing date 07/31/2012 07/31/2012 Sources: APCP Project Appraisal Document; APCP Implementation Completion and Results Report. 28 Table A.4. Staff Time and Cost Staff time and cost (World Bank budget only) Staff weeks Stage of project cycle (number) $, thousandsa Lending FY2005 36.09 236,542 FY2006 31.04 128,349.80 Total 67.13 364,891.80 Supervision/ICR FY2006 5.61 14,169.24 FY2007 26.85 82,968.98 FY2008 28.57 109,343.30 FY2009 35.33 113,611 FY2010 29.2 88,068.26 FY2011 21.42 53,539.43 FY2012 6.98 15,641.13 FY2013 9.5 22,746.16 Total 163.46 500,087.50 Note: These figures are based on the APCP Implementation Completion and Results Report, which provides total staff time and costs for both the Croatia Agricultural Pollution Project and the Agricultural Acquis Cohesion Project. a. Costs include travel and consultant costs. 29 Table A.5. Task Team Members Title (at time of appraisal and Responsibility or Name closure, respectively) Unit specialty Lending Aleksandar Nacev Senior Agriculturist ECSSD Task Team Leader Meeta Sehgal Operations Officer ECSS1 Task Team Member Sharifa Kalala Program Assistant ECSSD Administrative Support Solvita Klapare Environmental Economist EASER Task Team Member Garry Smith FAO Consultant FAO Technical consultant Supervision/ICR Aleksandar Nacev Senior Agriculturist ECSSD Task Team Leader Michael G. Carroll Consultant ECSSD Task Team Leader Sari K. Soderstrom Sector Manager ENV Task Team Leader Vera Dugandzic Senior Operations Officer ECSSO Task Team Leader Antonia G. Viyachka Procurement Specialist ECSO2 Procurement Rural Development Daniel Gerber Specialist ECSS1 ICR author Garry A. Smith FAO Consultant FAO Technical Consultant Helen Z. Shahriari Senior Social Scientist AFTCS Social Assessment Financial Management Lamija Marijanovic Specialist ECSO3 Financial Management Meeta Sehgal Operations Officer ECSS1 Task Team Member Mirela Mart Consultant ECSOQ Task Team Member Junior Professional Mustafa Ugur Alver Associate ECSS1 Task Team Member Natasa Vetma Senior Operations Officer ECSS3 Task Team Member Solvita Klapare Environmental Economist EASER Task Team Member Dubravka Jerman Program Assistant ECCHR Task Team Member 30 Appendix B. Fieldwork Methodology The overall aim of the Agricultural Pollution Control Project (APCP) was to develop and disseminate good agricultural practices to mitigate surface and groundwater pollution of agricultural origin in Croatia’s Danube River Basin. This performance assessment of the APCP aims to validate the relevance, efficiency, and effectiveness of the project results reported in the Implementation Completion and Results Report (ICR) and to assess the condition and functionality of the physical subproject investments (manure storages and piezometers) four years after closure. For this assessment, IEG conducted a mission in Croatia for two-and-a-half weeks during October and November 2016, including four days of field visits. Several validation tools were applied during the mission, including site visits, farm-level asset verification, semi-structured face-to-face interviews with a sample of the direct beneficiaries of manure storage investments, semi-structured phone interviews with the remaining direct beneficiaries of manure storage investments, and interviews with implementers and other stakeholders. Sampling of Site Visits Based on the project’s priority-setting on the country’s main agriculture production centers in the Danube River Basin, APCP subproject investments were implemented on beneficiary farms in the two neighboring counties of Osiječko-Baranjska and Vukovarsko-Srijemska in Eastern Croatia and in Varaždinska in Northern Croatia. The ICR reports that the APCP directly invested in the construction of 48 farm-level manure storages (and management equipment) across these three counties in addition to installing water analysis stations (piezometers) at a subset of these farms. Furthermore, 87 farms took part in field demonstrations of good agriculture- environmental practices, particularly green manure. Therefore, for this assessment, the unit of analysis is at the farm-level. IEG selected a sample of farms that directly benefited from subproject investments in manure storages based on the following criteria:  Geographic coverage. IEG ensured that a sample of farms in all three project counties would be visited during the field mission because there were no access or travel restrictions. Given the relatively even distribution of total subproject investments across these farms (16 in Osiječko-Baranjska, 18 in Vukovarsko-Srijemska, and 14 in Varaždinska), IEG originally sampled equally across the counties. However, IEG selected replacement farms because of a lack of response or the availability of selected farms on the mission’s field visit days. The final sample included more farms than planned in Osiječko-Baranjska and less in Vukovarsko-Srijemska. This does not raise major concerns for the assessment results given the two adjacent counties’ geographic proximity and similarity in agricultural production (mostly cattle and some pig farms). Figure B.1 shows a map with APCP manure storage investment and field demonstration sites, highlighting the sample of sites that IEG visited.  Investment size. IEG divided beneficiary farms in each county into two categories: farms that received equal to or more than the average APCP subproject investment amount of about $52,000 for manure storages, and farms that received support less than this amount (APCP subproject investments ranged from the smallest support of $16,000 to the highest 31 of about $81,500). To ensure broad coverage of the different investment sizes, about half of the farms sampled were purposively selected to be in the low investment category and half were in the high investment category. Based on these criteria, IEG sampled and visited 12 of the 48 APCP manure storage investments beneficiary farms during four field visit days (25% of total). Table B.1 shows the distribution across the three project counties. In addition to the manure storages, IEG found that eight of the 12 sample farms also had at least one piezometer installed by the APCP, representing about 27 percent of all water analysis location sites provided to IEG by the former PIU coordinator. Table B.1 Project Coverage and IEG Sample County Manure storages total IEG fieldwork sample Osiječko-Baranjska 16 6 (10 with piezometers) (4 with piezometers) Vukovarsko-Srijemska 18 3 (8 with piezometers) (2 with piezometers) Varaždinska 14 3 (10 with piezometers) (2 with piezometers) Total 48 12 (28 with piezometers) (8 with piezometers) Table B.2 shows the farm sample composition by geographic location (county and village), investment size (low or high), and livestock type (cattle, pigs, or poultry). Of the 12 farms that IEG visited, half are equal to or more than the average APCP subproject investment amount and half are less than this amount. Furthermore, 58 percent of the sample farms were cattle farms, 25 percent were pig farms, and 17 percent were poultry farms. This is similar to the composition of the 48 farms that received APCP investment support, of which 69 percent were cattle farms, 17 percent were pig farms, and 15 percent were poultry farms. Table B.2 IEG Sample Composition County Village Livestock Type Investment Level Osiječko- Selci Đakovački Cattle High Baranjska Satnica Đakovačka Cattle High Budimci Cattle Low Antunovac Pigs High Črnkovci Pigs Low Podgajci Pigs Low Vukovarsko- Berak Cattle High Srijemska Štitar Cattle Low Babi Greda Cattle High Varaždinska Novi Marof Cattle Low Petrijanec Poultry High Druzbinec Poultry Low 32 Figure B.1 Map of APCP Subproject and IEG Sample Sites 33 Validation Tools The IEG mission applied several validation tools, described in this section. ASSET VERIFICATION At the site visits of the 12 sample farms, the mission verified the existence of the investments listed by the project (that is, manure storage, manure management equipment, and piezometers, where applicable) and assessed their condition. IEG used visual verification in most cases (taking photographs where possible), or verbal confirmation by the beneficiary if the equipment was not onsite during the visit or piezometers were not accessible. Appendix F contains an asset verification summary table for all sites visited during the IEG mission. SEMI-STRUCTURED FACE-TO-FACE INTERVIEWS IEG conducted semi-structured face-to-face interviews with a sample of direct beneficiaries. During the site visits, 12 direct beneficiaries of manure storage investments (the farm household head) were interviewed one-on-one using a semi-structured survey. Appendix C lists the questions used in these interviews. SEMI-STRUCTURED PHONE INTERVIEWS IEG was unable to conduct face-to-face interviews with 32 of the remaining 36 direct beneficiaries of manure storage during the site visits, but could contact them by phone for interviews. These semi-structured phone interviews used a subset of questions from the semi- structured questionnaire used during the face-to-face interviews. Appendix C lists the questions used in these interviews. INTERVIEWS WITH IMPLEMENTERS AND OTHER STAKEHOLDERS IEG interviewed the former PIU staff and other project stakeholders from the government, academia, and civil society during the mission. These interviews were conducted using open questions, which are listed in appendix D. Appendix E provides information on the people interviewed. Field Assessment Limitation IEG was unable to visit an equal distribution of subproject investment sites across the three project counties because of a lack of response or the availability of beneficiaries and because of the mission’s time constraints. Therefore, one county is overrepresented in the IEG sample (Osiječko-Baranjska). However, this is not a major concern for the assessment results because overall the mission visited a large portion of APCP subprojects (12 out of 48 manure storage investments, equivalent to 25 percent). Furthermore, Croatia’s eastern counties’ geographic proximity, similarity in agricultural production, and the consistency of beneficiary responses across all counties are not expected to lead to major biases in the assessment results. 34 Appendix C. Interviews with Direct Beneficiaries During the IEG field mission’s site visits for the Agricultural Pollution Control Project (APCP), 12 of the 48 direct beneficiaries of manure storage investments were interviewed face-to-face using a semi-structured survey. During and after the mission, IEG interviewed 32 of the remaining 36 direct beneficiaries individually by phone. In the following list of questions, the subset of questions used for the phone interviews are underlined. IEG described its role to all interviewees and told them that this work related to an objectives-based evaluation of the APCP. IEG described the evaluation approach and told participants that it was also interviewing stakeholders at multiple levels, and that information was therefore being triangulated. Interviewees were also told that nothing they said would be directly attributed to them (anonymity will be maintained) and told them how to access the report once it is finalized. List of Questions for Face-to-Face Interviews GENERAL QUESTIONS 1. What is your main production? 2. How did you learn about the APCP? 3. Why did you decide to participate in the APCP? 4. Looking back, what was the main benefit of the APCP to you? 5. Looking back, what were the main challenges for you in participating in the APCP? QUESTIONS FOR THE BENEFICIARIES OF MANURE STORAGES (AND EQUIPMENT, WHERE APPLICABLE) 6. In which year was the manure storage built? 7. Is the manure storage still functioning/in use? 8. Are you satisfied with it? 9. Have you experienced any major problems with it? If yes, which ones? 10. Did you also obtain equipment from the APCP? If yes, is the equipment still functioning/in use? 11. Did you know about the benefits of manure storage/spreading before the APCP? 12. Did you have a manure storage before the APCP? If yes, what kind of manure storage did you have before the APCP? If no, would you have invested in a manure storage without the APCP? 13. What is the most important impact from the manure storage (and equipment) for your production? 35 14. Since the APCP ended, have you reduced the application of mineral fertilizer per ha [hectare] (for the same crop)? If yes, what is your estimation by how much you have reduced mineral fertilizer? 15. Have you taken any soil or water samples on your farm since the APCP ended? If yes, what are the results compared to before the APCP (if available)? 16. Do you know of nearby farmers who also invested in manure storages since 2012 (not with help from APCP)? If yes, do you know why they decided to do so? If no, do you know why neighboring farmers are not investing in manure storage? QUESTIONS FOR BENEFICIARIES WHO ALSO TOOK PART IN GOOD AGRICULTURAL PRACTICES DEMONSTRATION SITES/FIELD TRIALS 17. Did you participate in the APCP Good Agricultural Practices field trials? If yes, in what kind of field trial did you participate with the APCP? 18. In which year(s) did this take place? 19. Had you applied practices taught in the field trial before the APCP? 20. What was the most important impact this field trial had on your production? 21. Do you still apply what you learned in the field trial? If no, why not? 22. Since the APCP ended, have you reduced the application of mineral fertilizer per ha [hectare] (for the same crop)? If yes, what is your estimation by how much you have reduced mineral fertilizer? 23. Do you know of nearby farmers who also started applying similar practices since 2012 (not with help from APCP)? If yes, do you know why they decided to do so? If no, do you know why neighboring farmers are not investing in manure storage? QUESTIONS ON PIEZOMETERS 24. Was a piezometer installed on your farm by the APCP? If yes, how many? 25. Has somebody (for example from Croatian Waters) come by to check it/take samples? 26. Are you being informed by Croatian Waters or another institution about the local water quality? 36 Appendix D. Stakeholder Interviews This appendix lists the guidance questions IEG used for the open-ended interviews with former project implementation unit staff for the Agricultural Pollution Control Project (APCP) and other project stakeholders from the government, academia, and civil society. IEG asked additional questions depending on the interviewee and response context. Appendix E lists the persons interviewed for this PPAR. IEG described its role to all interviewees and told them that this work related to an objectives- based evaluation of the APCP. IEG described the evaluation approach and told participants that it was also interviewing stakeholders at multiple levels, and that information was therefore being triangulated. Interviewees were also told that nothing they said would be directly attributed to them (anonymity will be maintained) and told them how to access the report once it is finalized. List of Guidance Questions 1. What was your role/involvement in the APCP? 2. Do you think the APCP achieved its set objectives? If yes, how so? If no, what was not achieved? 3. In your opinion, what were the main achievements of the APCP? How are these achievements evidenced? 4. In your opinion, what were the main challenges of the APCP? How were they dealt with? 5. What could be highlighted as success factors of the APCP? 6. In retrospect, do you think anything should/could have been done differently? 7. How do you see the post-APCP/current situation regarding Croatia’s agricultural pollution? Are activities or lessons from the APCP repeated/picked up in current interventions? 8. To your knowledge, what is the current status of (inland) water quality in Croatia? 37 Appendix E. List of Persons Consulted Name Title Institution World Bank Group and Global Environment Facility Carlos Pinerua Country Manager Croatia and Slovenia World Bank Zagreb Office Vera Dugandzic Task Team Leader for APCP World Bank Zagreb Office Stjepan Gabric Sr. Water and Sanitation Specialist World Bank Zagreb Office Daniel Gerber Sr. Agricultural Specialist, APCP ICR author World Bank Group Christian Holde Focal Area Coordinator for International Global Environment Facility Severin Waters Steffen Cole Environmental Specialist Global Environment Facility Brandstrup Hansen Government of Croatia Dinko Polic Deputy General Manager Croatian Waters Fani Bojanic Deputy General Manager Croatian Waters Head of Sector for Water Pollution Croatian Waters Narcizo Dalsaso Prevention in Coastal Areas Sanja Barbalic Head of Water Management Institute Croatian Waters Head of Department for International Jadranka Hajdinjak Ministry of Finance Financial Cooperation Karmen Cerar Head of Sector for Water Management Ministry of Environment Mario Obrdalj Chief Engineer Croatian Waters Ministry of Agriculture, Forestry, Mario Njavro Assistant Minister and Water Management Ministry of Agriculture, Forestry, Krunoslav Karalic Assistant Minister and Water Management Head of Sector, Directorate EU Funds for Ministry of Agriculture, Forestry, Anita Sever-Koren Rural Development and Water Management Ministry of Agriculture, Forestry, Sanja Krnic Bastac Head of Section Assistant for Plant Nutrition and Water Management Ministry of Agriculture, Forestry, Miljenko Rakic Head of Agricultural Production Sector and Water Management Biljana Pozgaj Head, Section for State Aid Agriculture, Ministry of Agriculture, Forestry, Rubinic Former CAAP Project Coordinator and Water Management Senior Adviser, Former Deputy Minister of Miroslav Bozic Consultancy Viro Agriculture 38 Name Title Institution Croatian Agricultural Extension Hrvoje Horvat Director, Former APCP Project Coordinator Service Institute Payment Agency for Rural Bonaventure Nicolic Former APCP Project Coordination Unit Development Slaven Aljinovic Former APCP Project Coordination Unit Free consultant Head, Department of Agriculture and Rural Ministry of Agriculture, Andrija Matić Development Vukovarsko-Srijemska County Head, Department of Agriculture and Ministry of Agriculture, Dragutin Vincek Environmental Protection Varaždinska County Academia, Nongovernmental Organizations, River Basin Commissions, Other Donors Milan Poljak Professor, Department of Plant Nutrition University of Zagreb Assistant Professor, Department of Soil Monika Zovko University of Zagreb Amelioration Marina Bubalo Assistant, Department of Soil Amelioration University of Zagreb Lana Matijevic Assistant, Department of Soil Amelioration University of Zagreb Vladimir Professor, Department of Plant Nutrition University of Osijek, Vukadinovic Danijel Jug Professor, Department for Plant Production University of Osijek Professor, Department of Soil Chemistry, Irena Jug University of Osijek Biology and Physics Vesna Vukadinovic Professor, Department of Plant Nutrition University of Osijek Sonja Karoglan Director NGO Ecological Todorovic International Commission for the Ivan Zavadsky Executive Secretary Protection of the Danube River International Commission for the Adam Kovacs Technical Expert on Pollution Control Protection of the Danube River International Sava River Basin Dejan Komatina Secretary Commission, United Nations Industrial Stephen Sicars Director, Department of Environment Development Organization Guillermo Catella Unit Chief, Emerging Compliance Regimes United Nations Industrial Lorenzo Unit Development Organization United Nations Industrial Klaus Tyrkko Unit Chief, Stockholm Convention Unit Development Organization Industrial Development Officer, Industrial United Nations Industrial Carolina Gonzalez Resource Efficiency Division Development Organization 39 Appendix F. Asset Verification IEG Site Visits County and Village: Osiječko-Baranjska, Selci Đakovački Date visited: November 2, 2016 Livestock: Cows Asset type: Manure storage and pit; manure scraper Year built or purchased: 2011 Functionality of asset(s): Functioning and in use Main benefits according to beneficiary: Improved handling of manure; more even manure spreading across field; better soil structure Piezometer installed: Yes (n=1) Piezometer checked: Never County-Village: Osiječko-Baranjska: Satnica Đakovačka Date visited: November 2, 2016 Livestock: Cows Asset type: Manure storage and pit; manure spreader (verbally confirmed) Year built/purchased: 2010 40 Functionality of asset(s): Functioning and in use Main benefits according to beneficiary: Improved handling of manure; more even manure spreading across field; increased yields Piezometer installed: Yes (n=3) Piezometer checked: Never County-Village: Osiječko-Baranjska: Budimci Date visited: November 2, 2016 Livestock: Cows Asset type: Manure storage and pit; manure spreader Year built/purchased: 2011 Functionality of asset(s): Functioning and in use Main benefits according to beneficiary: More even manure spreading across field; improved handling of manure Piezometer installed: Yes (n=1) Piezometer checked: Never 41 County-Village: Osiječko-Baranjska: Antunovac Date visited: November 3, 2016 Livestock: Pigs Asset type: Manure storage and pit; manure spreader Year built/purchased: 2012 Functionality of asset(s): Functioning and in use Main benefits according to beneficiary: Increased capacity of storage; improved handling of manure Piezometer installed: No Piezometer checked: n.a. 42 County-Village: Osiječko-Baranjska: Črnkovci Date visited: November 3, 2016 Livestock: Pigs Asset type: Manure storage and pit; manure spreader (verbally confirmed) Year built/purchased: 2011 Functionality of asset(s): Storage not functioning (collapsed during construction); spreader functioning and in use Main benefits according to beneficiary: Improved handling of manure with equipment; more even manure spreading across field Piezometer installed: No Piezometer checked: n.a. County-Village: Osiječko-Baranjska: Podgajci Date visited: November 3, 2016 Livestock: Pigs Asset type: Manure storage and pit; manure spreader Year built/purchased: 2010 Functionality of asset(s): Functioning and in use 43 Main benefits according to beneficiary: Improved soil structure; improved handling of manure Piezometer installed: Yes (n=3), verbally confirmed Piezometer checked: Never County-Village: Vukovarsko-Srijemska: Berak Date visited: November 4, 2016 Livestock: Cows Asset type: Manure storage and pit Year built/purchased: 2012 Functionality of asset(s): Functioning and in use Main benefits according to beneficiary: Improved handling of manure Piezometer installed: Yes (n=2), verbally confirmed Piezometer checked: Never County-Village: Vukovarsko-Srijemska: Štitar Date visited: November 4, 2016 Livestock: Cows Asset type: Manure storage and pit; manure spreader Year built/purchased: 2010 Functionality of asset(s): Functioning and in use 44 Main benefits according to beneficiary: Improved handling of manure; increased capacity of storage Piezometer installed: Yes (n=3), 1 photo and 2 verbally confirmed Piezometer checked: Never County-Village: Vukovarsko-Srijemska: Babi Greda Date visited: November 4, 2016 Livestock: Cows Asset type: Manure storage and pit; manure spreader (verbally confirmed) Year built/purchased: 2011 Functionality of asset(s): Functioning and in use Main benefits according to beneficiary: Increased capacity of storage; more even manure spreading across field Piezometer installed: No Piezometer checked: n.a. 45 County-Village: Varaždinska: Novi Marof Date visited: November 8, 2016 Livestock: Cows Asset type: Manure storage and pit; manure spreader (verbally confirmed) Year built/purchased: 2012 Functionality of asset(s): Functioning and in use Main benefits according to beneficiary: More even manure spreading across field; increased capacity of storage Piezometer installed: Yes (n=3), 1 photo and 2 verbally confirmed Piezometer checked: Never County-Village: Varaždinska: Petrijanec Date visited: November 8, 2016 Livestock: Poultry Asset type: Manure storage and pit; manure spreader (verbally confirmed) Year built/purchased: 2011 Functionality of asset(s): Functioning and in use 46 Main benefits according to beneficiary: Improved handling of manure; increased capacity of storage Piezometer installed: Yes (n=2), 1 photo and 1 verbally confirmed Piezometer checked: Never County-Village: Varaždinska: Druzbinec Date visited: November 8, 2016 Livestock: Poultry Asset type: Manure storage and pit Year built/purchased: 2011 Functionality of asset(s): Functioning and in use Main benefits according to beneficiary: Improved handling of manure Piezometer installed: No Piezometer checked: n.a. Laboratory equipment County-Village: Osiječko-Baranjska Date visited: November 2, 2016 Asset type: Laboratory equipment, University of Osijeck Year built/purchased: 2011 Functionality of asset(s): Functioning and in use 47 48 Appendix G. Example of Green Manure Figure G.1 show an example of a field with the traditional postharvest practice of leaving the soil bare (right), which has a high risk of losing nitrogen. Green manure practices are applied on the other field (left), which more effectively fix nitrogen in the soil and lead to improved soil fertility without mineral fertilization. The photo was taken during an IEG site visit in Osiječko-Baranjska on November 3, 2016. Figure G.1. Fields with Different Postharvest Practices 49 Appendix H. APCP Brochures Figure H.1 shows a display stand with a sample of Agricultural Pollution Control Project brochures at the regional office of the Ministry of Agriculture in Varaždinska County. The photo was taken during an IEG site visit on November 7, 2016. Figure H.1. APCP Brochure Display in Varaždinska County Other guidance materials that the project produced for farmers can be downloaded from the project website at http://www.apcp.hr/dokumentacija.asp?pageID=15&lang=en. 50 Appendix I. Summary of Studies Commissioned by APCP The Agricultural Pollution Control Project (APCP) commissioned a Czech consultancy Ekotoxa to undertake a study titled Designation of Nitrates Vulnerable Zones (NVZs) and Economic Impact of Nitrates Directive. The assignment had four objectives: 1. Identify polluted and threatened waters and to designate NVZs in Croatia 2. Assess the economic impact of the Nitrate Directive (ND) implementation in Croatia 3. Develop recommendations and guidelines for implementing the Action Program in Croatia 4. Develop conclusions and recommendations relating to the ND implementation in Croatia. The study produced two reports: One on the designation of NVZs (Hrabánková et al. 2012) and another report on the economic impact of the implementation of the Nitrates Directive (Trantinová 2012). Designation of Nitrate Vulnerable Zones The procedure proposed for designating vulnerable zones (NVZ) was compliant with the EU Nitrates Directive requirements. The designation was based separately for nitrates in surface waters and in groundwater and for the eutrophication in surface waters (rivers, lakes, transitional, and coastal waters). Water quality monitoring results and land use (including natural vulnerability, N-pressure estimation, hydrogeological structure, intensity, and farming methods) were the main criteria for designation of NVZs. In areas with no clear pollution trends or with incomplete monitoring data, vulnerable zones were designated as preliminary according to natural vulnerability and land use. Vulnerable zones were first designated in natural borders-hydrological catchments and later transformed into administrative borders at the municipality level. At the beginning of the study, the authors decided to use a two-track approach to determine the designation of vulnerable areas: the whole territory of Croatia as one vulnerable area, and individual vulnerable areas in administrative borders. Based on the data on water pollution (notably eutrophication of surface water), natural vulnerability (more than 50 percent of the country is porous karst area prone to nitrogen leaching), and N-pressure estimation, the study team concluded that the entire country should be designating as an NVZ. However, because the proposal to designate the entire territory as a NVZ tended to raise resistance in some circles, an alternative option was also proposed. The analysis revealed strong evidence of water pollution by nitrogen and phosphorous from agricultural origin in 9 percent of the territory. Somewhat weaker evidence (mainly due to the lack of data) was found in another 44 percent of the territory, which is considered prone to pollution and termed “preliminary vulnerable zones.” Therefore, the study proposed that if a softer approach was used (not designating the entire territory as NVZ), 53 percent of the territory should be designated as an NVZ. This area includes 483 catchments: 15 hydrological catchments in vulnerable zones (9 percent of the territory with strong evidence of pollution) and 196 hydrological catchments (44 percent of the territory with weaker pollution evidence) in preliminary zones. The area covers 310 municipalities (of 552 in Croatia) and nearly 30,000 square kilometers. 51 ECONOMIC IMPACT OF THE IMPLEMENTATION OF NITRATES DIRECTIVE The cost-benefit analysis of the implementation of the Nitrates Directive in Croatia was based on the assessment of three scenarios:  The zero variant (V0) represents the scenario without major changes in agricultural practice, no investments in manure storage, no changes in the quantity of nitrates and phosphorous applied, livestock density, and so on. The provision of subsidies is subject to compliance with environmental standards. It is based on high-input agriculture leading to water pollution. The assumption is that this scenario will increase nitrates in drinking water and deteriorate human health. It also envisions an increased expenditure for buying bottled water.  The first investment scenario (V1) is based on designation of 53 percent pf the territory as an NVZ.  The second investment scenario (V2) is based on designation of the whole country as an NVZ. To comply with the EU Nitrates Directive, it is estimated that Croatia will need to invest €125 million in scenario V1 and €149 million in scenario V2. The expected annual savings on fertilizers in the scenario V2 is €11 million for nitrogen fertilizers, €12 million for phosphorous fertilizers, and €20 million for potassium fertilizers. The total cost for construction of manure storage facilities (excluding farms with less than 1 livestock unit that are expected to be exempted from Nitrates Directive requirements) is €487 million in scenario V2 and €250 million in scenario V1. Total annual operating costs were estimated at €2.9–3.8 million. Scenarios V1 and V2 were found to have a positive economic return and as such regarded as economically feasible. The zero variant V0 (no acceptance of Nitrates Directive) was found to be economically (and politically) unacceptable. Agri-Environmental Program Implementation Assessment The APCP hired WPA Beratende Ingenieure, an Austrian consultancy, to prepare an assessment of the implementation of the Croatian agri-environmental program (Kuderna et al. 2012). This study outlined an agri-environmental program for protecting water resources from nutrient loads deriving from diffuse agricultural sources. More specifically, the program does the following:  Provides an overview on agri-environmental programs across Europe  Discusses criteria for the design and selection of appropriate measures for water protection in Croatia  Proposes and describes in detail (a full technical description) a set of measures aimed at protecting Croatian water resources from adverse agricultural practices.  Assesses the mitigation potential of the proposed measures  Provides calculations for agri-environment payments (subsidies) according to EU requirements  Provides the cost efficiency of the proposed measures based on their mitigation potential and subsidies. Implementation of the following measures was proposed: 52  Organic farming  Reduced fertilization  Catch crops on arable land  Erosion control on permanent crops  Promotion, advisory, and training. Implementation of these measures is expected to reduce the nitrogen load into groundwater and surface water by 50–90 percent. Organic farming is calculated to require a subsidy of €218 per hectare of arable land, reduced fertilization €95 per hectare of arable land, catch crops 100–250 per hectare of arable land (depending of the type of catch crop), and erosion control €122 per hectare of arable land. The annual cost for implementing these measures is estimated at €80 million (including €1 million for extension and training). The estimate assumes the following uptake rates:  Organic farming: 5 percent of arable land  Reduced fertilization: 50 percent of arable land  Catch crops: 15 percent of arable land  Erosion control: 30 percent of permanently cropped area. Cost efficiency was calculated for mitigating effects regarding groundwater and surface water by dividing the payments by the expected mitigating effect. Therefore, lower values indicate a higher cost efficiency. Cost efficiency for measures related to groundwater protection ranges from 1.9 to 3.3, and reduced fertilization has the highest cost efficiency and catch cropping the lowest. Organic farming is within that range. Regarding surface waters, erosion control has a cost efficiency of 1.3 and catch crops 3.3. Catch crops are less cost-efficient for just a single protection goal. However, they are multipurpose, also protecting groundwater. If their mitigation effects are combined, the added cost efficiency is 1.7, which is better than most of the other measures. Field Trial: Faculty of Agronomy of the University of Zagreb APCP financed a field trial conducted by the University of Zagreb’s Faculty of Agronomy. The objective of the trial was to assess the concentration of nutrients (nitrate and phosphorus) in groundwater in vegetable crops fertilized with different level of nitrogen. The trial was conducted in three locations in Varaždin County: Hrastovsko, Greda, and Donja Voća. All three locations are in regions of intensive vegetable production with high nutrient input, partly because of the proximity of numerous cattle and poultry farms. All three sites have alluvial soil with shallow, active profile and mainly on gravel base, which enhances the risk of nitrogen loss in surface and groundwater. In April 2011, three zero-tension pan lysimeters were installed at each of the three sites (nine piezometers total). Each lysimeter was installed in an open soil profile at a depth above less- permeable horizon or at the solum-gravel contact. Polyvinyl chloride–reinforced hose was connected to the lysimeter drains and set with sufficient slope to ensure percolate flow. Percolate water was collected in a plastic container. 53 In May 2011, the Hrastovsko site was sown with silage maize, the Greda site with pepper, and the Donja Voća site with grain maize. Silage maize at the Hrastovsko site received 153 kilograms of nitrogen from mineral fertilizers and livestock manure, and grain maize at the Donja Voća site received 161 kilograms of nitrogen from mineral fertilizers and livestock manure. Pepper at the Greda site did not receive any nitrogen or phosphorous input from mineral fertilizers or livestock manure. Table I.1 outlines the experiment’s main features. Nitrogen and phosphorous content in the percolate water varied substantially between sites. The mean nitrate content in percolate water found at the Greda site exceeded the maximum admissible concentration by 4.5 times and at and the Donja Voća site by four times. At the Hrastovko site, the mean nitrate content was just at the threshold. The mean nitrogen leached ranged from 1.6 kilograms of nitrogen per hectare at the Hrastovsko site to 7.5 kilograms of nitrogen per hectare at the Donja Voća site. The mean phosphorous content ranged from 0.23 milligrams of phosphorous per liter at the Donja Voća site to 2.9 milligrams of phosphorous per liter at the Greda site. The mean phosphorous leached ranged from zero to 0.25 kilograms of phosphorous per hectare. The study suggests that the results regarding water pollution by nitrogen and phosphorous are inconclusive, notably because of the exceptionally dry growing season. During May 2011 to January 2012, the total rainfall at all three locations was approximately 50 percent lower than the average monthly rainfall value of five-year series, which prevented water percolation to lysimeters. Finally, the study concludes that this research should be continued to obtain more reliable and conclusive results. Field Trial: Faculty of Agriculture of the University of Osijek APCP financed a field trial conducted by the University of Osijek’s Faculty of Agriculture. The fertilizing field trials titled Optimization of Crop Fertilization by Nitrogen were conducted at two sites in two APCP regions: in Šljivoševci (Osijek-Baranja County) and Ćelije (Vukovar-Sirmium County). The trial objectives were to:  Determine the concentration of nitrates in the form of nitrogen in groundwater and soil  Identify the level agricultural yield  Determine uptake and removal of major biogenic elements  Determine the optimal fertilization rate regarding yield amount and quality. 54 Table I.1. Main Features of the Experiment Hrastovsko site Greda site Donja Voća site Crop Silage maize Pepper Grain maize Soil tillage depth 30 cm 30 cm 30 cm Sowing date May 2011 May 2011 May 2011 Livestock manure 105 kg N/ha (before None 123 kg N/ha (before sowing) sowing) Mineral fertilizers 48 kg N (March 2011) None 38 kg N/ha (at sowing) Total nitrogen applied 153 kg N/ha None 161 kg N/ha Total phosphorous applied Not specified Not specified Not specified Harvest date Late August 2011 Not specified Not specified Irrigation No Yes Not specified, probably not Postharvest fertilization 10 tn manure (late Sep Not specified Not specified 2011) Number of lysimeters 3 3 3 Lysimeters depth 90 cm (all three) 50, 70, and 85 cm Not explicitly specified, but the graphical figure indicates approximately 55, 65, and 86 cm Sampling period Jan 2012 Sep 2011 (2 times) Oct 2001 Jan 2012 NO3-N (range) 1.9–16.2 mg N/L 14.1–108.0 mg N/L 18.6–58.2 mg N/L NO3-N (mean) 9.72 mg N/L 45.3 mg N/L 39.7 mg N/L Nitrogen leached (range) 1.1-3.1 kg/ha 0.03-20.6 kg/ha 0.1-24.2 kg/ha Nitrogen leached (mean) 1.6 kg N/ha 4.2 kg N/ha 7.5 kg N/ha Phosphorous (range) 0.2–0.4 mg P/L 0.09–7.2 mg P/L 0.0–0.8 mg P/L Phosphorous (mean) 0.3 mg P/L 2.9 mg P/L 0.23 mg P/L Phosphorous leached 0.02–0.08 kg P/ha 0.0–0.21 kg P/ha 0.0–0.25 kg P/ha (range) Phosphorous leached 0.06 kg P/ha 0.06 kg P/ha 0.03 kg P/ha (mean) Source: University of Zagreb (2014). Note: cm = centimeter; ha = hectare; kg = kilogram; L = liter; mg = milligram; N = nitrogen; P = phosphorous; tn = ton. At both and Šljivoševci and Ćelije sites, piezometers were placed at 250 centimeters depth to monitor the groundwater level and to determine the nitrogen concentration (nitrates and ammonium) in groundwater. The experiment was set up as a randomized block, with four repetitions with identical fertilization plans for maize and winter wheat and different doses of nitrogen for soybeans. Table I.2 contains the nitrogen, phosphorous, and potassium (NPK) fertilization rates that were applied. 55 Table I.2. Fertilization Rates Applied in Field Trial Maize and winter wheat Soybean 1. Control (no fertilizer and crop residues) 1. Control (no fertilizer and crop residues) 2. N30P100K100 2. N30P100K100 3. N50P100K100 3. N50P100K100 4. N70P100K100 4. N70P100K100 5 . N90P100K100 5. N90P100K100 6. N110P100K100 6. N110P100K100 7. N130P100K100 8. N150P100K100 9. N170P100K100 10. Manure 170 kg N/ha Source: University of Osijek (unpublished). Note: Soybean was inoculated by Rhizobium bacteria before sowing, and fertilization was identical as in the first six fertilization treatments for wheat and maize. h = hectare; K = potassium; kg = kilogram; N = nitrogen; P = phosphorous. Winter wheat was sown in October 2010, and soybean and maize were sown in April 2011. The concentration of nitrates in November and December 2010 ranged from 16–30 milligrams of nitrate per liter. These are well below the admissible maximum concentration of 50 milligrams of nitrate per liter and mainly lower than the recommended concentration value of 25 milligrams of nitrate per liter. During January to June 2011, the concentration of nitrate in water samples ranged from 17.5–21 milligrams of nitrate per liter in Šljivoševci, and in Ćelije from 29–34 milligrams of nitrate per liter, which is also below the admissible maximum concentration of 50 milligrams of nitrates per liter. Wheat achieved the highest yields, the best plant density, and the maximum crop residue mass at the plot that received 170 kilograms of nitrogen from mineral fertilizers (N170P100K100). The lowest yields, the least density, and the lowest mass of wheat crop residues were measured on the control plot and the plot treated with manure. The highest soybean yield was 3.7 ton per hectare in the plot treated with 50 kilograms of nitrogen (N50P100K100), and the lowest yield of 2 tons per hectare was recorded at the control plot. The highest maize yield was 9.57 tons per hectare at the plot that received 170 kilograms of nitrogen from fertilizers (N170P100K100). The lowest yield of 5.41 tons per hectare was achieved at the control plot with no fertilization. The study concludes that the amount of nutrients in the soil, especially nitrogen, determines the yield level and that fertilizers must be applied in accordance with biological, economic, and environmental conditions. It also concludes that to increase the reliability of the obtained results and conclusions derived from it, it is necessary to conduct this experiment for several years and in different environmental conditions. 56 Appendix J. Croatia River Basin Management Plan 2016–21 The most recent key document on water quality in Croatia is the River Basin Management Plan 2016–21 prepared by Croatian Waters (CW) in 2016.1 The plan provides the most comprehensive, integrated, and updated overview of water quality in Croatia. According to this document, unsatisfactory “ecological condition” was found in the following:  58 percent water bodies, comprising 66 percent of the total length of water streams with surface greater than 10 square kilometers  54 percent lakes (20 total)  55 percent surface of the transitional waters  12 percent coastal waters. Unsatisfactory “chemical condition” of the surface waters was found in the following:  8 percent water bodies, comprising nearly 10 percent of the total length of water streams with surface greater than 10 square kilometers  15 percent surface of the transitional waters  6 percent coastal waters. In general, both the ecological and chemical condition of surface water is much more favorable in the Adriatic River Basin area than in the Danube River Basin area. The assessment of chemical status of surface water is based on total nitrogen and phosphorus content. The assessment found that 37 percent of rivers do not have a satisfactory status on nitrogen content and 41 percent on phosphorous content. Agriculture is a significant source of both of these pollutants, but the plan does not conduct an apportionment assessment on agriculture. Contrary to the data for surface water that are considered as reliable, the reliability of the assessment of groundwater quality is considered relatively low because the water quality monitoring program is modest. However, the existing data on groundwater quality show a much more favorable situation than surface water. In the Danube River Basin, all groundwater is in good condition concerning the quantity of water. Poor chemical status of groundwater bodies was found only in the Varaždin region (high nitrates) and some groundwater in the Zagreb region (presence of tri chloroethene and tetrachloroethene). In the Adriatic River Basin district, the poor chemical status of the groundwater was found only in South Istria (nitrates), Bokanjac- Poličnik (intrusion of saltwater), and Bokanjac-Poličnik (excessive use). The Water Management Plan 2016–21 concludes on page 258 that comparative figures for the period 2009–12 show no improvement in water quality, mainly because of the modest progress in the implementation of concrete protection measures. The comparison between the 2009 figures from the River Basin Management Plan and the 2012 figures from the River Basin Management Plan 2016–21 on nitrogen and phosphorous content in rivers suggests that the situation concerning the chemical status of rivers has worsened (table J.1).2 In the period 2009– 12, the number of river body streams with unsatisfactory conditions for nitrogen rose by nearly 70 percent and for phosphorous by nearly 20 percent. 57 Table J.1. Nitrates and Phosphorous Content in Rivers in 2009 and 2012 Unsatisfactory condition Parameter 2009 2012 Nitrates (total) 22 percent of river body streams 37 percent of river body streams Phosphorous (total) 35 percent of river body streams 41 percent of river body streams Source: CW (2013). The River Basin Management Plan 2016–21 suggests that currently there are some shortcomings regarding water quality monitoring. Therefore, the 2012 results on water quality should be viewed with caution because of the following shortcomings:  Both surface and groundwater data in 2012 were based on insufficient density of sampling sites.3  The time span of groundwater monitoring data (2007–12) is too short to determine valid trends.  The frequency of water sampling and testing was not always consistent – at many sampling sites, data are based on less than 12 samples per year (using 50 percent instead of 90 percent percentile as in other EU member states).  There were no measurements of groundwater quality at different depths. For example, phreatic groundwater: 0–5 meters (shallow), 5–15 meters (deep), 15–30 meters (deep), greater than 30 meters (deep); captive groundwater, and so on. This is particularly important considering Croatia’s high consumption of nitrogen and phosphorous per hectare (among the highest in Europe). Groundwater quality is currently measured at quite deep points, so the water quality is high. However, it is likely that nitrates are abundantly present in the upper layers and might gradually move downward and reach deeper groundwater. Water soaks slowly through permeable layers of soil and rock, taking decades to reach the water table below, but still carrying nitrates.  Data on surface water do not reflect seasonal differences. For example, there are no summer means versus annual mean values (important eutrophication parameters).  Water quality monitoring stations for surface water were located along large rivers instead of smaller streams where agricultural pollution is likely to occur more. 1 The River Basin Management Plan 2016–21 is available at voda.hr/sites/default/files/plan_upravljanja_vodnim_podrucjima_2016._-_2021.pdf. 2 The plan, published by Croatian Waters (CW) in 2013, is available at www.voda.hr/sites/default/files/dokumenti/plan.pdf 3 A study commissioned by CW and prepared by the University of Zagreb’s Faculty of Agronomy also reached this conclusion in December 2014. According to this study, an additional 145 stations are required for groundwater monitoring and 39 for surface water monitoring. However, it remains unclear if this conclusion is still valid. The number of surface water monitoring stations expanded from 321 in 2011 to 530 in 2015, while monitoring of groundwater quality expanded from 256 to 337 measurement stations. CW claims that since 2015 the water quality monitoring program fully complies with the requirements of the EU Water Framework Directive and the Marine Strategy Framework Directive.