Document of The World Bank Report No: ICR2222 IMPLEMENTATION COMPLETION AND RESULTS REPORT (IBRD-48230) ON A LOAN IN THE AMOUNT OF EURO 10.0 MILLION (US$ 13.725 MILLION EQUIVALENT) TO THE FORMER YUGOSLAV REPUBLIC OF MACEDONIA FOR A LEGAL AND JUDICIAL IMPLEMENTATION AND INSTITUTIONAL SUPPORT PROJECT March 28, 2013 Poverty Reduction and Economic Management Unit South East Europe Country Unit (ECCU4) Europe and Central Asia Region CURRENCY EQUIVALENTS (Exchange Rate Effective June 30, 2012) Currency Unit = FYR Macedonian Denars (MKD) 1.00 = US$ 0.02 US$ 1.00 = MKD 49.30 FISCAL YEAR January 1 – December 31 ABBREVIATIONS AND ACRONYMS ABA-CEELI American Bar Association/Central and Eastern European Law Initiative ABA-ROLI American Bar Association/Rule of Law Initiative ACCMIS Automated Court Case Management Information System BEEPS Business Environment and Enterprise Performance Survey CAS Country Assistance Strategy CBC Court Budget Council CPS Country Partnership Strategy DPL Development Policy Loan EBRD European Bank for Reconstruction and Development EMP Environmental Management Plan EU European Union FYR Former Yugoslav Republic GoMK Government of the Former Yugoslav Republic of Macedonia ICR Implementation and Completion Report ICT Information and Communications Technology ISR Implementation Status Report LJIIS Legal and Judicial Implementation and Institutional Support Project MOJ Ministry of Justice NGO Non-Governmental Organization PAD Project Appraisal Document PDO Project Development Objective PDPL Programmatic Development Policy Loan PIO Project Implementation Office QER Quality Enhancement Review ROSC Reports on Standards and Codes SAI State Administrative Inspectorate SFRY Socialist Federal Republic of Yugoslavia TA Technical Assistance USAID United States Agency for International Development VPN Virtual Private Network Vice President: Philippe H. LeHouérou Country Director: Ellen Goldstein Sector Manager: Roumeen Islam, Acting Project Team Leader: Klaus Decker ICR Team Leader: Beth Anne Hoffman FORMER YUGOSLAV REPUBLIC OF MACEDONIA LEGAL AND JUDICIAL IMPLEMENTATION AND INSTITUTIONAL SUPPORT PROJECT CONTENTS Data Sheet A. Basic Information B. Key Dates C. Ratings Summary D. Sector and Theme Codes E. Bank Staff F. Results Framework Analysis G. Ratings of Project Performance in ISRs H. Restructuring I. Disbursement Graph 1. Project Context, Development Objectives and Design ............................................... 1 2. Key Factors Affecting Implementation and Outcomes .............................................. 6 3. Assessment of Outcomes .......................................................................................... 10 4. Assessment of Risk to Development Outcome......................................................... 15 5. Assessment of Bank and Borrower Performance ..................................................... 15 6. Lessons Learned ....................................................................................................... 19 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners .......... 20 Annex 1. Project Costs and Financing .......................................................................... 22 Annex 2. Outputs by Component ................................................................................. 23 Annex 3. Economic and Financial Analysis ................................................................. 28 Annex 4. Bank Lending and Implementation Support/Supervision Processes ............ 30 Annex 5. Beneficiary Survey Results ........................................................................... 31 Annex 6. Stakeholder Workshop Report and Results................................................... 32 Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR ..................... 33 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders ....................... 34 Annex 9. List of Supporting Documents ...................................................................... 35 A. Basic Information Legal & Judicial Macedonia, former Implementation & Country: Project Name: Yugoslav Republic of Institutional Support Project Project ID: P089859 L/C/TF Number(s): IBRD-48230 ICR Date: 03/28/2013 ICR Type: Core ICR MACEDONIA Lending Instrument: SIL Borrower: GOVERNMENT Original Total USD 12.40M Disbursed Amount: USD 13.04M Commitment: Revised Amount: USD 12.03M Environmental Category: B Implementing Agencies: Ministry of Justice - Implementation Office Cofinanciers and Other External Partners: B. Key Dates Revised / Actual Process Date Process Original Date Date(s) Concept Review: 11/28/2005 Effectiveness: 10/12/2006 10/12/2006 10/28/2010 03/01/2011 Appraisal: 05/03/2006 Restructuring(s): 10/17/2011 03/27/2012 Approval: 06/01/2006 Mid-term Review: 07/31/2009 09/09/2009 Closing: 07/31/2011 06/30/2012 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Moderately Satisfactory Risk to Development Outcome: Substantial Bank Performance: Moderately Unsatisfactory Borrower Performance: Moderately Satisfactory C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Moderately Quality at Entry: Government: Moderately Satisfactory Unsatisfactory Quality of Supervision: Moderately Implementing Moderately Satisfactory i Unsatisfactory Agency/Agencies: Overall Bank Moderately Overall Borrower Moderately Satisfactory Performance: Unsatisfactory Performance: C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments Indicators Rating Performance (if any) Potential Problem Project Quality at Entry Yes None at any time (Yes/No): (QEA): Problem Project at any Quality of No None time (Yes/No): Supervision (QSA): DO rating before Moderately Closing/Inactive status: Satisfactory D. Sector and Theme Codes Original Actual Sector Code (as % of total Bank financing) Central government administration 2 3 Law and justice 98 97 Theme Code (as % of total Bank financing) Access to law and justice 14 15 Judicial and other dispute resolution mechanisms 29 71 Law reform 14 Legal institutions for a market economy 29 12 Other rule of law 14 2 E. Bank Staff Positions At ICR At Approval Vice President: Philippe H. Le Houerou Shigeo Katsu Country Director: Ellen A. Goldstein Orsalia Kalantzopoulos Sector Manager: Roumeen Islam Cheryl W. Gray Project Team Leader: Klaus Decker David S. Bernstein ICR Team Leader: Beth Anne Hoffman ICR Primary Author: Beth Anne Hoffman ii F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) The objective of the LJIIS Project is to contribute to improving judicial efficiency and effectiveness and the business climate in FYR Macedonia by: (i) enhancing ministerial and judicial capaciy to systematically implement the Government's Judicial Reform Strategy and key laws; and (ii) improving judicial infrastructure. Revised Project Development Objectives (as approved by original approving authority) (a) PDO Indicator(s) Original Target Formally Actual Value Values (from Revised Achieved at Indicator Baseline Value approval Target Completion or documents) Values Target Years Share of firms that report functioning of the judiciary as a problem for doing Indicator 1 : business (BEEPS0. Value 43% (10% 51% from BEEPs quantitative or 53% from BEEPs 2005. decrease) 2008 Qualitative) Date achieved 01/01/2006 07/31/2011 06/05/2012 Comments BEEPs discontinued this question after 2008. A 2010 Life in Transition Survey (incl. % noted an 18% increase in satisfaction with service delivery in the civil courts. achievement) Share of respondents to court user survey indicating satisfaction with the Indicator 2 : operation of the courts 50.8% of the Value 45.7% of respondents 25% increase in respondents quantitative or "very" or "fairly" satisfied share of expressed their Qualitative) by results of court visit respondents satisfaction with the courts. Date achieved 11/02/2009 07/31/2011 06/30/2012 Comments At the end of the project, 50.8 % of respondents expressed their satisfaction with (incl. % the courts, measuring only a 5.1% increase in overall satisfaction. achievement) Indicator 3 : Number of days to enforce contracts Value 15% reduction 370 days (Doing 509 days (Doing Business quantitative or from 2006 Business data from data from 2005) Qualitative) baseline 2011/2) Date achieved 01/01/2006 07/31/2011 11/03/2011 Comments There was a 27.3% reduction in number of days to enforce contracts, as (incl. % measured by Doing Business in 2011. This number remained unchanged in achievement) 2012. Indicator 4 : Share of complaints against trustees resolved by Trustees Chamber Value 0% of complaints 80% of complaints 3 individual and 1 iii quantitative or resolved. Trustees resolved firm complaints Qualitative) Chamber established by fhave been resolved end 2007 within the legal deadline. Date achieved 01/01/2006 07/31/2011 11/03/2011 Comments (incl. % Target met in 2011 and sustained through the end of the project. achievement) Indicator 5 : Number of courthouse facilities improved. 10 courthouse renovations and Value furnishings 10 courthouses quantitative or 0 courthouses improved completed. The improved Qualitative) Skopje 1 Criminal Court is under construction. Date achieved 01/01/2006 07/31/2011 08/01/2011 Comments (incl. % 10 Courthouse renovations completed. New construction (Skopje 1) achievement) (b) Intermediate Outcome Indicator(s) Original Target Actual Value Formally Values (from Achieved at Indicator Baseline Value Revised approval Completion or Target Values documents) Target Years Share of inspectors and staff trained on new laws and operating procedures, Indicator 1 : including use of IT. Value 100% of 0% of inspectors and staff 100% of inspectors (quantitative inspectors and trained and staff trained. or Qualitative) staff trained Date achieved 01/01/2006 07/31/2011 03/04/2011 Comments (incl. % achievement) Indicator 2 : Share of licensed trustees trained on new Bankruptcy Law and regulations. Value 100% of trustees 0% of licensed trustees (quantitative trained prior to 100% trained. or Qualitative) receiving licenses. Date achieved 01/01/2006 07/31/2011 09/08/2011 Comments After initial training, Ministry of Economy and Trustee Chamber preparing (incl. % training for re-licensing exams as well as continuous training for Trustees. achievement) Number of basic courts with enhanced jurisdiction operating in renovated court Indicator 3 : buildings. Value At least 7 courts 11 basic courts with 0 (quantitative are renovated enhanced iv or Qualitative) jurisdiction are operating in renovated/newly furnished buildings. Date achieved 01/01/2006 07/31/2011 11/03/2011 Comments (incl. % 157% of the original target value is met. achievement) Indicator 4 : Number of courts increased in which Court Information System is in use. Value ACCMIS is (quantitative 0 At least 7 functional in all or Qualitative) courts Date achieved 01/01/2006 07/31/2011 09/08/2011 Comments The automated court system (ACCMIS) is operating in all courts in Macedonia. (incl. % The end of project target is achieved. achievement) G. Ratings of Project Performance in ISRs Actual Date ISR No. DO IP Disbursements Archived (USD millions) 1 06/25/2007 Satisfactory Satisfactory 1.31 2 05/22/2008 Moderately Satisfactory Moderately Satisfactory 1.70 3 02/20/2009 Moderately Satisfactory Moderately Satisfactory 2.69 4 12/30/2009 Moderately Satisfactory Moderately Satisfactory 4.03 5 07/14/2010 Moderately Satisfactory Moderately Satisfactory 5.20 6 03/22/2011 Satisfactory Moderately Satisfactory 8.06 7 10/02/2011 Satisfactory Moderately Satisfactory 9.56 8 11/22/2011 Satisfactory Moderately Satisfactory 10.35 9 06/24/2012 Moderately Satisfactory Moderately Satisfactory 11.26 H. Restructuring (if any) ISR Ratings at Amount Board Restructuring Disbursed at Restructuring Reason for Restructuring & Approved Restructuring Date(s) Key Changes Made PDO Change DO IP in USD millions Changes in Intermediate Indicators 1 due to change in indicator source; Change in 10/28/2010 MS MS 6.26 Intermediatie Indicator 2 Result due to changes in Skopje 1 Court Construction. Increase to disbursement percentage to v ISR Ratings at Amount Board Restructuring Disbursed at Restructuring Reason for Restructuring & Approved Restructuring Date(s) Key Changes Made PDO Change DO IP in USD millions 100%. Eight month extension to allow for completion of planned 03/01/2011 MS MS 8.06 activities, including Skopje 1 Court construction. Reallocation of proceeds to finance IT equipment for High Administrative Court and 10/17/2011 S MS 9.56 hardware for the State Administrative Inspectorate and Ministry of Justice. Extension to finish construction 03/27/2012 S MS 10.98 of the Skopje 1 Courthouse. I. Disbursement Profile vi 1. Project Context, Development Objectives and Design (this section is descriptive, taken from other documents, e.g., PAD/ISR, not evaluative) Nearly 15 years after gaining its independence from the former Socialist Federal Republic of Yugoslavia (SFRY), the Former Yugoslav Republic of Macedonia (FYR Macedonia) was granted candidate status for accession to the European Union (EU) in November 2005. At the time of its independence, FYR Macedonia was among the least developed republics of the former SFRY. In response to its EU candidate status, the country identified a number of reforms to improve the business climate. Key among these reforms was a strong emphasis on improving the country’s legal framework and building capacity to reach and enforce judicial decisions in a timely manner. Beginning in 2005, these reforms were ultimately supported by a series of Programmatic Development Policy Loans (PDPLs)1 from the World Bank, and later, this Project. In 2005, at the time the Project was designed, Macedonia’s judicial system was characterized as inefficient and opaque. The Project Appraisal Document (PAD) noted that confidence in the legal and judicial system’s ability to enforce property and contract rights had declined since 20022. The judiciary in particular was seen as one of the more serious obstacles to improving the investment climate. Furthermore, the justice system was characterized as unable to adequately protect property, contract, and creditor rights. The Macedonian Ministry of Justice (MoJ) completed a Strategy for the Reform of the Judiciary, Public Prosecution, Penitentiary Institutions System, Public Attorney, the Bar and the Notary in November 2004. The Strategy provided an analysis and framework for the development of the present Legal and Judicial Implementation and Institutional Strengthening Project. It presented a diagnosis of the system’s weaknesses that was then vetted by donors and the public. It also provided a roadmap for improvements and reforms in the judiciary and a number of related institutions, including the bar association, prosecutors, the corrections and penitentiary system, and notaries. The Strategy aimed to strengthen the judiciary’s independence and efficiency through: (i) substantive law reform addressing the functions of the judicial system; (ii) litigation law reform to optimize processes and procedures; and (iii) structural and organizational changes in the judiciary. These objectives of the proposed reforms were to establish a new legal framework in line with European and international standards, along with a new system of legally recognized and protected values based on human freedoms and rights and other principles of democratic society. During the second quarter of 2006, the Government of 1 Loan Number 4806-MK, Loan 7441-MK, and Loan 7533-MK in the amounts of US$30 million (equivalent), US$30 million (equivalent) and US$25 million (equivalent), respectively. 2 Former Yugoslav Republic of Macedonia, Project Appraisal Document: Legal and Judicial Implementation and Institutional Support Project. Report No. 35506-MK, May 10, 2006, pg. 8. 1 the FYR Macedonia (GoMK) passed a comprehensive package of new and revised legislation as the first stage of implementing the Strategy. 3 Building on the Strategy, the Project’s development objective, as set out in the PAD, was to contribute to improving judicial efficiency and effectiveness and the business climate in FYR Macedonia by: (i) enhancing ministerial and judicial capacity to systematically implement the Government’s Judicial Reform Strategy and key laws; and (ii) improving judicial infrastructure. 1.1 Context at Appraisal (brief summary of country and sector background, rationale for Bank assistance) In anticipation of the EU accession, GoMK had adopted a comprehensive road map for justice sector development. The Bank Project was founded on the information and approach provided in the Strategy. The Strategy mapped key reforms to be taken as well as donors’ on-going or planned work in the sector. The Legal and Judicial Implementation and Institutional Support (LJIIS) Project also benefitted from a Country Economic Memorandum, a Report on Standards and Codes (ROSC), a series of Development Policy Loans (DPL) supporting justice related aspects, and two background reports on the justice sector.4 At the time the Project was designed, the LJIIS Project was not specifically included in the Country Assistance Strategy (CAS) for the period 2004-6 (Report No. 26513). Rather, the CAS concentrated on the promotion of private sector development and job creation, as well as strengthening the protection of contract, creditor and property rights. The Project activities proposed in the PAD were selected so as to have synergy with other on- going Bank projects in the country, including land reform, activities to improve the business environment and the financial sector. Simultaneously, the Bank’s series of three DPLs supported a number of actions related to the judiciary and legal reform. Key actions under the DPL series included the adoption and implementation of a number of 3 Included in these reforms were the: Law on Civil Procedure (Law on Litigation), the Law on Enforcement, the General Administrative Procedure Law, the Labor Law, the Audit Law, the Law on Courts, the Law on Misdemeanors, the Bankruptcy Law and implementing regulations, and the Administrative Dispute Law. In addition, the Ministry of Justice adopted Court rules of procedure based on the opinion of the Supreme Court, which created specialized divisions in first instance and enhanced jurisdiction courts. With the passage of the Law on Misdemeanors, a number of laws needed to be harmonized – including the Law on Customs, the Law on Profit Tax, the Law on Personal Income Tax, the Law on Value Added Tax, the Law on Safety in Traffic and on Roads, and the Law on Public Order and Peace. An Administrative Court was adopted, and the Bankruptcy Law was amended to remove the three (3) case limit on trustees. 4 The World Bank, Private and Financial Sector Development Unit and the Poverty Reduction and Economic Management Unit. (June 2005) Improving the Business Climate in Macedonia: A Legal and Judicial Enforcement Assessment. and Russell-Einhorn, Malcolm and Jacek Chlebny, “Assessment of the Administrative Justice System in Macedonia – the World Bank Legal and Judicial Enforcement Project.� (mimeo). 2 key laws. Included in this menu of laws were, inter alia, the Law on General Administrative Procedure, the Law on Courts and the Bankruptcy Law. The LJIIS Project was envisaged to provide a critical commitment of medium-term support for aspects of the FYR Macedonia’s Judicial Reform Strategy. The Project was meant to complement the technical assistance and support provided by many of the other donors in this field. As a result, the Project components alone did not form a holistic approach to judicial reform, but rather were selected to scale up and provide economies of scale in areas were other development partners were already active. Chief among these activities was support for the continued roll out of the USAID-financed Automated Court Case Management Information System (ACCMIS). 1.2 Original Project Development Objectives (PDO) and Key Indicators (as approved) The objective of the LJIIS Project as set out in the PAD was to contribute to improving judicial efficiency and effectiveness and the business climate in FYR Macedonia by: (i) enhancing ministerial and judicial capacity to systematically implement the Government's Judicial Reform Strategy and key laws; and (ii) improving judicial infrastructure. The PAD also stated that achievement of the Project Development Objective was to be measured by performance indicators, including Government and judicial statistics supplemented by broad-based surveys (for example, BEEPS), expert assessments (USAID'S Commercial Law and Institutional Reform Assessments and Judicial Reform Index), annual European Commission (EC) reviews and others to be developed with the Government during Project implementation. 1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification No revisions to the PDO were made during Project implementation. 1.4 Main Beneficiaries, (original and revised, briefly describe the "primary target group" identified in the PAD and as captured in the PDO, as well as any other individuals and organizations expected to benefit from the project) The primary beneficiaries of the Project include: the Ministry of Justice, the courts of expanded jurisdiction, the newly created Judicial Council, the Supreme Court, the newly created Macedonian Bankruptcy Trustee Chamber (attached to the Ministry of Economy), the State Administrative Inspectorate, and the Judicial Training Center. Other beneficiaries included the Macedonian business and legal community as well as the general public. 1.5 Original Components (as approved) The original Project components were: 3 1. Ministerial and Judicial Capacity Building (Euro 1.855 million): This component aimed at strengthening the capacity of key institutions to implement the selected reforms under the Judicial Reform Strategy. The component had three subcomponents, namely: a) Improving the quality of judicial management and the Judiciary (Euro 630,000) b) Administrative Inspectorate and the Administrative Dispute Resolution (Euro 675,000) c) Improving Bankruptcy Administration and Ministry of Economy Supervision (Euro 550,000). 2. Infrastructure (Euro 7.725 million): Under this component 11 courts were to be rehabilitated and provided with basic furniture and equipment. The courts that were selected were the first instance courts of expanded jurisdiction. This selection complimented USAID’s earlier rehabilitation of the basic first instance courts. 3. Enhancing Judicial Information Technology Systems (Euro 1.92 million). This component provided for computers, training, software design and support. In addition, the Supreme Court’s Administrative Office made a specific request for the Bank to provide assistance for building its capacity to gather information from the Court Information System and its new budget and financial management software that were both being developed by USAID. 1.6 Revised Components There were minor revisions to the Project components during Project implementation. The first restructuring--which took place in February 2011--extended the closing date due to delays in Project activities—specifically court construction, contracting the functional analysis, and changes in management of the public administration for the Administrative Legislative Reform activities. No changes were made to the actual Project components at this time. The second restructuring took place in November 2011 and increased the Bank financing percentages from 84 percent to 100. Further, it reallocated funds in order to finance ICT needs for the High Administrative Court (which was established in June 2011), hardware for the State Administrative Inspectorate and the Ministry of Justice, and other expenditures related to the extension of the Project. In March 2012, a final restructuring extended the Project for an additional three months to complete the functional analysis (component 1) and the construction of the Skopje 1 courthouse (component 2). No changes were made to the Project components at this time. 1.7 Other significant changes (in design, scope and scale, implementation arrangements and schedule, and funding allocations) During implementation of the Project, the Supreme Court Budget Office was moved from the Supreme Court and placed under the newly created Judicial Council, as prescribed under the Law on the Amendments to the Judicial Budget (approved August 2008 and 4 entered into force in January 2009). The same law named the President of the Judicial Council as well as the President of the Budget Office. Some questions were raised regarding the centralizing of functions such as budget planning, judges’ evaluations, and training under the Judicial Council. Underlying these critiques, however, seem to be more fundamental issues related to the independence of the judiciary and the oversight function, more generally. In the case of the technical assistance component, some capacity building activities were dropped due to a lack of Government interest. It was only at the end of the Project, and after a change in leadership within the Judicial Council, that the Council began to take fuller advantage of the activities and financing available under the Project. With respect to the infrastructure component, there were two major changes in the construction/rehabilitation of court houses in Bitola and Skopje. In the case of the Bitola court expansion, the proposed changes and construction plans were never approved by the municipal council. This was due to the selection of a Project site in an area already in use by a waste water treatment plant and that could not be easily relocated. Subsequently, the Project had to be scaled back. A portion of the funds destined to build the new courthouse were used instead to remodel and refurbish the existing building. These changes financed the updating of the overall facilities, most of which dated back to 1974. The refurbished courthouse incorporated a counter (intake) system. It has been modified to allow for access for people with disabilities (e.g., ramps, elevator). The building’s electrical installations have been replaced, along with windows and doors. New furniture and equipment was also financed under the Project. By contrast, the scope of the Skopje 1 court only expanded after Project approval. The initial target date set for signing the construction contract was June 2008, with an October 2009 completion date. The court feasibility study financed by the Project in 2008 presented two different scenarios for construction and rehabilitation of Skopje 1 and the Government chose the more costly alternative. By mid-term review (2009) the Bank asked for a formal confirmation from the Ministry of Economy that the GoMK would be able to cover the US$1.0 million funding gap between construction costs and Bank financing. Furthermore, the selection of a larger building plan increased the overall tax liability for the construction. The estimated tax liabilities were not raised by the GoMK, and therefore no corresponding costs were incorporated into the original Project costs during appraisal. By 2009, the Minister of Justice arranged to cover the municipal taxes for the new building under the state budget; Project construction only began in May 2010, and its duration was estimated to be 18 months. The Skopje 1 feasibility study either underestimated or missed the issues related to the new courthouse site. Not only was the site located on an area of National Cultural Heritage (see Safeguards discussions below), the building footprint also required moving an electrical feed. Both of these issues had major cost implications and led to further delays on an already ambitious activity. As an end result, and despite an additional 18 months granted in Project extensions to complete the construction, the courthouse was not complete at the end of June 2012. Further complicating the construction process, the 5 austerity measures put in place by the Ministry of Finance led to a gap in payments to the construction contractor and a delay in signing the design supervision contract. In mid-2010, and after the construction of the Skopje 1 Courthouse had begun, the GOMK requested that the Bank eliminate the requirement for counterpart financing and increase its financing of activities to 100 percent. At the time of its request, the Ministry of Finance anticipated that counterpart shortfalls would continue due to the pending economic crisis. This request was reflected in the first Project amendment noted in Section 1.6 (above). It is important to note that this requirement was eliminated not just for this particular Project, but all projects in the FYR Macedonia portfolio. In order to access this additional financing, the Bank requested that the Government identify activities that could be ultimately cut from the Project. The Bank sent a clear message to the Government that the elimination of the counterpart financing requirement should not have a negative effect on the capacity building activities. Ironically, the courts received an overall budget funding increase in 2010, as their share of total finance rose from 0.4 percent of the total Government budget to 0.8 percent. The Information Technology component benefitted greatly from the development of a sector-side ICT strategy for 2007-11 developed with USAID. As a result, there were comparatively fewer changes to this component during Project implementation. When wiring the courts for the ACCMIS system, there were issues in connecting between floors (not contemplated in the bidding documents) which added costs. The Bank originally financed the costs of internet connectivity for one year through Macedonia Telekom, but due to issues of price, connectivity and low bandwidth, it was decided that the court would create its own virtual network. As a result, the Government proposed that the Bank Project finance the creation of Virtual Private Network (VPN) through a creation of a separate fiber optic network. This added costs to the Project, as new fiber optic channels needed to be dug, but it reduced the price of connectivity. It also allowed the ACCMIS system to operate on a higher bandwidth, as the ACCMIS system had been unable to properly replicate on the MK-Telekom system. While the VPN was a more sustainable solution overall, it has added costs to the Project and an extra burden on an already taxed and small Supreme Court ICT department. In terms of implementation arrangements, it should be noted that the Project Steering Committee indicated in the PAD did not function as originally intended. The Steering Committee met infrequently. This, however, did not ultimately impact the Project outcomes. Instead, the Project was primarily centered at and implemented by the Ministry of Justice through the PIO. 2. Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design and Quality at Entry (including whether lessons of earlier operations were taken into account, risks and their mitigations identified, and adequacy of participatory processes, as applicable) In preparing the Project, the Bank team built on analytical and diagnostic work undertaken in the areas of the business climate and administrative justice. Furthermore, it 6 built on the prevailing wisdom in judicial reform at the time, namely that sector strategies developed by the Government signal ownership. The increased independence of the judiciary created through multiple legal reforms, linked with the Government’s desire for EU accession led to tangible reforms. The Quality Enhancement Review (QER) provided useful guidance on narrowing the scope of the Project to highlight those aspects of the Bank’s specific support to the justice system. Furthermore, the QER emphasized that the Project should not place conditionality on the passage of additional laws as a means of managing the external factors related to reform. Much of the conditionality originally contemplated for this Project was passed on to the series of DPLs. The QER team suggested that the effectiveness should be timed to after the formation of a new Government. Questions were raised as to the effectiveness of the proposal of having the FYR Macedonia Judicial Reform Council serve as the Project’s Steering Committee, when the Government had not necessarily supported this idea. In the end, the PAD notes that the Project would be overseen by a Steering Committee, appointed by a formal Government decree. The Steering Committee was to be comprised of members of the Supreme Court, the Judicial Council, the Ministry of Justice and the Ministry of Economy. Shortly before the QER, questions arose regarding the site selection and preparations for the construction of the Skopje 1 Court. Indeed, for an investment of this size, it appears that more thorough analysis and preparation needed to be done. At the time of Project approval, neither the site selection for the Skopje 1 court was finalized, nor were the plans for the construction of the courthouse fully developed. The construction plan was finally bid out in 2008, almost three years after Project approval. Unlike the construction aspects of the Project, the Information Technology component had a strong technical basis to start from, as USAID had been working with the Macedonian Judiciary on a case management system since 2002. These early interventions not only provided a model and infrastructure for the Bank to build on, it also afforded the Project team cost data to build Project estimates on. Furthermore, the USAID project led the introduction of court technology into the courts – a process that can take many iterations and also needs a cultural shift. The investments the project made by USAID in terms of human resources and inputs greatly reduced the need for large scale investments or sweeping changes by the Bank Project. 2.2 Implementation (including any project changes/restructuring, mid-term review, Project at Risk status, and actions taken, as applicable) At the beginning of the Project, there were difficulties in staffing the Project Implementation Office (PIO). High turnover continued well into 2009; this had a negative impact on all Project activities, but particularly the Ministerial and Judicial Capacity Building component (component 1). 7 A few issues arose during the mid-term review, but none required major restructuring. The Project faced a number of implementation issues – some of which have already been discussed above. The Bank ultimately agreed to drop the counterpart financing requirement in 2010, with the understanding that the Judiciary could not disregard the capacity building activities contemplated under Component 1 and solely focus on court construction. 2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization Considering the importance of FYR Macedonia’s efforts toward EU accession, a considerable amount of information and some quantitative data on more general aspects of the Macedonian judiciary was available to the Project team at the time of Project design. This allowed the Project team to maximize its monitoring and evaluation through the use of independent and quantifiable data that was produced in regular intervals. The data was then used as proxy for concrete results with the reforms and activities planned under the Project. Nevertheless, the Project’s dependence on externally produced indicators led to hiccups in monitoring (described below) and also may have reduced the Bank’s ability to hold a more specific dialogue on Project results, the need for capacity building and impact. The Project used the EBRD-World Bank Business Environment and Enterprise Performance Survey (BEEPS) as a main source of Project indicators. While the data was collected and reported in regular intervals, the criteria for both the Bank-sponsored BEEPS survey and Doing Business changed over the lifetime of the Project. As these indicators have been collected independently of the Project, the changes were outside the scope of the Project’s control. As a result, these Project indicators at the end of the Project are more of a general benchmark than a clear indicator of Project success/failure. As for Intermediate Results Indicator # 1 – the American Bar Association Rule of Law Initiative’s (ABA-ROLI) Judicial Reform Index Facts 2,15, and 17 on judicial selection and appointment- the last time this indicator was measured was in 2006. The indicator was discontinued when ABA-ROLI closed their FYR Macedonia program. The fact that this indicator could no longer be measured was only noted in the Project ISR in 2009. Similarly, the Intermediate Outcome Indicator that measured the time to dispose of a commercial case was also dropped in 2009, and a new indicator was introduced to better measure the effectiveness of component 3. This indicator measured the use of the enhanced case management and court information system implemented with Project financing. As for Project-specific indicators, Project Development Indicator #2, the share of respondents to court user survey indicating satisfaction with the operation of the courts, was not formally measured until a user survey was carried out in 2009 for the justice system. No baseline survey was ever carried out; the survey carried out (in 2009) was proposed in its place. The user survey was repeated at the end of the Project in mid-2012. The later iteration of the user survey suffered some alterations to the criteria and the 8 questions asked. Therefore, it is difficult to make clear comparisons between the survey carried out in 2009 and in 2012. Similarly, a functional analysis of court cases was carried out near the end of Project implementation. One of the key objectives of the functional analysis was to design a methodology to be used in the calculation of costs per case. Due to many delays on the Government side, the functional analysis was only completed in early 2012. The Court Budget Council (CBC) requested that the courts prepare an annual plan for court performance in June 2012. This annual plan was to contain a projection of ongoing cases for the subsequent year. Based on these projections, the CBC was to analyze the data presented and for use in their future budget requests. 2013 will mark the first draft court budget sent to the Ministry of Finance that is based using the model of calculation of the court case price cost. 2.4 Safeguard and Fiduciary Compliance (focusing on issues and their resolution, as applicable) At the Project outset, the LJIIS Project was not expected to have any significant or irreversible environmental impacts. The Project was classified as a Category “B� project due to the court infrastructure component. An Environmental Management Plan (EMP) was developed by the Ministry of Justice and a draft was provided to the affected community and NGOs prior to its implementation. The EMP focused on 12 existing facilities where court rehabilitation and court construction were to take place throughout FRY Macedonia. No land acquisition was anticipated as all the sites for new construction or expansion belonged to the Government. Perhaps because of this, the EMP was optimistic and silent on the potential processes and issues related to the actual Project sites, despite the fact that the locations were predominantly in urban areas. By Macedonian law, any new construction or expansion of existing buildings must be included in the municipality’s urban plan. As noted earlier, in the case of the proposed Bitola court expansion, the Project-financed proposal was never approved as part of the municipality’s urban plan, and the overall infrastructure activity was subsequently scaled back. During the municipal review process for the Skopje 1 building, and well into Project implementation, it was noted that the Skopje court site was an area of historical and cultural importance. Delays were incurred as the Project team worked to have a monument to World War II resistance removed and re-sited near the construction area. Furthermore, an electrical transformer station serving the entire court complex also needed to be moved prior to initiation of construction. Altogether, these delays added an additional time to the construction timetable. During the reconstruction of the Kocani basic court, a potential resettlement issue was noted in the construction of a planned perimeter fence. This fence would have run into three structures located on or near the court property. The MoJ and the local authorities addressed the resettlement issue through the development of a detailed municipal plan 9 which would call for their ultimate removal. The Project did not finance this municipal plan; it was developed and implemented by the Government authorities. During the construction of the perimeter fence, access was given to the inhabitants of structures until the municipal plan was complete. In terms of fiduciary controls, the Project has consistently maintained a clean and transparent accounting for Project activities and funds. Throughout Project implementation, there has been no qualified opinion on the use of Project funds. 2.5 Post-completion Operation/Next Phase (including transition arrangement to post-completion operation of investments financed by present operation, Operation & Maintenance arrangements, sustaining reforms and institutional capacity, and next phase/follow-up operation, if applicable) The largest area of concern for the post-completion phase of the Project is the incomplete construction of the Skopje 1 courthouse. Despite the Bank’s best efforts to expedite construction over the prior year, the building was not completed at the time of Project closing. The Government extended the contract until September 30, 2012; the additional Euro 3.0 million estimated to complete the contract was to be paid out of the state (MoJ) budget. It should be noted that in July 2012, the Bank Country Manager sent a letter to the Ministry of Finance raising the issue that a number of invoices eligible for Bank financing had yet to be paid. The total amount of these invoices was approximately Euro 1.0 million. Plans to furnish the Skopje 1 court – using World Bank financing—were stopped due to delays in the overall construction. Due to the delays in finalization of the construction and challenges with counterpart funding, the bidding documents to equip the courts with furniture and appropriate ICT equipment for Skopje 1 were developed and finalized but not put out for bid during the Project. There were no funds allocated for their purchase in the MoJ’s budget in 2012. The MoJ is still exploring possibilities of financing this activity with other means of financing (grants, credits etc.). 3. Assessment of Outcomes 3.1 Relevance of Objectives, Design and Implementation (to current country and global priorities, and Bank assistance strategy) As noted previously, the Project initially was not contemplated in the first Bank Country Assistance Strategy for the years 2004-6. At the time the Country Partnership Strategy (CPS) was developed, the Project’s objectives played a more prominent role (Report 54928-MK, September 20, 2010). The Project fit squarely within the CPS’s discussion of improvements to the business environment and overall competitiveness. The original Project objectives have largely been met, and six years after Project design remain relevant. The Project design was flexible and broad enough to incorporate a number of actors within the justice sector. In some instances, the Project may have been 10 too flexible with its financing (e.g., the purchase of equipment for the High Administrative Court), which was not included in the original Project design. 3.2 Achievement of Project Development Objectives (including brief discussion of causal linkages between outputs and outcomes, with details on outputs in Annex 2) The Project’s development objective has been largely met. Above and beyond the issues related to the Skopje 1 courthouse, it must be said that much additional effort needs to be placed in modernizing and improving the court’s human resource management system. Human resources planning needs to be based on whole court functions, without prejudice over one particular function. The current ban on hiring court staff (i.e., staff other than judges) can and will negatively impact the judges’ performance in the near to mid-term. Considering that greater workloads will be placed on the same staff due to the country’s adoption of e-process serving and audio/visual recording of court sessions – this rationalization of staff, the standardization and unification of job profiles and extensive training will be needed, in addition to a real increase in the number of staff, generally. 3.3 Efficiency (Net Present Value/Economic Rate of Return, cost effectiveness, e.g., unit rate norms, least cost, and comparisons; and Financial Rate of Return) Measuring the efficiency of this Project is difficult, as there was little economic/financial data that was presented in the PAD. Overall indicators, couched in more general terms were presented, but the overall economic impact of these reforms have been affected by a number of exogenous factors, the most important of which has been the economic crisis that began in 2010. In terms of cost-efficiency, the Project was most cost-efficient in providing ICT support to the judiciary (component 3) as well as support to the State Administrative Inspectorate and Bankruptcy Trustees. The Project was much less efficient in delivering technical assistance to the Judicial Council and in providing infrastructure. The relative success of the provision of ICT solutions to the courts in general, as well as State Administrative Inspectorate, was due to the fact that the Ministry of Justice had been working closely with USAID on the development of an ICT solution for case management. A specific ICT strategy was developed for 2007-11 and proxy costs were available to the Bank team to develop reasonable estimates. As a result, the investments made toward equipping the courts, networking them and engineering a technological platform for case management were well targeted, and the potential pitfalls were identified and resolved. Similarly, the SAI’s small size and procedural system lent itself readily to an ICT solution. The SAI case management system has ultimately led to a drastic reduction in overall time resolving disputes, as well as a reduction in certain overhead costs (e.g., postage for official business). As noted above, the less successful elements of the Project were those that involved technical assistance activities aimed to build capacity. The Judicial Council showed little 11 interest in the technical assistance component and its activities for the first years of the Project. As a result, related disbursements were low. It was only at the end of the Project, and under new leadership, that the Judicial Council began to show more interest in implementing the activities. With regards to the infrastructure component--the Project’s largest component in financial terms--the delays and complexity added during Project implementation detract from the relative success of the remodeling and refurbishing of the other eight courts of expanded jurisdiction. Prior to the beginning of the current economic crisis in 2010, the construction of Skopje 1 was already over budget. The economic crisis, numerous construction delays, and the fact that the Project at appraisal did not build in specific time and cost contingencies have jeopardized the outcome and sustainability of the Project as a whole. 3.4 Justification of Overall Outcome Rating (combining relevance, achievement of PDOs, and efficiency) Rating: The overall outcome rating for this Project is Moderately Satisfactory. The Project was relevant at the time of preparation and (overall) implementation. It did achieve all/most of its development indicators. The largest stumbling block for the Project, however, were the risks posed due to the construction of the Skopje 1 court. Construction delays and non-payment of invoices by the GoMK related to the country-wide austerity measures detracted from dialogue on the need for continued technical assistance and capacity building. As the stress has been largely on completing the court–little dialogue has been made with respect to longer term capital investment planning to sustain the activities financed by the Project, particularly in the area of information technology. 3.5 Overarching Themes, Other Outcomes and Impacts (if any, where not previously covered or to amplify discussion above) (a) Poverty Impacts, Gender Aspects, and Social Development The Project did not have a specific poverty or gender strategy inherent in its design. Moreover, the financial accessibility of the courts by the user was not addressed by the Project. One anomaly arose in the analysis between the 2009 and 2012 court user survey. In the final user survey 20.6 percent of those surveyed felt that court costs were high; this represented a decrease in the findings of the 2009 survey. 5 While the decrease (1.8 percent) is well within the standard margin of error, it is interesting to see that not more users did cite a high cost in going to court as a result of the protracted economic crisis and its impact on disposable incomes. More importantly, however, was the development in the view of the ethnic minorities at the end of the Project. In 2012, the court user survey noted that ethnic Albanians were 5 Ibid, page 37. 12 17% more likely than ethnic Macedonians to believe that trials are never impartial or fair. This represented a 7 percent increase from the first user survey.6 (b) Institutional Change/Strengthening (particularly with reference to impacts on longer-term capacity and institutional development) The Project contributed greatly to the greater professionalization of the Macedonian judiciary, insofar as it provided the tools for more skilled or technical decision-making within the courts, both on matters of justice, as well as resource use. The modest size of the Project has allowed for some progress to be made in terms of institutional strengthening. The establishment of the Judicial Academy during Project implementation was a positive contribution in terms of institutional change and strengthening. There is great acknowledgement within FYR Macedonia –though oftentimes begrudgingly- that the current performance measurements for judges and courts alike are now the norm in FYR Macedonia. The courts now rely on data generated by the case management system, which ultimately provides greater transparency and predictability for the public and court users. At the same time, the Project’s successes have also led to the uncovering of many more fundamental problems within the sector – such as the lack of capital planning for ICT issues as well as the court infrastructure. Furthermore, many judges and lawyers were quick to point out that while the judges are now being held accountable for their work, the Judicial Council’s has emphasized speed in clearing dockets over the quality of the judgment. A similar complaint was that the existing system does not capture case complexity as part of its measures or evaluations. One of the more salient problems apparent within the Macedonian courts is the fact that court presidents face great pressures to be more productive in disposing of cases, while they face inadequate operating budgets and hiring rigidities and freezes. The hiring freezes on key court staff (administrators, accountants, ICT personnel) jeopardize the court’s functioning as these personnel are now integral components of the judicial process. They are responsible for key portions of the ACCMIS system and furthermore are responsible for providing inputs to the Judicial Council. Despite a high level of centralization and control over performance-related issues, the decentralization of procurement and human resources activities leads to some disconnects within the system, and leads to potential issues of conflict of interest and even corruption. The Project’s ICT purchases for the ACCMIS system led to economies of scale the Project and more robust staffing strategies could create economies of scale and smoother operation within the courts. 6 Institute for Political Research-Skopje, Court User Survey (Comparative Analysis 2009-2012), June 2012, p. 38. 13 The impending economic crisis has, indeed, jeopardized some of the progress made. This Project has served to highlight that a much more robust and sustained effort is still needed to address systemic issues within the judiciary. The Justice sector is in need of more robust and holistic planning capacity. The timeframe of the Justice Sector Strategy has expired, and no new strategy has emerged. Similarly, the Strategic Plan for the courts’ ICT department has not been developed or funded. (c) Other Unintended Outcomes and Impacts (positive or negative) The World Bank Project was implemented during a period of rapid reform, realignment and expansion in justice sector institutions. Much of this change had a positive and complementary impact on the Bank’s Project. Nevertheless, such massive change necessitates proper management as well as sufficient time to consolidate the reforms. During the course of Project implementation, and under the auspices of the USAID project, the Macedonian courts have begun a process of opening and transparency. Court presidents have been given the responsibility of court spokespersons. They have received training in dealing with the media, and brochures and information have been provided for dissemination to the public. In addition, each court now has its own webpage that provides information to the public on court cases and court activities. Judgments are published on the web and can be consulted by the public when finalized. Some court websites provide information to the general public on the court’s work. Nevertheless, it was noted that the information provided on courts websites was not always systematic or user-friendly. There was little systematic indexing of decisions released. Over time, the continued openness of the courts is likely to lead to more positive outlooks and greater predictability. In order to achieve this milestone, the courts need to analyze more closely what the public and stakeholders would like to see and know about the courts. Furthermore, the courts need to think more strategically about their overall messages and the best ways to convey them in order to improve the public’s perception of the judiciary. 3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops (optional for Core ICR, required for ILI, details in annexes) At the end of the Project, a user survey was carried out as a follow-up to a 2009 user survey. The follow-up survey loosely used the same criteria, and provides a comparison between the Project’s mid-point and end. The final user survey noted that the country as a whole remained committed to judicial reform. Overall awareness of the courts and their functions had grown over the course of the Project. There was a marginal increase in the overall confidence in the judicial system, though the difference appeared more marked between confidence levels of the general public and the ethnic communities. The latter of which- particularly the Albanian communities—were more likely to cite the system as being unfair and costly. The overall cost of the judicial proceedings was cited as one of the system’s weaknesses at the time of Project design. Though costs were not specifically targeted as part of the 14 Project, a perceived high cost of justice negatively impacts potential court users. This perception remained throughout the Project, despite the fact that a system of proportional court fees was put into place in 2009. 4. Assessment of Risk to Development Outcome Rating: Substantial. There are a number of real risks to the Project’s development outcome. These risks include financial, material, human resources and capacity. Each of these risks is spelled out below. The country’s overall economic downturn, coupled with the intense focus on completing Skopje 1 has taken away from other necessary and important capital improvement planning and financing. The fact that the largest portion of the courts’ budget goes to salaries, and comparatively little is allocated to capital investment and operating costs jeopardizes many of the reforms and advances made under the Project. The delays incurred in completing the construction of the Skopje 1 building has ultimately meant that the Government must now finance on its own an estimated Euro 3.0 million to complete the building; in 2009, this financing gap was estimated at only Euro 1.0 million. There is a very real risk that even if this court is completed in 2013 that it will sit empty for quite some time (a period of months) without furniture. While this was not a specific outcome of the Project, the impact of individual courts carrying debts within their own locales has a potential for grave corruption and miscarriage of justice within the system. Currently, the Macedonian judiciary does not have a centralized purchasing unit for the procurement of recurrent costs such as printer toner, paper, and computer service. Due to very tight provincial and municipal budgets and cumbersome procurement procedures, there is a potential for individual courts to carry debts with creditors in their locale. Some courts reported having daily contact with their debtors. This not only takes away from time that could be used more productively in the court’s business, it also jeopardizes the judiciary’s independence and integrity. There are significant human resources and capacity risks to the Project’s development outcome. The Project has been most successful at providing the needed tools and inputs for the judiciary to improve its functioning. As the GoMK has placed focus primarily on judges for the delivery of key results, the staffing of other key actors – such as court administrators and clerks of court, ICT professionals, and financial officers—are being overlooked. Complicated hiring procedures and hiring freezes for non-judges put at risk the capacity and court management model that the Bank Project helped to finance. 5. Assessment of Bank and Borrower Performance (relating to design, implementation and outcome issues) 5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry (i.e., performance through lending phase) Rating: 15 The Bank’s performance in ensuring quality at entry was moderately unsatisfactory. The team built its program on the Government’s own Strategy for the Judiciary. As a result of the QER review, the Bank team sharpened the focus of the Project activities. Furthermore, the Bank team worked with other donors to ensure that any Bank-financed activities fit within the Strategy and were complementary to those of the other donors. The Bank leveraged its Monitoring and Evaluation activities by using existing methodologies and data stores (e.g., Doing Business, BEEPs, ABA-ROLI’s Rule of Law index) which were considered cutting edge indicators at the time of Project design. The Project could have had a much higher rating had there been more concrete plans for the construction generally, and the Skopje 1 courthouse specifically. The original Project procurement plan set 2009 as the date the Skopje 1 construction was to be completed. Furthermore, as the Government selected a more costly option for the construction of the Skopje 1 courthouse, the lack of Project contingency funds in the original Project designs, coupled with price escalations meant that tough decisions needed to be taken as to what would be financed under the Project. (b) Quality of Supervision (including of fiduciary and safeguards policies) Rating: Moderately Unsatisfactory. During the early implementation of the Project, the Bank team struggled due to the poor functioning of the Project Implementation Office. The issue was flagged with Bank management, and the Ministry of Justice ultimately responded. Once the PIO was adequately staffed, the Bank team began holding videoconferences in order to resolve issues and move ahead with the Project activities more quickly. Closer supervision by the Bank team on the construction of the Skopje1 was urged on a number of occasions by Bank management, and was most certainly warranted. Issues such as the location of the World War II Monument the electrical substation and the water pipes on the site selected for construction were areas that should have been flagged by the Government—if not detected by the Bank team--during the design phase. These “chance finds� unfortunately lead to delays and price escalations that escalated into major problems by Project close. The Project Mid-Term Review documentation did not discuss the potential problems or risks--particularly those that related to the construction of Skopje 1-- in much detail. The Mid-Term Review Aide Memoire was optimistic about the completion of Skopje 1 and relatively optimistic about subsequently financing the remaining Project activities. As a result, there was no record of serious discussions with GoMK about restructuring Project components and activities or scaling back. As the Project progressed into 2010, the general economic crisis as well as delays in construction put further pressures on the GoMK and the Bank team, and ultimately the Project’s capacity building activities were dropped and scaled back. 16 The inclusion of a local Bank Staff member on the Project team–who incidentally had a legal background-- helped to maintain the pace and pressure on the Government beginning at the Mid-Term review. His own dedicated interaction with the Project Implementation Office beginning in December 2009 helped to ensure that major issues were brought to Project and Bank management’s attention and dealt with in a timely manner. As the Project neared completion, the Bank Task Team Leader worked within the existing Project structures to ensure that Project activities were completed prior to closing. Despite his best efforts, through the frequent interaction of local staff and frequent missions to Skopje, the construction was not completed at the time of the Project close. The non-payment of construction invoices by GoMK in the final months of the Project- due to a lack of fiscal space in the Government’s overall budget-made a difficult position intractable. The delays made within the Judicial Council in the completion of the functional analysis meant that the Bank-financed activity was only completed and approved for internal use at the end of the Project. Even with the interventions of the Country Manager, the scope of the Project had been cemented during the Mid-Term Review, meaning that the Bank Team had little room within the Project to maneuver better outcomes. (c) Justification of Rating for Overall Bank Performance Rating: Moderately Unsatisfactory. The Bank missed opportunities in deepening the dialogue with the GoMK on judicial reform on a number of occasions. The reliance on externally produced indicators during Project design reflected the most advanced thinking on justice sector performance at that point. The fact that the Bank team did not insist on complementary indicators—for instance, the baseline survey of court users--meant that at the end of the Project it was difficult to measure the Project’s success or failure. Similarly, the Project’s Mid-Term review was largely a tacit approval of the Project activities already underway. The Bank team missed an opportunity to discuss and document difficult discussions about major slippage in the original construction timetable and cost overruns in the construction of Skopje 1, the biggest and most complex Project activity. The Bank could have used the Mid-Term Review as an opportunity to leverage the GoMK’s enthusiasm of court construction to build a greater dialogue around overall capacity building for areas such as capital investment, budgeting and human resources. The plan set in motion during the Mid-Term Review meant that the vast majority of Bank resources would remain focused on ensuring the completion of the Skopje 1 courthouse by Project close. This plan and the Mid-Term Review’s optimism proved to be unrealistic. 5.2 Borrower Performance (a) Government Performance Rating: The overall rating for the Government Performance was moderately satisfactory. The Project started with significant Government commitment in the form of a fully fledged sector strategy that identified key priorities for reform within the justice sector. The 17 strategy was an extremely important input in helping develop the Bank Project. Despite a rocky start, the Ministry of Justice’s commitment to justice reform and the Bank Project was strong throughout the Project. The second Minister of Justice was particularly dedicated to seeing the Project become a success. During the course of the Project, the Government created or implemented 10 justice sector institutions, and enacted sweeping changes in the country’s legislation. Given the breadth and the complexity of the changes introduced, the Government made admirable progress toward consolidating the judiciary as a whole, and providing it with tools (such as the ACCMIS system) to improve its functioning. Judges and court staff appreciated and accepted the case management system as an important tool in their work; however, some disagreed with the reporting and analyses that were made at the Judicial Council regarding the overall efficiency of the courts themselves. On paper, the court efficiency criteria captured by the ACCMIS system are transparent and straightforward. Some judges cited the fact that speed should not be the only criterion for measuring judicial performance. The World Bank Group’s research in this area supports the judges’ claim. At the same time, the issue is common throughout the region and the focus on efficiency is often a result of the fact that it is easier to measure than quality. As the ACCMIS system reflects the current law, and has been accepted as a tool for measuring court performance, perhaps a second step or tool to be developed in the near future is one that also incorporates the overall complexity of a case or the level to which the existing laws and precedents are applied to a case. At the end of the Project, fiscal pressures due to the larger poor economic climate led to Government-wide budget pressures that delayed payments and affected overall implementation. The Bank sent multiple notices to the Ministry of Finance about the GoMK’s non-payment of Project invoices (with Bank funds). These issues were ultimately resolved, but only after lengthy delays. (b) Implementing Agency or Agencies Performance Rating: The overall rating for the Implementing Agency Performance was moderately satisfactory. At the outset of the Project there were delays in staffing the Project Implementation Unit, some of which is to be expected in a small country with a large donor presence. Turnover in the Project Implementation Office staff was a constant problem for the first three years of Project implementation. Changes in staffing for Component 1 in particular led to disruptions in dialogues necessary to help build consensus around some of the Project’s proposed capacity building activities. In areas where there was considerable Government ownership-such as information technology and court construction—the PIO’s performance was generally much better. As a result, the Project largely met its Project development objective. The economic crisis, however, loomed large in Project implementation. The Government-wide ban on signing new contracts in 2012 led to delays in the signing of critical contracts and issuing payments. 18 (c) Justification of Rating for Overall Borrower Performance Rating: Moderately Satisfactory. The Government was left with a number of incomplete activities at the end of the Project, above and beyond the status of the Skopje 1 courthouse. There were many missed opportunities for improved dialogue on capacity building issues within the Judiciary in particular, despite the efforts of the PIO to keep Project activities moving forward. For the first years of the Project, the GoMK’s focus was on court construction, thereby eclipsing capacity building activities. Opportunities were missed in terms of building capacity for the courts to better plan and utilize their budgets. Ultimately, some of this capacity building effort could have been focused on promoting greater sustainability of the courts and their operation. 6. Lessons Learned (both project-specific and of wide general application) One clear lesson that has emerged generally from the World Bank’s ICRs on justice sector projects is that these types of investments are often ill-suited for the time period of a single investment loan. It should be no surprise, then, that the Project was extended from its original four year window to six years. The Project’s heavy emphasis on infrastructure and the renovation of courts in particular necessarily led to delays in implementation, and two Project extensions, totaling 18 months. The addition of many new justice sector institutions and activities by the GoMK complicated Project implementation. The infrastructure activity took the lion’s share of the Bank financing as well as supervision efforts. If infrastructure is seen as a precursor or the first sequence in reforming the judiciary, then a secondary or equally important reform would be to building the capacity – technical, human and planning capacities of the judiciary to maintain and consolidate the achievements made to date. While the Project had built in this balance into its Project design, the overall implementation of these activities was uneven. Unfortunately, efforts to increase the planning capacity came late in the Project, when it should have been addressed as an on-going concern from Project effectiveness. While it is important to create better physical working conditions for court personnel and court users, without the proper planning or staffing, the overall impact of these changes on the court’s total performance can and will only be limited. The national judicial training academy has made important advances in improving the overall capacity of the judges, prosecutors and supporting staff at a time of rapid change in the country’s legal framework. Nevertheless, it must be said with the rapidly evolving legal framework, consolidation of this learning – and the reforms will take more time than the timeframe of the Project, and is subject to reversals. 19 At the outset the Project team relied on externally produced indicators (Doing Business, ABA-ROLI, EBRD) to measure Project success. While this was both a shrewd decision, given the overall expense and expertise needed to properly monitor programs such as these, it also means that there are fewer results and successes that can quantifiably be linked to the Project. Changes in the Doing Business indicators, the discontinuation of the ABA-ROLI judicial selection and appointment indicator during implementation meant that the Project team needed to rethink and retool the Project indicators at a time when most of the Project funds were already committed for other activities. Delays in contracting the user survey and inconsistencies within the survey itself also detract from an analysis of the Project’s direct impacts at Project completion. Furthermore, with the main Project development objective being tied to court improvements, greater emphasis should have been placed on carrying out both the baseline survey of court users, as well as the functional analysis of the courts at an earlier stage in the Project. The change in leadership within the Judicial Council led to a better understanding of the usefulness of the functional analysis and greater ownership of the capacity building activities. As regards sustainability, there is little evidence that there has been any serious planning or projecting the needs for the full establishment of Skopje 1. The Government had committed to financing the completion of the building by September 2012 using its own funds, but no bidding processes have begun to furnish and equip the new building. Given FYR Macedonia’s current economic crunch, it is unlikely that the Government will have the money to pay for this massive capital investment in the near term. Similarly, the lack of capital investment planning to support and maintain the current ACCMIS system leads to questions about the viability of this solution in the medium and long-term. Finally, serious consideration should be given to the medium and long term staffing of the courts, expanding the judiciary’s narrow focus on the judges and extend this to court administrators and clerks, financial officers and ICT professionals in order to maintain the integrity of the current system. Finally, measuring this Project’s success is extremely difficult given the Macedonian justice sector’s rapid transformation during Project design and implementation. It will take considerable time and resources to consolidate all of the reforms set in place during Project design and implementation. The Project made positive contributions to the implementation of new court case management and bankruptcy procedures, and streamlining administrative cases, and as such, the Project’s success should ultimately be measured with this in mind. Sustainability becomes a pressing issue, however, when the Project is viewed in the light of additional pressures from the GoMK’s adoption of e- service of judgments and the electronic recording of court hearings. The overall success of the Project will depend largely on a sustained, open dialogue on issues of capacity and continued support and technical assistance by donors to help in the consolidation process. 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners (a) Borrower/implementing agencies No issues raised. (b) Cofinanciers 20 The Project was implemented in parallel with other donor projects, notably two successive USAID projects. During the design and implementation of the LJIIS Project, the cooperation between these two entities was highly beneficial to both parties. It should be noted that the early USAID project created the groundwork that led to the ultimate success of the World Bank funded Project. Without USAID’s initial investment and technical assistance, the Bank Project would not have created the same impact. (c) Other partners and stakeholders (e.g. NGOs/private sector/civil society) The private sector was the largest group of stakeholders within Macedonia. They did see some improvements in the sector. The passage of new laws led to reduced processing times within the courts, which was recognized by judges and lawyers alike. Nevertheless, there was a general sense that FYR Macedonia’s laws could be improved. Also of note was the general view by the public that there was great room for improvement in the public outreach initiative begun by the courts. Private lawyers commented that the judicial performance measurement had little impact on the quality of judges. This sentiment was also echoed by the Macedonian Association of Judges. The Macedonian Association of Judges in particular felt that there should be some changes made to the current system of judicial performance measurement to include the overall complexity of the cases. In addition, they noted that some criteria should also be introduced in the measurement to include the overturning or upholding of cases at the appellate level. 21 Annex 1. Project Costs and Financing (a) Project Cost by Component (in EUR Million equivalent) Appraisal Estimate Actual/Latest Estimate Components (EUR millions) (EUR millions) Percentage of Appraisa Ministerial and Judicial Capacity Building 1,855,000 889,953 48% Improving Judicial Infrastructure 7,735,000 5,203,866 67% Enhancing Judicial IT System 1,920,000 1,510,629 79% Implementation Office 305,000 403,440 132% Total Baseline Cost 0 0 Physical Contingencies 0 0 0 Price Contingencies 0 0 0 Total Project Costs 0 0 Front-end fee IBRD 25,000.00 25,000.00 100% Total Financing Required 11,900,000 8,032,888 68% (b) Financing Actual/Latest Appraisal Estimate Estimate Percentage of Source of Funds Type of Cofinancing (EUR millions) (EUR millions) Appraisal Borrower State Budget 2.36 0.79 3 International Bank for Reconstruction and Development 12.41 8.03 6 22 Annex 2. Outputs by Component Activity Deliverables/Result Impact Component 1a.Improving the Quality of the Judicial Management and the Judiciary Renovation and expansion of Renovated space as well as an Better working conditions for the office space for Judicial Council. extension of 300m2 office area Judicial Council. for Judicial Council. Software for electronic delivery Web application for distribution Use of software to provide for system for distribution of court of court documents. electronic delivery of court documents. documents. Survey of the public, judges’ and The Judicial Council receives JC employees’ experience feedback from the public, staff, regarding the functioning of the and judges on the competencies Judicial Council and the judicial and functioning of the Judicial reforms. Council as an institution. Study trips for the Judicial Visits to judicial institutions of The Judicial Council members Council. Slovenia, Italy, Lithuania. gaining comparative experiences in the area of selection, evaluation and promotion of judges from other countries. Security equipment for Judicial Improved security for the offices Council and airconditioning for of the Judicial Council; basic the Server Room of the MoJ. conditions for secure operation of the server room of the Ministry of Justice. Functional Analysis of the Courts Analysis of the entire judicial The analysis is a base for areas for the CBC. system, identification of gaps in that need future improvement in the area of monitoring and the judicial system. evaluation, strategic management of the courts and determining the cost of court cases. Study trip for the Administrative IACA Conference in Istanbul. Better understanding of the Office of CBC. capacities and competencies of the administrative staff of the newly instituted Court Budget Council on court administration issues. Component 1b. Administrative Inspection and Administration Dispute Resolution Training for SAI Management. EGPA Conference-Madrid Improvement of the capacities IIAS Conference-Helsinki and competencies of the director EGPA Conference-Malta and inspectors of the State NISPA Conference-Warsaw Administrative Inspectorate in the area of public administration. Training of SAI on 8 two-day trainings should be Better and more consistent Implementation of Internal organized for 45 understanding and application of Regulation. participants/inspectors the internal procedures for (discussions and sharing conducting inspection of the State experience with resolution of Administrative Inspectorate for typical problems, addressing all inspectors throughout the problems regarding the country. implementation, which occur in the FYR Macedonian administrative practice). Development of Management Web based management The system supports the everyday 23 Information System for SAI and information system for State work of inspectors and other subsequent training for Administrative Inspectorate. inspectorate’s staff, professional inspectors. development of the administrative inspectors. Furthermore it strengthens communication and information flow. The system captures enterprise-level data to allow the inspection management to set priorities, allocate resources and, evaluate the outcomes (results) through quantitative performance indicators. Hardware support for SAI and Mobile computer access to the Ministry of Justice -laptops and SAI Management Information multifunctional devices. System for the inspectors for uninterrupted field work. Completing the reform of the Organization of a set of Better implementation aspects of administrative legislation, workshops providing direct public/administrative law vis-à- administrative judiciary and analysis, assistance, advice and vis the European legislation with public administration. concrete suggestions by experts regards to the Human Rights in the administrative-legal area in order to better harmonize national Court in Strasbourg and the legislation with European law. principles of the public administration performance. Furthermore, promotion of a common understanding by the State Administrative Inspectors on : (a) the Law on Organization and Work of the State Administration Bodies; (b) the Law on General Administrative Procedure and (c) the Law on Administrative Disputes. Component 1c. Improving Bankruptcy Administration and MoE Supervision Training workshops and study Study trip to Estonia and Creating, maintenance and trips for the bankruptcy trustees. Romania; improvement of the bankruptcy Two workshops related to trustee profession, which increasing the knowledge and contributes to the improvement of specialization of the licensed the judicial efficiency in trustees. conducting the bankruptcy procedures. Support for MoE Bankruptcy Design of an electronic database The system supports the everyday Supervision Office- Statistics/IT for the Bankruptcy supervision work of Trustees. The system consultant. Unit of the Ministry of Economy captures enterprise-level data to which meets the requirements of allow the inspection management content, timeliness, access, and to set priorities, allocate resources ease/cost of maintenance. and, evaluate the outcomes (results) through quantitative performance indicators. Bankruptcy Training Program. Provide judges and prosecutors Full understanding of conducting specialized bankruptcy law bankruptcy procedures for the training by enhancing their ability judges, prosecutors and 24 to coordinate the procedure and bankruptcy trustees. increase the cooperation with judges from commercial department who are part of the bankruptcy councils; -Improve and strengthen judicial function towards protection of the domestic and foreign investments and cooperation between bankruptcy judges, bankruptcy trusties and creditors; -Promote the concept of sustainable bankruptcy law training for judges and other legal professionals at the national level by including such training as part of the regular Curriculum and activities of the Academy for training of judges and prosecutors of FYR Macedonia. Equipping the Chamber of Complete furnishing of the office Proper working conditions for the Trustees. space for the Chamber of Chamber of Trustees. Bankruptcy Trustees. Provision of Experts in the legal Long-term, in-house Expert input on bankruptcy legal and financial areas. consultancies. procedures in the course of training the bankruptcy trustees and judges. Administrative and Legal support Consultancies. Proper administrative functioning for the Chamber of Trustees. of the Chamber of Trustees. Component 2. Improving Judicial Infrastructure Design and Supervision for the Designs, civil works for Better functionality of the space, renovation of the 11 court renovation of 9 court buildings better working conditions for the buildings. for Basic courts with enhanced court. jurisdiction and furniture for All courtrooms are renovated and renovated space area: equipped with new furniture, -Renovation of the court archives are renovated, access for buildings in Strumica, Tetovo, handicapped people and separate Veles, Struga, Shtip and Ohrid. entrance for employees is - Renovation of the court provided in each court, buildings in Gostivar, Kumanovo replacement of windows and and Kochani . electric lights are energy -Surveillance equipment for the efficiency methods, replacement courts in Gostivar, Kochani and of water supply and sewage Kumanovo. installations and damaged roofs are methods for improving the structure and working conditions. Geotechnical report and revision Designs for court extension, civil Construction on the extension of of the main project design for the works for renovation of existing building is not realized because reconstruction of the court in court building for Basic court licenses are not provided by Bitola; Renovation of the court Bitola with enhanced jurisdiction municipality Bitola, but design building in Bitola. and furniture for renovated space can be useful in future. area. Better working conditions for the Furniture is procured for the court court in Bitola 25 in Bitola. Furniture for the court in Prilep. Complete furniture for the new Better working conditions for the extended part of court building in court. Prilep. Equipment of the representative 3000 m2 court space with courtrooms, offices for judges, offices for court administration, archive and intake hall. Design and construction of a new Designs and civil works for new New energy efficient building court building – Basic Court court building and designs for with 9000 m2 space on 10 floors Skopje 1. renovation of existing building in accordance with needs of 7 Basic Court Skopje1. departments of Criminal Court Design Supervision for Skopje 1. Skopje and 3 separate levels for employees, public and prisons. Protection with video surveillance equipment and control for access. Standards for modern court buildings are implemented during design process. Furniture for the Administrative New office furniture for complete Proper functioning of the Court. equipment. Administrative Court. Furniture for the Judicial Council. New office furniture for complete Proper functioning of the Judicial equipment. Council. Component 3. Enhancing Judicial IT System Hardware, software and Servers, racks, UPSs, PCs, Proper functioning of the additional services for the rollout operating systems, and software Automated Court Case of the ACCMIS: Servers, Racks, for virtualization for servers, Management System, i.e UPS, software licenses software for central management ACCMIS application is of virtual servers, DBMS integrated in the courts’ everyday software, antivirus licenses, user activities. CALs for virtual server operating systems and new DBMS, additional services for proper function of the Information system-for 32 courts, Judicial Council and Ministry of Justice. Hardware, software and Central Backup solution for 34 Proper functioning of the additional services for the rollout servers, connected in VPN with Automated Court Case of the ACCMIS: Central backup the central location (for Supreme Management System, i.e. Solution for Supreme Court Court) and installation and ACCMIS application is adjustment of the software for integrated in the courts’ everyday central backup/restore of data activities. from 34 locations that are connected through IP VPN (MPLS) with the Supreme Court of RM as a central location. Network Equipment for the Switches, network management Upgraded network system in Courts and monitoring software, cabling order to meet the requirements of infrastructure. the new application technologies, i.e. for proper functioning of the ACCMIS (Automated Court Case Management System) and ABMS (Automated Court Budget Management System). 26 Improving the Network Function Connection of 77 judicial Enhancement of the network of the Courts-VPN connectivity facilities in FYR Macedonia on connectivity among courts in RM for 12 months, supply and 46 access points through WAN / for proper functioning of installation of switches and IP VPN network. The scope of ACCMIS (Automated Court Case extended warranty of existing work includes supply, Management System) and ABMS routers. installation, configuration, (Automated Court Budget management, monitoring and Management System). maintenance of VPN lines. Computer hardware for the court Personal computers for the court Electronic registration of registry offices. registry offices to automate the documents. process of registration of documents Computer Hardware for the Personal computers, switches, Proper functioning of the Administrative Court. server, copier. Administrative Court. LAN and telephone system for Telephones, fax, scanner, LCD Proper functioning of the the Administrative Court. projector. Administrative Court. Computer Equipment for the Database Server, File Server, FTP Successful entry, updating and LDBIS-Supreme Court and Server, Web Server, Workstation keeping of all legal documents Judicial Council. PC and Printers. published in the Official Gazette of RM into the Legal Data Base Information System (LDBIS) which is operating successfully and with free public access through the Ministry of Justice website. Office equipment and telephones Telephones, faxing machines, Proper functioning of the main for SAI. scanners office of the State Administrative Inspectorate. Document scanners for the Document scanners. Scanning documents now courts. necessary for proper filing in the ACCMIS (Automated Court Case management System). IT equipment for High Personal computers, printers, Operation of the newly Administrative Court. server. constituted High Administrative court. 27 Annex 3. Economic and Financial Analysis (including assumptions in the analysis) The PAD suggested a number of factors that could be used in measuring the Project’s economic and financial impact. Many of these, however, were –and remain – difficult to tie directly back to the Project objectives. The PAD posited that the Project’s indirect benefits might include more predictable and uniform application of the rule of law, strengthened business confidence, and a more stable investment climate. While these are all noble ideas on the grand scale, the economic crisis that swept through Europe beginning in 2009 and continued through the end of the Project made even casual inferences about the Project’s economic benefits difficult to surmise. Similarly, the PAD also posited that the “improved delivery of judicial services and other Pproject outcomes would help lower the costs of doing business in FYR Macedonia, reduce the cost of, and accelerate litigation, bankruptcy and administrative disputes, increase confidence in, and access to, the judicial system, as well as make more transparent the conduct of judges and other personnel.� Some of these criteria have been met – notably the greater transparency in the conduct of judges and the acceleration of bankruptcy and administrative disputes. Nevertheless, the increasing complexity of the Macedonian justice system and the looming economic crisis in Europe create tremendous externalities that then make it difficult to measure any one factor in the Project. There have been demonstrable economic benefits reaped by the Project’s modest investment in the State Administrative Inspectorate, and with the adoption and implementation of Bankruptcy Trustees. The reduction in dispute resolution time for the SAI through the adoption of an information technology solution has been cut by – on average – at least two weeks of staff time, and has reduced postage fees incurred. Similarly, the Project financing used to create the Bankruptcy Trustee’s Chambers has helped to form the first, fully-fledged Bankruptcy Chambers in all of the Balkans. Certain interactions between judges and the Bankruptcy trustees need to be worked out, but for the most part the trustees now work independently in resolving cases. Using the World Bank 2004 Cost of Doing Business survey and data as a baseline, the following progress can be noted: 28 Table A3.1 Indicator 20047 20128 Time to Complete a 3.7 2.0 Bankruptcy (in years) Cost as a % of the estate 38 10 Recovery Rate (%) 7.9 42.0 As illustrated by this table, the time to complete a bankruptcy case has been nearly halved – from 3.7 years to 2 years, and the cost of a bankruptcy has dramatically decreased (28%). The recovery rate on bankruptcies has also increased dramatically, demonstrating that the bankruptcy trustees are more efficient in recovering assets than the previous system which relied solely on the courts. There is better monitoring of the performance of the courts and the judiciary, as well as better utilization and maintenance of the ICT system. Subsequently, the increased utilization and better monitoring have only highlighted the lack of general capital and human resources to maintain the current systems. For instance, while the ICT system is currently functioning, it is working well beyond capacity. Without a plan to renew or upgrade existing ICT systems (already being taxed) through new and improved hardware, the overall continuity is in danger. Similarly, the judiciary’s focus on hiring judges and overall rigidities in hiring other support staff (e.g., clerks of court, financial officers, ICT specialists) jeopardize the court model that the Bank has financed for the past six years. As a result, some short-term economic and financial benefits accrued by the Project may not be sustainable unless the Judiciary takes great strides to improve its capital planning. The lack of a follow-on Capital Investment program for the court ICT -- beyond the year 2010—leads one to believe that overall financial sustainability is not yet within reach. 7 Doing Business – Benchmarking Business Regulations, World Bank Group (2005), as cited in draft Improving the Business Climate in Macedonia: A Legal and Judicial Enforcement Assessment., World Bank Group (2005). 8 Doing Business – Measuring Business Regulations, World Bank Group (2012), as referenced on August 13, 2012: http://www.doingbusiness.org/data/exploreeconomies/macedonia,-fyr#resolving-insolvency 29 Annex 4. Bank Lending and Implementation Support/Supervision Processes (a) Task Team members Responsibility/ Names Title Unit Specialty Lending Gerhard Botha Sr Financial Sector Spec. ECSPF Olav Rex Christensen Senior Public Finance Specialist HDNED Frederic Gielen Consultant ECCAT Mala D. Johnson Program Assistant ECSPE David Nagy Consultant LCSFM Camille Anne Nuamah Country Manager LCCNI Gurcharan Singh Senior Procurement Specialist TWICT Private Sector Development Jasminka Varnalieva ECSPF Specialist Supervision/ICR Denis Boskovski Country Operations Officer ECCMK Olav Rex Christensen Senior Public Finance Specialist HDNED Aleksandar Crnomarkovic Financial Management Specialist ECSO3 Amy Evans Consultant (Safeguards) ECSS1 Ahmet Gokce Consultant (Procurement) ECSO2 Claire Louise Greer Consultant (Operations) ECSPE Evgenij Najdov Sr Country Economist ECSP2 Philip Gerald Thacker Consultant ECSPE Jasminka Varnalieva Private Sector Development Spe ECSPF (b) Staff Time and Cost Staff Time and Cost (Bank Budget Only) Stage of Project Cycle USD Thousands (including No. of staff weeks travel and consultant costs) Lending FY06 200.85 FY07 0.00 FY08 0.00 Total: 200.85 Supervision/ICR FY06 0.00 FY07 104.71 FY08 97.78 Total: 202.49 30 Annex 5. Beneficiary Survey Results (if any) 31 Annex 6. Stakeholder Workshop Report and Results (if any) 32 Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR No comments received. 33 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders 34 Annex 9. List of Supporting Documents 35