Document of The World Bank Report No: ICR00002885 IMPLEMENTATION COMPLETION AND RESULTS REPORT (IDA-44930 IDA-H4130) ON A CREDIT IN THE AMOUNT OF SDR 2.07 MILLION (US$ 3.7 MILLION EQUIVALENT) AND GRANT IN THE AMOUNT OF SDR 1.03 MILLION (US$ 1.3 MILLION EQUIVALENT) TO MONGOLIA FOR AN ENHANCED JUSTICE SECTOR SERVICES PROJECT December 26, 2013 Justice Reform Unit The Legal Vice-Presidency CURRENCY EQUIVALENTS (Exchange Rate Effective September 30, 2013) Currency Unit = Mongolian Tugriks MNT 1.00 = US$ 0.0006 US$ 1.00 = 1650 MNT FISCAL YEAR January 1 - December 31 ABBREVIATIONS AND ACRONYMS CDEA Court Decision Enforcement Agency CAS Country Assistance Strategy CPS Country Partnership Strategy EA Environment Assessment EJSSP Enhanced Justice Sector Services Project FA Financing Agreement GCC General Council of Courts GIZ Deutsche Gesellschaft ffir Internationale Zusammenarbeit (GIZ) GOM Government of Mongolia IDA International Development Association IDF Institutional Development Fund ICR Implementation Completion Report IPP Indigenous Peoples' Plan ISN Interim Strategy Note ISR Implementation Status Report IT Information Technology LJSP Legal and Judicial Reform Project LIL Learning and Innovation Loan M&E Monitoring and Evaluation MES Ministry of Education and Science MOF Ministry of Finance MOJ Ministry of Justice MTR Mid-Term Review NLI National Legal Institute of Mongolia OP Operational Policy PAD Project Appraisal Document PCN Project Concept Note PDO Project Development Objective PHRD Japan Policy and Human Resources Development PID Project Information Document PIU Project Implementation Unit PSC Project Steering Committee QER Quality Enhancement Review SPA Special Protected Area TOR Terms of Reference UIN Unified Information Network US AID United States Agency for International Development Vice President: Axel van Trotsenburg Country Director: Klaus Rohland Sector Manager: Christina Biebesheimer Project Team Leader: Heike P. Gramckow ICR Team Leader: Heike P. Gramckow MONGOLIA Enhanced Justice Sector Service Project CONTENTS Data Sheet A. Basic Information B. Key Dates C. Ratings Summary D. Sector and Theme Codes E. Bank Staff F. Results Framework Analysis G. Ratings of Project Performance in ISRs H. Restructuring I. Disbursement Graph 1. Project Context, Development Objectives and Design....................... 1 2. Key Factors Affecting Implementation and Outcomes ................ ..... 5 3. Assessment of Outcomes ........................................... 11 4. Assessment of Risk to Development Outcome......................... 17 5. Assessment of Bank and Borrower Performance ............... ......... 18 6. Lessons Learned ........................................ ...... 20 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners ..... 22 Annex 1. Project Costs and Financing ................................. 23 Annex 2. Outputs by Component ............................... ..... 26 Annex 3. Economic and Financial Analysis ................................ 35 Annex 4. Bank Lending and Implementation Support/Supervision Processes ...... 40 Annex 5. Beneficiary Survey Results................................. 42 Annex 6. Stakeholder Workshop Report and Results......................... 43 Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR .... ..... 48 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders . .......... 49 Annex 9. List of Supporting Documents .......................... ..... 50 Annex 10. Project Activities per Component (Original and Revised) . ............ 51 Annex 11. Supporting Documentation ................................. 52 MAP........................................................ 89 A. Basic Information Enhanced Justice Country: Mongolia Project Name: Sector Services Project - Mongolia Project ID: P101446 L/C/TF Number(s): IDA-44930,IDA-H4130 ICR Date: 12/27/2013 ICR Type: Core ICR Lending Instrument: SIL Borrower: MONGOLIA Original Total XDR 3.10M Disbursed Amount: XDR 2.95M Commitment: Revised Amount: XDR 2.95M Environmental Category: B Implementing Agencies: Ministry of Justice and Home Affairs Cofinanciers and Other External Partners: B. Key Dates Revised / Actual Process Date Process Original Date Date(s) Concept Review: 02/14/2007 Effectiveness: 06/16/2009 06/16/2009 04/25/2012 Appraisal: 04/23/2008 Restructuring(s): 06/06/2013 06/06/2013 Approval: 06/26/2008 Mid-term Review: 04/09/2012 04/28/2012 Closing: 12/31/2012 06/30/2013 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Moderately Satisfactory Risk to Development Outcome: Low or Negligible Bank Performance: Moderately Satisfactory Borrower Performance: Moderately Satisfactory [C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Moderately Quality at Entry: nsatisfactory Government: Moderately Satisfactory J)rImplementing Moderately Quality of Supervision: Moderately Satisfactr gecy/Agencies. Unssfctory Overll Bnk KOvel Boroe odertey PeformAgnce: Ustsatr OvrllBn oertl Satisfactory OealBro r Moderately Satisfactory Performance: Performance: i C.3 Quality at Entry and Implementation Performance Indicators Implementation Indicators QAG Assessments Rating Performance (if any) Potential Problem Project Quality at Entry No Nn at any time (Yes/No): (QEA): Problem Project at any Yes Quality of None time (Yes/No): Supervision (QSA): DO rating before Moderately Closing/Inactive status: Unsatisfactory D. Sector and Theme Codes Original Actual Sector Code (as % of total Bank financing) Law and justice 100 100 Theme Code (as % of total Bank financing) Judicial and other dispute resolution mechanisms 40 40 Legal services 20 20 Other rule of law 20 20 Personal and property rights 20 20 E. Bank Staff Positions At ICR At Approval Vice President: Axel van Trotsenburg James W. Adams Country Director: Klaus Rohland David R. Dollar Sector Manager: Christina Biebesheimer Anthony G. Toft Project Team Leader: Heike P. Gramckow Robert Buergenthal ICR Team Leader: Heike P. Gramckow ICR Primary Author: Beth Anne Hoffman F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) The project's development objective is to enhance the efficiency, transparency and accountability of justice sector institutions. This objective will be achieved by assisting the Government of Mongolia to mainstream and scale-up the results achieved, and lessons learned, through the Mongolia Legal and Judicial Reform Project (LJRP). In order to achieve the goals of transparency and accountability the following areas will be supported under the project: (i) Enhancing public legal education on the justice sector (ii) ii Increasing transparency through improved access to legal information (iii) Enhancing judicial operations, enforcement and monitoring of court decisions. The key outcome indicators which will be used to measure success are: (i) Information on enforcement of court decisions is available and shared between the Court Enforcement Agency and the General Council of Courts (ii) Discrepancies between statistical information provided by Court Enforcement Agency and General Council of Courts is shared and reduced (iii) Improvement in level of court decisions enforced. Revised Project Development Objectives (as approved by original approving authority) The objective of the project is to enhance access to justice sector information and legal advice, and increase the efficiency and transparency of the courts and the court enforcement agencies. (a) PDO Indicator(s) Original Target Formally Actual Value Values (from Revised Achieved at approval Target Completion or documents) Values Target Years Indicator 1 IData and statistics on enforcement of court decisions are shared regularly Ibetween the CDEA and GCC. A MoU was signed on May 29, 2013; a working group to t o Both agencies establish data Information on enforcement of submit sharing court decisions is quarterly data requirements has Value No regular sharing of data available and reports to the been created. Exact . . Oher.data to be quantitative or and analytical results shared between the other. Qualitative) between the agencies Court Enforcement Both agencies exchanged will be Agency and the compare established after the General Council o quarterly data new law is in effect and discuss on November 1, analysis. 2013. Both agencies exchange data irregularly. [Date achieved 12/31/2009 12/31/2012 06/30/2013 06/30/2013 The end target is partially met; it will be fully met by the end of the current Comments calendar year. (incl. % Both agencies exchange data irregularly at the moment. Exact data to be achievement) exchanged will be established after the new law is in effect on November 1, 2013. Indicator 2 : Local courts submit electronic case data to the GCC. Value Discrepancies 60% of courts 100% of courts quantitative or No electronic case data between statistical submit submit case data by Qualitative) system in place. information monthly case GCC website. provided by Court statistics 111 Enforcement electronically Agency and to GCC. General Council of Courts is shared and is reduced. Date achieved 12/31/2009 12/31/2012 06/30/2013 06/30/2013 Comments Target fully met. (incl. % With the new software and fiber optic linages, real time case management is achievement) possible. Indicator 3 Court decision enforcement processes are unified and monitored for effective enforcement. 60% of 100% of locations locations using (in all 21 Aimags) Improvement in new system, alu e o No unified system in level of court 10% increase are fully conne ' Quanitativer place. decisions in successful areu t enforced. enforcements system .enforcement data is over previous shared with HQ. year. ____________ [Date achieved 12/31/2009 12/31/2012 06/30/2013 06/30/2013 Comments Target fully met. (incl. % By August 31, 2013 58.9% of the amount to be enforced was collected, in achievement) comparison in 2012 the rate was 40.1%, in 2011 44.2%, in 2010 43.0%. [Access to legal information and advice increased across all UB districts and Indicator 4: : ias _______ ______________ Aimags. 60% of UB districts and Legal advice Aimag centers available via Legal distribute legal Aid Centers in all information in 21 Aimags and 12 No values related central Legal Aid Clinics information community in UB; in 15 out of Value locations, in 21 Aimags (=70%) .paralegals are available, quantitative or 60% of Aimag and in 9 UB Qualitative) centers, soums districts paralegals minority languages and horoos are providing paralegals are services; Kazakh providing legal glossary and advice, 10 14 laws in Kazakh laws available distributed plus in minority legal info. informlanguages. in_2_Aimgs_ad1 Date achieved 12/31/2009 06/30/2013 06/12/2013 Comments Target fully met. Leaflets and posters on different legal issues provided across the country; legal (incl. % comics and posters provided to public schools; "Your Legal Guide" distributed to achievement) all Aimags. iv (b) Intermediate Outcome Indicator(s) Original Target Actual Value __________ _______Values__ __ (from___ - omly Achieved at Indicator Baseline Value Values (from Revised cmplet approval Completion or documents) Target Years Indicator 1: Improved law school syllabi implemented. 17 new syllabi; 19 law schools which 9 new syllabi, meet new national 26 course 8 new syllabi and .certification handbooks 31 syllabi and 14 23 handbooks Value handouts developed standards; 75% of completed; # completed and . professors meet of courses that (quantitative within the previous WB distributed to Law . new national apply new or Qualitative) project but not regularly qul fation sy nd Schools; they are used. qualification syllabi and .sdi ll1 a standards; 70% of handouts us students pass in increased by new standard 5%. courses. Date achieved 12/31/2009 12/31/2009 06/30/2012 06/30/2013 Comments Target met. (ic. % 27 courses at 14 law Schools apply material (total of 378 courses). achievement) 8 syllabi, 23 handbooks completed and used in all 13 law schools; 1 syllabi and 8 handbooks dropped by the client due to changes in the relevant laws. Indicator 2: Development of standards for Legal Clinics implemented. Value # of legal 10 Legal Clinics (quantitative No standards developed. Clinics comply comply with legal or Qualitative) standard standards. increased to 8. Date achieved 12/31/2009 06/30/2013 06/30/2013 Comments (incl. % Target met. achievement) Indicator 3 : Increased access to legal information and legal advice. # increase of 50,000 leaflets (25 publcatons r oherublications; select issues), 4,000 Spublications or otherinfo public education material Improvements in opulation osters for each value, N/A % of levels of public eached by TV the country; legal population reached by TV knowledge: 50% spots; # comics at grade Value spots, 0 # of minority able to increase of schools; "Your (quantitative population served by differentiate inority Legal Guide"/DVD or Qualitative) legal aid center (by institutions and opulation around 7 subjects; gender), 0% of rural coure functions; served by legal411 TV spots in 17 population with access to 50% aware of aid center; 5% Aimags + Capital paralegals, # of rural court divisions. increase of population actually ral Ctic total ie of98 opulation Ethnic mioris. ppuith access to 10,474 akhs v paralegals; # increase of rural population actually served ________________ by paralegal Date achieved 12/31/2009 12/31/2012 06/30/2013 06/30/2013 357 paralegals in 15 rural Aimags (pop of 1,086,382) provided advice to 130,331 (incl. % people, i.e. 12% of the rural population. achievement) Target met. Data on TV spots outstanding. Indicator 4: Connect courts, CDEA, and prosecutors via fiber optic cable. 21 Aimag courts, 4 Inter-Soum courts, 100% of courts, 21 CDEA Aimag Value 0 courts, CDEAs, 100% of court 'Cs, offices, 3 CDEA enforcement CDA, Soum offices, 21 (quantitative prosecutor offices officecnte prosecutor Admnistrie .rQaiaie)cnetd offices connected ofcs Administrative or Qualitative) connected. tnaialfbr offices to national fiber Courts, and 9 optic network. locations in UB are connected, i.e. total of 79 locations. Date achieved 12/31/2009 12/31/2012 06/30/2013 06/30/2013 Comments (mm. t Target met. ac. t Prosecutors opted out of the project. achievement) Indicator 5: Improvement of court enforcement agency facilities. Value enforcement Construction of all (quantitative 0 CDEAs newly built. 3 3 CDEA offices is or Qualitative) agency offices completed. renovated. Date achieved 12/31/2009 12/31/2012 06/30/2013 06/30/2013 Comments (incl. % Target met. achievement) Indicator 6: Improvement of court facilities. Value Supreme Court Construction of 4 (quantitative 0 courts newly built or facilities new court buildings . reurbihed.and 1 refurbishment or Qualitative) refurbished. constructed. .I furhd. is fished. Date achieved 12/31/2009 12/31/2012 06/30/2013 06/30/2013 Comments (incl. % Target met. achievement) Indicator 7: Development of case management software for administrative courts. Value TSoftware is 0 administrative courts successfully . using new software. piloting in 5 or Qualitative) aplinis iadministrative Vi courts; full launch on November 1, 2013. Date achieved 12/31/2009 06/30/2013 06/30/2013 Comments (incl. % Target met. achievement) Indicator 8: Training of CDEA staff All CDEA officers 40% increase in trained on current Value 0% of CDEA officers ttlcutaducmn (quantitative trained (in new enforcemet 5% increase. and soft .rQaiaie rceig) enforcement legislation and soft or Qualitative) proceedings). reep. kls nldn receipts. skills, including ethics. IDate achieved 12/31/2009 12/31/2012 06/30/2013 06/30/2013 Comments (incl. % Target met. achievement) G. Ratings of Project Performance in ISRs Actual Date ISR Ata No. Ate DO IP Disbursements (USD millions) 1 01/23/2009 Satisfactory Satisfactory 0.00 Moderately 2 06/22/2010 Unsatisfactory MoUnsatcy 0.50 Unsatisfactory 3 02/6/2Moderately Moderately 0.65 Unsatisfactory Unsatisfactory 4 1Moderately Moderately Unsatisfactory Unsatisfactory 5 07/01/2012 Moderately Satisfactory Moderately Satisfactory 2.55 6 01/02/2013 ModeratelyModerately Satisfactory 3.41 Unsatisfactory 7 06/25/2013 Moderately Moderately 4.10 Unsatisfactory Unsatisfactory H. Restructuring (if any) ISR Ratings at Amount Restructuring Board Restructuring Disbursed at Reason for Restructuring & Date(s) Approved Restructuring Key Changes Made PDO Change DO IP in USD millions The primary reason for the 04/25/2012 N MU MU 2.55 restructuring was to drop, at the Government's request, the vii ISR Ratings at Amount Restructuring Board Restructuring Disbursed.at Reason for Restructuring & Date(s) Approved Restructuring Key Changes Made PDO Change DO IP in USD millions construction of the Supreme Court building and co-located central judicial archives (representing 43% of total project costs) as well as the downscaling of construction of Court Decision Enforcement Agency buildings (from 11 to 3). Funds intended for the Supreme Court building are reallocated for other priority civil works at local courts in Aimag centers and at the Soum level. This Level 2 Restructuring reflected a reallocation request by GoM to address a funding shortfall due to exchange rate 06/06/2013 N MU MS 4.10 fluctuations, and minor cost variances in the civil works categories and to ensure that all activities can be fully funded at project closing. I. Disbursement Profile Original ---- Formally Revised -- Actual 6- 4 Ln/ 2 ------------------- 0 - 1 N 1 i IN I IN IN IN IN IN IN IN IN m 9 rj 9 m rq 9 m 9 1. Project Context, Development Objectives and Design 1.1 Context at Appraisal Country and Sector Background. The dissolution of the Soviet Union in the 1990s ushered in a period of rapid change in Mongolia's justice sector. A new constitution and a flurry of new laws were subsequently enacted in Mongolia. It is estimated that over 400 laws and 3000 administrative regulations were passed or issued during the period 1992 to 2008 as the country moved toward a market economy.' While still considered one of the poorest countries in East Asia at the time of project design, Mongolia's economy had been growing rapidly driven by the demand for commodities and minerals on the international market. The project was billed as a potential means to curb some of the negative effects of a "natural resource curse" by improving the pool of highly skilled professionals and enhancing the operations of administrative and justice sector institutions. The Government of Mongolia (GOM) had embraced a comprehensive approach to justice sector reform and modernization by the late 1990s. The GOM received support for its modernization efforts from a number of donors, including USAID, JICA and GTZ (GIZ). The World Bank-financed Legal and Judicial Reform Project (LJSP) (Credit 3595-MOG) -- a Learning and Innovation Loan (LIL) which was approved in 2001 for US$5.0 million equivalent - aimed to enhance public trust and confidence in the legal system as a whole and the judiciary in particular through the design and testing of new tools and systems that promote better access to legal information, the creation of specialized courts and an improved legal education and profession. Rationale for Bank Assistance. The EJSSP was built upon a long standing discussion between the Government of Mongolia and the Bank on good governance and the justice sector's importance to the country's economic development. According to the Project Appraisal Document (PAD), the project was to build on the successes of the innovative approaches piloted under the LJRP, particularly as they related to transparency, good governance, the modernization of public legal education and the protection of citizens' rights and reflected the ongoing need for and government interest in Bank support in this sector. 1.2 Original Project Development Objectives (PDO) and Key Indicators (as approved) The project's original development objective, as stated in the original Financing Agreement (FA), was "to enhance the efficiency, transparency and accountability of the Recipient's justice sector institutions." In order to achieve this objective three components were supported under the project: (i) Enhancing public legal education on the justice sector, (ii) Increasing transparency through improved access to legal information, and (iii) Enhancing judicial operations, enforcement and monitoring of court decisions. The key Project Development Outcome indicators chosen to measure success were: 1 Enhanced Justice Sector Services Project. Project Appraisal Document, [World Bank Report Number 44056-MOG,] June 6, 2008, pg. 1. 1 * information on enforcement of court decisions is available and shared between the Court Decision Enforcement Agency (CDEA) and the General Council of Courts (GCC); * discrepancies between statistical information provided by CDEA and GCC is shared and reduced, and; * improvement in level of court decisions enforced. In addition, the PAD included fourteen intermediate results indicators (see Annex 10). 1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification A Level 1 restructuring was initiated in late 2010 and approved in April 2012. It was justified by the following events: * While the project was initially to be co-funded by a PHRD grant, the grant did not materialize and created a financing gap of $1 million; * The project incurred a one-year delay in effectiveness; * The construction site proposed by the GOM for the Supreme Court building would not meet the World Bank safeguards policy on Natural Habitats (see section 2.4); and, * Construction costs skyrocketed during the period of project implementation due to the boom of the sector in Mongolia. The PDO was revised to the following: "The objective of the Project is to enhance access to justice sector information and legal advice, and increase the efficiency and transparency of the courts and the Court Enforcement Agencies." The Results Monitoring Framework was also revised to match with the new PDO, with the following PDO indicators: * data and statistics on enforcement of court decisions is shared regularly between the CDEA and the GCC; * local courts submit electronic cases data regularly to the GCC; * CDEA processes are unified and monitored for more effective enforcement; * access to legal information and advice is increased across all UB districts and Aimags (provinces). The now 12 revised intermediate results indicators and their achievements are described in Annex 2. 1.4 Main Beneficiaries The project's beneficiaries as defined in the original Project Information Document (PID) were: * the Judiciary, the Supreme Court, the GCC, and the Executive Branch, through MOJ and its specialized agencies were the project's main institutional beneficiaries; * Justice sector officials and law school professors were also seen as direct beneficiaries of the project. * In addition, Mongolia's citizens in both urban and rural areas also stood to directly benefit from the project's legal information campaigns. 2 The PAD also highlighted the Kazakh ethnic minority living in the Bayan Ulgii Aimag as one of the direct beneficiaries. The restructuring did not modify primary target groups. 1.5 Original Components (as approved) The original Project components were: I. Enhancing Public Legal Education in the Justice Sector (US$250,000 Grant, plus US$250,000 PHRD): The original design of Component 1 included technical assistance for the development of comprehensive legal education reform program and the implementation of a national public information strategy, complementing the work of other donors and NGOs as a means to enhance the transparency and accountability of justice sector institutions. In addition, this component was to support efforts to reform national legal education standards, certification programs and licensing practices. Finally, this component was also to support the development and implementation of a special bilingual legal information program for the Kazakh speaking population of the Bayan- Ulgii Aimag. II. Increasing transparency through Improved Access to Legal Information (US$1.26 million Grant, plus US$150,000 PHRD): The original objective of this component was to improve access to legal information by implementing national justice sector information and networking strategies for both the Supreme Court and MOJ's CDEA. The proposed project activities included the development of a fiber optic network to improve connectively between MOJ and the judiciary, in particular the GCC, all Aimag courts, and all CDEAs offices. These networking activities drew from the successes of the Unified Information Network (UIN)-created under the LJRP, that provided justice sector agencies in Ulaanbaatar for the first time with ready access to important legal information such as laws, administrative regulations, Supreme Court decisions and constitutional court decisions. The EJSSP proposed to work to integrate court case management systems nationwide, linking Mongolia's regional Aimag courts with the Supreme Court and providing real-time monitoring tools for better court management and transparency. The objective was to provide more efficient planning and monitoring systems and improve overall efficiency and transparency of the courts. To ensure effective data sharing a justice information strategy, data sharing agreements and capacity building to ensure proper use of the network were planned. Another activity was the creation of a centralized judicial archive and public information system. III. Enhancing Judicial Operations, Enforcement and Monitoring of Court Decisions (US$3.34 million Grant, US$0.45 PHRD, US$1m GOM): This component had two main activities centered around: (a) strengthening and modernization of the CDEA; (b) the construction of a new Supreme Court. The planned activities were to move the CDEA from manual operations to electronic operations, with databases, systems, and processes to respond and effectively enforce judgments, as well as address complaints concerning enforcement. The project was to finance site surveys and the refurbishment or construction of (10) select rural Aimag CDEA offices. Additionally, the component was to support the creation of a nationwide network to assist the Agency to 3 effectively document, enforce and monitor court decisions. The project was also to provide adequate and modern tools and equipment for the CDEA network to function properly. Lastly, the Project was to provide training for CDEA staff on the new systems and tools, as well as other relevant topics. The main aim of the training was to improve practices and procedures to be carried out by the CDEA staff, based on a study the project would conduct to identify improvement methods. The second subcomponent included the construction of a new Supreme Court facility, with new court rooms and court space, a functional IT network linked to regional courts, a public space for viewing of public records. The Supreme Court facility was to be built to international standards and include the creation of a national court archive. IV. Project Management (US$150,000 Grant, plus US$100,000 PHRD): Under this component the Project Implementation Unit (PIU) established under the previous project would be strengthened to enable it to ensure quality implementation, management, monitoring and supervision of all proposed Project activities. The PIU was to be housed in the MOJ. Specialized PIU staff (consultants) were to provide the required expertise, support, and management of proposed project activities. The PIU was also to manage monitoring and evaluation activities including the preparation and distribution of both an interim and full baseline survey. The Project was to also contract a private auditing firm to undertake the yearly project financial audits. The MOJ, as implementing agencies was to provide physical and logistical support, a multi-stakeholder Project Steering Committee was to provide overall guidance, support and coordination for effective implementation. 1.6 Revised Components See section 1.7 1.7 Other significant changes The 2012 Level I restructuring did not modify the four components but implied a change in the scope and scale of the components as follows: * The construction of a new Supreme Court building and judicial archives was dropped and instead, it was agreed to finance the construction of four courts and refurbishment of one. The number of CDEA offices to be financed out of the project was reduced from 10 to three. All of the proposed construction sites were outside the capital, thereby addressing the GOM's access to justice and service delivery concerns. * The amount allocated for associated goods (i.e. construction) was reduced, allowing for a reallocation of credit funds towards consulting services and training which would have been funded under the PHRD Grant. Funds were re-allocated, and the project closing date was extended by six months, from December 31, 2012 to June 30, 2013. Since the construction activities changed, the project safeguard category also changed from C to B. These changes are depicted in detail in Annex 10. In June 2013, a minor reallocation of funds was processed. It addressed an adverse drop in the exchange rate, and slight cost differences in the civil works category. 4 The reallocations were in line with the project's priorities and did not reflect any substantive changes to the project's scope or activities. 2. Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design and Quality at Entry Soundness of the Background Analysis. The experience of the prior justice project LJRP, of an Institutional Development Fund (IDF) Grant for the Administrative Courts, and of work of other donors such as USAID confirmed the need for this project to invest in expanding public legal information and improving legal education activities. Those projects had also demonstrated the potential for using IT solutions as a cost effective way to bring greater transparency and efficiency to judicial operations. The envisioned components were relevant to the country context and requested by the GOM and other stakeholders. Assessment of Project Design. Building upon prior project experience the Bank continued support for much needed court infrastructure, networking, as well as public legal education and law school reform. The addition of first-time support for the court enforcement agency provided for a more sector wide approach, rare in justice sector projects. Nevertheless, there were issues with the project design. In particular, questions were raised by peer reviewers during the QER and decision meeting about: (i) alternative options to constructing a Supreme Court building and instead delivering justice services to remote population through mobile courts; (ii) the history of cost overruns and delays in judicial infrastructure components in other projects; and (iii) the reliance on the PHRD grant to finance all public legal information activities and training. Implementation showed that those questions had not been sufficiently addressed in the original design. The public and legal education components, on the other hand were based on good concepts developed in the earlier projects, engaging relevant stakeholders with a long term vision of creating sustainable capacities for reforms. Government Commitment. The previous project's ICR had noted that future projects should emphasize greater involvement from GOM. The Mongolian justice sector was highly committed to the project, its objectives and activities during preparation. The Supreme Court was committed to developing a new building and archive; the CDEA was enthusiastic about receiving international assistance for the first time ever. A Project Steering Committee (PSC) was to be established in order to oversee all project activities as part of project implementation. The great commitment of the MOJ to maintain the scope of Component 1 during restructuring and its own contributions as well as the strong engagement of the law school community is reflected in the responsive design. A Government change following Board approval of the project in part explains the delay in effectiveness. GOM commitment, however, remained strong. Readiness of implementation. In hindsight, the project was not fully ready for implementation at the time of approval. As a result, critical pieces of project preparation slipped into implementation and caused delays. In particular, site specific surveys for the 5 construction, technical specifications for the fiber optic network, and preparation for the CDEA software were not sufficiently advanced at the time of appraisal. The timetable for implementation was overly optimistic, as the bulk of the activities were to be implemented in year two. Construction costs were underestimated, though it would have been hard to project the increase in construction costs that Mongolia experienced since the late 2000s. Several conditions of effectiveness and disbursement were put in place, also suggesting that the project was not fully ready for implementation by the time of project negotiations. One condition pertained to the approval of the PHRD Grant although neither the Bank nor the GOM had any leverage on the outcome. This condition and others were ultimately waived during effectiveness. Expected construction sites were not analyzed in detail during project preparation, which meant that a series of operational and environmental issues had to be handled during project implementation. (See further discussion in Section 2.4). Assessment of Risks. The PAD identified some generalized risks to the Project Development Objective. The component risks and their ratings did not take into account the above mentioned issues raised during the QER meeting. Again in hindsight, additional risks could have been identified and addressed better at preparation time: * although training and technical assistance were essential to the project achieving its PDO, these were mostly to be funded from the PHRD grant creating an upfront risk; * the lack of site specific surveys for the proposed construction and procurement preparation meant that there was a high risk of disbursement delays; * the project costs included minimal cost and price contingencies. 2.2 Implementation Despite many challenges and start-up delays, the GOM delivered a holistic reform project in de facto 2 V2 years, beginning with the initiation of the restructuring with a reduced scope of construction. This included delivery of 4 new court buildings and refurbishment of one, construction of three CDEA buildings, linking all courts and CDEA offices across the country via fiber optic cable, reform of law school education, country-wide education programs and increased access to legal information for all and special target groups and legal assistance for the poor across the country. There were a number of factors outside the control of Government or the implementing agencies that influenced project implementation and ultimately project outcomes: * Project ratification by Parliament and thus effectiveness was delayed by one year due to elections, by which point the material costs for construction had already risen dramatically. * The PHRD Grant, which was expected to fund technical assistance and M&E activities, was not approved. Without timely technical assistance, the pace of implementation slowed. This financing shortfall, along with skyrocketing construction costs limited the project's ability to progress towards achieving its development objective and led to the restructuring. 6 Otherfactors subject to Government control influenced project implementation: * The lack of detailed analysis of expected construction sites during project preparation meant that a series of operational and environmental issues had to be handled during project implementation and led to delays. It took until 2011 for the GOM to meet all the disbursement conditions for the construction of the CDEA offices. The construction of the new Supreme Court building also proved problematic as it turned out that the site selected by GOM was in Bogd Khan Uul, a Special Protected Area (SPA) and a UNESCO biosphere reserve. Construction would have been in violation of the Bank's Natural Habitat policy (OP 4.04). It took a year of discussions with GOM for them to agree to drop its request to finance this building. The GCC asked that in lieu the project finance four court buildings and refurbish another. * The staffing of the PIU was a chronic issue cited for improvement in multiple project Aide-Memoires throughout most of project implementation. In practical terms, the PIU was not equipped to implement the complex project as designed. In the long interim between Board Approval and effectiveness, all but one of the prior PIU staff left because they were not paid. From this point on in implementation, key hires for technical positions were delayed, underfilled or never filled. The PIU was reasonably staffed to implement the public legal education and information component, but the lack of specialized staff to manage the day-to-day aspects of the construction and fiber optics created problems that adversely impacted implementation. A number of recommendations were made to the GOM to improve the PIU's staffing and performance, but few concrete actions were taken. On the Bank's side, the rapid succession of Task Team Leaders in the initial phase of the project did not facilitate the engagement with GOM nor tight supervision. By the time the 4th Task Team Leader took over in 2010, the project was rated a project at risk. Thanks to the high level of familiarity of the TTL with judicial reforms, including in Mongolia, the Bank team was able to intensify supervision, have a frank dialogue with the Borrower about the delays and implementation problems, and agree on an action plan with the GOM to move the project forward while the restructuring was finalized. The dialogue, past experience, and the Bank's flexibility and responsiveness throughout the lengthy restructuring mobilized counterpart engagement and ensured that the project activities were implemented, and the redesigned M&E framework was realistic given the remaining time for implementation. Strong supervision by the Bank in the last two years of the project and greater counterpart engagement ensured that the project activities remained largely on track. The discussions and decisions made leading up to the restructuring, including agreement on interim procurement plans helped to speed up project implementation by 2011 before the restructuring was finalized. Once the site issues were worked out with the GCC and CDEA, and after overcoming many procurement challenges, the key construction contracts were signed two years behind the original schedule, in late 2011. By the time of the Mid-Term Review (MTR), most components of the project had progressed well but it was clear that attention to timely implementation could not falter again. The Bank team's use of frequent videoconferences in the period leading up to the restructuring and through the implementation was critical in resolving issues and for 7 keeping tabs on implementation progress. Intensified supervision, specific action plans, more regular reporting, greater engagement of the steering committee and an enhanced role for the MOJ in project supervision had a positive effect on project implementation. 2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization M&E Design. The original design of the M&E was overly complex and there was a disconnect between the PDO, interim indicators, and envisioned project activities. The Financing Agreement required that the baseline survey be completed by Project Effectiveness; some data were collected but only in relation to the construction of CDEA buildings and to law school education and many were not relevant to the PDOs or interim indicators; as a result, this condition was not met. M&E implementation. The project's M&E was almost consistently rated moderately unsatisfactory during implementation. The original project design had placed the funding for the M&E efforts solely within the PHRD Grant. When the grant was not approved, the GOM and Bank team had to rethink how to use existing sources to produce a meaningful assessment of project impact. The PIU began collection of M&E data in earnest only in 2010. Following restructuring, the M&E framework focused on data to be collected as part of implementing agency or counterpart activities to ease the burden of data collection and develop related capacities in country. This approach in the end garnered a good set of output and even some impact data and created awareness of the importance of gathering performance data among stakeholders. However, the results of these efforts came late, not just because implementation was often delayed but due to lack of dedicated PIU staff to handle and coordinate the needed M&E activities. Data collection and analysis was mainly progressing during supervision missions and with considerable input and guidance from the Bank. The lack of specialized staff within the PIU and options to address this was discussed with the MOJ frequently but continued to be an issue; it was only in the final four months of the project that an M&E specialist was hired. M&E utilization. M&E utilization varied for this project but demonstrated that when data was available, it could be used to demonstrate the effectiveness of the activities and gain support for future funding. For example, under Component 1, the project implemented a number of cost-effective and innovative approaches to public legal education centered around nine societal values. The subject matter for the various campaign activities, such as TV and radio spots, special events, pamphlets and posters were developed based on stakeholder feedback, including the National Legal Institute's (NLI) helpline and often developed in coordination with other relevant ministries (i.e., environment, health, education, etc.). Unfortunately, there was no system in place to measure the impact of these activities - leaving only output data, some of which at least provide support for overall impact assumptions. These are currently used by the MOJ to develop subsequent campaigns. The project's public legal education activities for the Kazakh and Mongolian minorities in the Western region included data collection requirements related to services provided and populations served by gender. The data demonstrate a high need for information and advice being met at low cost. This convinced the MOJ subsequently to maintain these activities after the project closed. The greatest use of survey results and service delivery data was made by the law schools. The satisfaction rates with training events and publications informed further 8 activities and the number of clients served as part of the legal clinic efforts in the ger districts as well as results from the work with high school students led the participating law schools to continue these activities after the projects closed. Due to implementation delays, two major project activities - court construction and the fiber optic connections - were only completed at project closure and related data collection naturally lagged behind. This limited the extent to which a greater analysis of project outcomes could take place. Nevertheless, surveys of staff and user satisfaction with the new buildings were conducted in 2012 when the first buildings were completed and again at the end of the project in August 2013. Both survey cycles indicate improvements in staff and user satisfaction. It will take more time to measure the impact of increased data exchanges on the performance of agencies (GCC and CDEA) as IT systems became operational at the very end of the project. Anecdotal evidence so far indicates important improvements and cost savings but more data would have to be collected a year from now to demonstrate the impact more rigorously. 2.4 Safeguard and Fiduciary Compliance Safeguards. The Bank's Safeguard Policies on Environmental Assessment (OP 4.01) and Indigenous Peoples (OP 4.10) applied to the project. Environmental Safeguards. OP 4.01 was triggered under the initial design. The initial screening of the project activities at the Project Concept Note (PCN) stage was based on design information available at that time. The PCN stated that the project would focus on the refurbishment or construction of a few CDEA buildings and that "potential environmental impacts will be highly localized in Aimag locations". Accordingly, the project was initially classified as a Category "C". The PAD later included support for construction of a new Supreme Court building and about 10 smaller buildings for the CDEA and proposed that the EJSSP use the same environmental guidelines as the LJSP, modifying any bidding documents for works, especially as it related to environmental mitigation measures on-site as a means to comply with the Bank's safeguards policies. A basic site selection criterion was established in June 2009. The GOM agreed to donate unoccupied public lands for construction. The public land selected by the GOM for the construction of a Supreme Court Building was located in a UNESCO biosphere reserve; construction on that site would have been in violation of the Bank's Safeguards Policy on Natural Habitats (OP 4.04). This activity was dropped with the restructuring April 2012. As of 2010, talks began with the GOM to instead support the construction of smaller, regional (Aimag) courts. With the support of a Bank safeguards specialist, an Environmental Safeguard Guideline for Small Civil Works was developed and adopted in June 2011. By October 2011, additional siting problems had arisen, as site preparations at one of the new court sites detected that it was situated in a river basin2 and additional delays were incurred as a suitable site had to be found. The restructured project complied with the Environmental Safeguard Guideline, and all Bank policies. Social Safeguards. The project design gave considerable weight to social safeguards. Component 1 (Legal Education) included special activities to ensure the Kazakh minority's full participation in the project. Specific legal outreach activities were 2 Project Aide-Memoire, October 1-11, 2011, pg. 2. 9 designed for Bayan Ulgii Aimag, and a simplified Indigenous Peoples' Plan (IPP) was developed and put in place. In applying the Indigenous Peoples' Policy (OP 4.10), the IPP was drafted to extend basic principles established in the LJRP through awareness campaigns for the Kazakh language speaking population of Bayan Ulgii Aimag. A social assessment and local consultations were undertaken, and the project activities were designed with these results in mind. Fiduciary: Financial Management. The overall Financial Management risk was deemed high at project appraisal. To address this risk the PAD contained recommendations for holding the Special Account in a Commercial Bank and for private auditors to undertake project financial audits. There were no issues related to the project's financial management during implementation; financial management reports were provided to the Bank in a timely manner, all audit reports were clean, and any recommendations made by the Bank to improve the Financial Management systems were promptly addressed. Procurement. Operational aspects of project procurement were almost consistently rated unsatisfactory in the Bank's Implementation Status Reports (ISRs). By the time project implementation started most staff from the prior PIU had left, and along with changes at the MOJ most of the prior capacities to handle Bank procurement were no longer available. Project procurement plans were often prepared several months late, requiring several revisions until final approval was possible. Procurement requests and bidding documents were often sent to the Bank with insufficient information and errors; regularly requiring additional guidance from the Bank team and several revisions before the Bank's final no objection could be issued. These procurement delays particularly plagued the construction activities and the development of the Fiber Optic Network. As a result, key implementation activities could only be completed late in the project leaving little room to demonstrate actual impact. 2.5 Post-completion Operation/Next Phase Project design and ongoing communications during implementation aimed at the sustainability of all activities. Regarding Component 1, the MOJ's mandate includes public legal education and it has pledged to continue the related activities with its own budget. The project provided the MOJ with a variety of models (e.g., website, paralegals, public information campaigns) and activity costs to draw from. Concrete plans have been made to ensure that the NLI carries out future public legal education efforts, maintaining the two upgraded websites financed by the project and working directly with legal officers in all Aimags and within Ulaanbaatar districts to disseminate information. The MOJ will also continue the legal aid centers for the Kazakh and Mongolian minorities in Mongolia's western region, though the focus will likely shift to legal aid in criminal cases. The improvements in the legal education system at the university level are well institutionalized. Education standards for legal clinics have become law and, as the project focused on creating standard curricula, the needed human resources to apply and advance the efforts, and strengthened the professional network to continue reforms, the payoff should be long term. The model created to modernize law school curriculum and professors' skills has been cited by the Ministry of Education and Sciences (MES) as a best practice for higher education, and is likely to be sustained over the long term. 10 Related to Components 2 and 3, all future maintenance and operating expenses for facilities and IT networking systems improved through the project will be incorporated in future budget allocations to the MOJ, the judiciary and the CDEA. The firms that developed the Administrative Court and CDEA case management systems will remain under contract for technical support and any minor debugging through the end of June 2014. No follow-on activities for the justice sector have been prepared by the Bank. 3. Assessment of Outcomes 3.1 Relevance of Objectives, Design and Implementation Given the importance of justice to good governance, international trade and economic development in Mongolia, the project received attention from the GOM throughout implementation. Continued reform and support to the justice sector remain key priorities for the GOM. The project also remained highly relevant to the Bank's assistance to Mongolia. Weak governance, as a risk to sustainable growth and an impediment to the effective delivery of services by government to businesses and citizens, in particular vulnerable citizens, has been a core theme of the Bank's engagement in Mongolia since 2005, starting with the Country Assistance Strategy (CAS) for FY2005-9, then the Interim Strategy Note (ISN)3 of 2009-2010 and now the Country Partnership Strategy (CPS) for the period FY2013-2017. The CPS notes that the justice sector is facing issues related to lacking capacities of judges and prosecutors, increasing allegations of corruption, lack of access to justice in rural areas, and limited public trust in the justice sector. In addition, lack of access to relevant sector information is at the core of ineffective management and opacity of justice sector operations. Subsequently, CPS Outcome 3.2 lists the Justice Sector as one of the key delivery partners in public services. The investment in the sector's ICT infrastructure and information sharing capacities furthermore supports other continuing government and Bank priorities for supporting the creation of e-government solutions and achieving greater transparency. 3.2 Achievement of Project Development Objectives Since the project underwent a first order restructuring in 2012, the ICR will assess the project outcome "before" and "after" restructuring. Achievement of PDO before restructuring Rating: Unsatisfactory As noted above, the project's progress towards meeting the PDO was impacted by design issues, limited preparedness at appraisal and lack of progress in implementation until April 2010, when discussions about restructuring began. At that time most project activities had not started, actual disbursement was at 1.5% (US$78,000) and no detectable progress towards the original PDO indicators could be established. Once the restructuring was underway, the Bank and the GOM agreed on interim work plans, better defined active implementation roles for the PIU, the PSC and Report Number 48311-MN 11 stakeholders to speed up implementation even before restructuring was completed. As a result, disbursement increased to 44.2% (US$2,211,000.00) in April 2012 when restructuring was approved. These positive developments were largely due to increased construction activities and progress in the legal education component. The original PDOs, however, depended on progress in ICT capacities which required that construction and networking activities were completed. Accordingly progress towards the PDO was limited at the time the restructuring became effective. Achievement of PDO after restructuring Rating: Moderately Satisfactory The project closed having implemented all of the agreed activities and meeting all of its PDO indicators. For this reason, the PDO rating after restructuring is moderately satisfactory. Following restructuring and a period during which progress became more visible, the rating was increased from Moderately Unsatisfactory to Moderately Satisfactory. At the time of the last ISR, as final delivery of several components and collection of final M&E data were pending, it was agreed to rate the project as moderately unsatisfactory with the understanding that the rating could be adjusted upwards if all outputs were delivered in time and data demonstrating outcomes were provided at ICR time. The PDO (to enhance access to justice sector information and legal advice, and increase the efficiency and transparency of the courts and the CDEA) was measured by four indicators: L Data and statistics on enforcement of court decisions is shared regularly between the CDEA and the GCC. This target has been partially met. At the start of the project no data exchanges occurred between the courts and the CDEA which delayed the start of enforcement processes and also meant that the court did not have sufficient information to understand if and how their decisions could be enforced. Effective data sharing required that data definitions and sharing requirements were agreed between the courts and the CDEA. Electronic data collection and sharing is another element for efficient and low error data exchanges. While data could have been shared manually, only the courts had an effective way to gather timely information from all locations via their website. As a result, the development and implementation of the CDEA software and fiber optic connectivity had first to be in place before effective data sharing could be achieved. With delays in those activities, data sharing agreements were also delayed. A data sharing MoU was signed on May 29, 2013 and a working group to establish data sharing requirements has been created to develop the needed exchange standards. Exact data to be exchanged will be established after a new law that requires that all judges monitor enforcement rates for their cases comes into effect on November 1, 2013. Due to the delays in automation, this indicator target was only partially met by the end of the project. By early December 2013 initial data sharing agreements were in place and data exchanges were piloted between the GCC and CDEA. The combined effect will be improved capacities to monitor court and CDEA performance, eventually leading to greater efficiency in agency performance. IL Local courts submit electronic case data regularly to the GCC. This target has been met. At the beginning of the project, data from the local courts were sent in hard copy on a monthly basis to the GCC. The reports were not always received in time and provided 12 the GCC with limited information to monitor the performance of individual courts. When the project supported improvements to the UIN, local courts across the country were able to upload case data electronically at least on a monthly basis. The fiber optic connection and case management software supported by the project now also provide real time case status information for effective performance monitoring at least for the administrative courts. The positive first experiences with this software prompted the civil case working group, which was supported by the GIZ, to halt its efforts and reflect the substantially greater capacities of the administrative case software in their own design which should come online in early 2014. III. CDEA processes are unified and monitored for more effective enforcement. This target has been met. At the beginning of this project enforcement processes were applied differently across the country, leading to inefficiencies and impeding enforcement efforts. Lack of reliable enforcement information made monitoring of activities and timely identification of problems difficult. The automation now forces the uniform application of required processes and, combined with the fiber optic connectivity now provides for effective monitoring of cases. All CDEA offices are submitting weekly or even daily (in UB) data. Since the system only came online towards the end of the project, impact information is not yet available. Staff report time savings due to reduced data entry requirements, electronic access to registry databases and automated notification processes. Training of CDEA officers supported by the project, investments made by the agency itself and a greater focus on effective service delivery seem to be reflected in improved enforcement outcomes. By August 31, 2013 58.9% of the amounts to be enforced was collected, in comparison in 2012 when the rate was 40.10%, or in 2011 44.2%, or in 2010 43.0%. IV. Access to legal information and advice is increased across all UB districts and Aimags (provinces). As outlined in more detail in Annex 2 and 3, the project was successful in enhancing access to justice sector information and legal advice. Legal information campaigns were implemented throughout the country using multiple means and media in accordance with a targeted outreach strategy. Options for sustainable legal advice were created for poor and vulnerable groups, and addressed a growing need. Modernization of law school education took place, which will serve the sector well into the future. The project collected good output data for all components supporting PDO and interim PDO indicator achievement, but data to demonstrate that the project had impacts beyond the PDO indicators, such as actual increase in court and CDEA efficiency require that the supported reforms are operational for several months, ideally at least a year. To establish impact, follow-up data collection would be needed beyond the duration of the project. Given the fact that most of the project activities are institutionalized and that there is great ownership of the reforms among the project's many stakeholders, it is very likely that more fundamental impacts will be achieved. Longer term monitoring and evaluation would be needed to confirm this trend. 3.3 Efficiency Measuring the efficiency of this project is difficult, not just because there was little related data collected for the PAD but rather because economic rate of return for this kind of investments is difficult to calculate - what is the price of justice? 13 A few efficiency estimates exist. First, after restructuring the project conducted a cost-effectiveness study of different options for connectivity that informed the choice for the fiber optic connection and the service provider. Other initial efficiency assumptions and results include the following: The expectation that the fiscal cost of the investments in ICT would be offset by improvements in the efficiency of operations in the selected courts and CDEA offices has been initially proven correct: * The administrative court software almost eliminated procedural errors which in the past had led to approximately 38% of all cases going to appeal. The CDEA software also reduced errors and streamlined processes translating into more time for enforcement actions; and availability of all enforcement information to managers eliminates the need to recall enforcement officers from enforcement actions in the often distant soums. First data through end- September 2013 show that CDEA offices registered a 59% enforcement rate for CY2013, compared to 40% for all of 2012. The agency also recorded a fourfold increase in the total value of cases enforced compared to the previous 3 years. Additional cost efficiency information for connecting 29 CDEA facilities and 26 courts in all Aimags and UB, developing and providing the case management software and some IT equipment, is outlined in more detail in Annex 3. * The total investment for all activities supporting reform of law school education equated to US$38.50 per student enrolled in the 2013/14 school year - a small amount considering that the same investments already benefitted earlier semesters and will continue to serve students in the future. * Support for legal education and legal aid in the underserved Kazakh region was provided for US$0.77 per person for all years. * US$234,000 was spent (not counting contribution made by the MOJ) for targeted legal education campaigns that reached all sectors of the population across the country several times and over three and a half years, equating into US$0.08 per person for all years. The investment in the construction activities, on the other hand, may not have been the best value for money. While the new buildings and refurbishment were much needed, comparison information from the CDEA and the courts indicates that the government can build such facilities at lower cost and in a shorter period of time. While the overall budget situation of the justice sector was weaker at the time the project was designed, the courts and the CDEA have since funded several buildings out of their own budget. Additional cost and economic information is provided in Annex 3. 3.4 Justification of Overall Outcome Rating - combined relevance, PDO achievement and efficiency Rating: Moderately Satisfactory The overall outcome rating for this Project is Moderately Satisfactory. The project completed all its activities within a 2 V2 year period after initial start-up problems, was and remains relevant for the country, investments are sustainable and had positive implications beyond the justice sector, most were efficient and the PDOs were largely 14 met. The last ISR suggested this rating if all project deliverables would be completed by the end of the project - which is what was achieved. The public legal information and legal advice component did reap the intended benefits and were and remain relevant for the country. There is a marked improvement in the quality and content of legal education in Mongolia compared to project inception. There are clear standards for law schools, and law students now enjoy a modem curriculum that reflects Mongolia's rapidly changing legal framework and is more in tune with the needs of practicing lawyers. These investments were relatively low cost and efficient. The end results of this modernization process will only be seen in a number of years, when new law graduates begin to exercise their profession. Anecdotal evidence points to the fact that law school graduates already enjoy increased employability. Improvements in service delivery by CDEA agents should translate into performance improvements and initial enforcement data support this assumption. The adoption of case management programs in both the CDEA and the Administrative Courts will improve the efficiency of these institutions by streamlining data entry and process steps and real time performance information. Overall, the combined IT infrastructure investments not only provide for more effective operations and monitoring for the CDEA and the courts but also provide the backbone for further IT solutions that can improve operations, lower cost and provide greater access to justice. The civil works were much needed and are an important contribution to developing a better functioning justice system that meets the needs of users across the country. The high cost and significant delays in delivering the buildings indicate that this particular investment choice was less efficient. 3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development Estimating poverty and gender impacts is difficult within a project such as this one that generally serves all people as relevant socio-economic data of populations served by the courts and related outcomes are currently not collected by the agencies involved. The information available at this time indicates that the project made important contributions to the legal education and access to legal aid for Kazakh and Mongolian minorities, the blind, and the poor in selected project areas and that women often reflected a greater share of this assistance than man. For example, 54% of the individuals assisted by the two legal aid clinics in Bayan-Ulgii Aimag were women. The percentage of women served by legal aid clinics in poor ger districts in UB was 69%, while women represented 68% of the population served in the Narantuul Market clinic. The increased number of legal clinics at laws schools, including special clinics serving the ger districts expanded the availability of free legal aid to the poor in Ulaanbaatar as did the training of paralegals in Aimags (see Annex 11 Table I b-5 for more detail). The multimedia campaigns designed under Component 1 targeted specific age groups for important societal issues. Printed information disseminated across the country and the websites for legal information further provides free access to legal information for all. From a social development perspective, improving, facilitating and expanding individual and collective access to law and justice supports economic and social 15 development. Public legal education gives the poor and minorities the opportunity to assert their individual and property rights; improved access to justice via legal advice and assistance empowers the poor to enforce those rights. Increasing accessibility to courts lessens and overcomes the economic, psychological, informational and physical barriers faced by women, indigenous populations, and other individuals who need these services. (b) Institutional Change/Strengthening The project strengthened the operations of its key counterpart institutions leading to changes that are likely to show long term impacts. Under Component 1, the project's support to the development and coordination of a network involving all law schools to work together to improve education standards, teaching capacities and material led to fundamental improvements in law school education. Linking this network to relevant MOJ and MES counterparts was important to ensuring that accreditation criteria reflect the needed changes and that reforms are institutionalized. Engaging with the Mongolian Association of Law Schools strengthened this professional network and resulted in a number of long-reaching recommendations that are now taken forward and will further improve legal education in Mongolia. The project also supported progress in instilling a service delivery culture within the CDEA via creation of a training center and development of trainers. These capacities remain with the agency and will continue to serve it well. In addition, the user-friendly ICT systems in both the CDEA and the Administrative Courts lead to greater efficiency capacities in both institutions that remain, evolve and are already used as models for other courts and justice sector agencies. These systems also provide for real time monitoring information and create performance data that can be readily shared with the public for greater transparency. (c) Other Unintended Outcomes and Impacts (positive or negative) The work to revise the law school curriculum was deemed a best practice for Higher Education by the MES at project closing. Anecdotal evidence showed that other education sectors were interested in adopting similar teaching methods. The posting of Supreme Court decisions on the www.legalinfo.mn website not only increased transparency but meant that law faculties could utilize these decisions as part of their new teaching approaches. In addition, there is anecdotal evidence that judges now pay closer attention to how they draft their opinions. The project also provided important legal outreach to Mongolia's blind population through a legal information campaign in partnership with the National Association of the Blind. Laws had not been published in braille before due to the relatively high costs to produce. For the first time, blind Mongolians now can directly access these laws which is particularly important in relation to the Law on Social Welfare and Insurance, and children's rights. With branches in each Aimag, the National Association of the Blind can easily disseminate laws and other materials ensuring a nationwide reach that will be sustained beyond the project. One unintended, positive outcome of the project was that the civil case management software for the courts that had been developed with support of the GIZ for the past 2 years was adjusted when it became clear that the administrative software supported by the Bank project had greater functionalities and was more user-friendly. 16 3.6 Summary of Findings of Stakeholder Workshops A Stakeholder Workshop was held on June 12, 2013 in Ulaanbaatar. Approximately 50 people attended, including a broad group of representatives from the various agencies and stakeholder groups involved in the project (the MOJ, the GCC, the CDEA, the administrative courts, the courts, the Kazakh and ethnic minorities, the National Association of the Blind, and law schools, among others) as well as other donors. For the most part, the stakeholders felt that the project had accomplished a great deal in relation to their own individual activities. The stakeholders applauded the project's flexibility in meeting the evolving policy and needs of the sector. There was some consensus around the fact that all the project stakeholders could have been more efficient in implementation activities. It was also noted that in many cases, the project had begun or piloted activities that will be continued under government programs or policies. Similarly law school and MOE representatives mentioned the important contribution the project made to bringing them together and improving legal education and that many activities will be continued by the law schools themselves. Representatives from the Kazakh region stressed the importance of the project for increasing access to justice for minorities. The courts and the CDEA were particularly grateful for the installation of the fiber optic network, as it will improve efficiency and ultimately cut costs for postage, telephone, and travel. Similarly, the physical works for the courts and CDEA offices were also viewed positively, as the staff of both institutions now enjoys much improved working conditions. The construction and rehabilitation has allowed some courts to shed the vestiges of their socialist past, which provoked fear in some and avoidance in other court users. More details regarding this workshop can be found in Annex 6. 4. Assessment of Risk to Development Outcome Rating: Low At the time of project closing, the Government and other stakeholders are committed to sustaining and advancing the achieved reforms and have adopted a number of policies that support the activities financed under the project. Public legal education activities are a MOJ mandate and the models supported by this project are being advanced for future activities. Legal aid investments are being sustained either by the MOJ, Aimag-level budgets or law schools. Changes supported to modernize undergraduate legal education were largely institutionalized and championed by the Mongolian Association of Law Schools and endorsed both by the MOJ and the MES. As a result, there is little risk of reversal of these changes. There was strong Government commitment to provide budget allocations for maintenance of buildings and ICT investments. The ICT activities are further supported by the current e-government strategies and the software solutions have been rolled out and are being used in the administrative courts and the CDEA offices. They are already used as models for similar ICT solutions in other courts and justice sector agencies. 17 5. Assessment of Bank and Borrower Performance 5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry Rating: Moderately Unsatisfactory As described in section 2.1, the original project design had its challenges and preparation especially for components 2 and 3 was not sufficiently advanced, leaving too much to be determined during implementation (e.g. sites for construction, software development preparations). On the positive side, the design of the law school and legal education component was a good continuation of prior Bank supported activities and built upon experience of other donors. (b) Quality of Supervision Rating: Moderately Satisfactory. In the first years of implementation, the project failed to get the larger activities (construction, fiber optic network) underway despite active communications with the PIU. Efficient engagement with the GOM began in 2010 with difficult discussions regarding the project's unsatisfactory implementation and the need to revise the proposed constructions in light of rising costs and the inappropriate location selected for the construction of the Supreme Court building. These discussions helped refocus the GOM's priorities and activate the PSC and other stakeholders for technical input on project activities. Action plans were regularly agreed upon, followed-up and discussed with intensified supervision. As for the financial and procurement aspects of the project, local Bank staff in Ulaanbaatar provided day-to-day support, with additional assistance from the Bank's office in China, in addition to the DC-based task team. The supervision of the procurement aspects was particularly challenging as noted in every Aide Memoire starting from the end of 2010. The inability of the client to expand the PIU capacity contributed to the late delivery of major project activities and required significant additional support from the World Bank team. (c) Justification of Rating for Overall Bank Performance Rating: Moderately Satisfactory Despite project design issues, startup problems and delays, the Bank was able to overcome these flaws by identifying with the GOM viable project activities that were linked to the PDO. Furthermore, the Bank worked diligently with the PIU to arrive at rigorous action plans that would catalyze the implementation activities already throughout the lengthy restructuring. The Bank team made a case for increased project supervision budget to keep the revised project activities on track, and augmented supervision missions by actively supervising the project from Washington through videoconferencing. The result was a marked increase (almost 80 percent) in disbursement and completion of all project activities after the project restructuring. The flexibility and guidance provided by the Bank, especially during the last 2 V2 years assisted the GOM's efforts to ultimately successfully implementing all project activities. 18 5.2 Borrower Performance (a) Government Performance Rating: Moderately Satisfactory The GOM supported this project throughout project design, appraisal and implementation. At project closure, it remains committed to continuing and advancing the project activities. Justice sector reform remains a key element of the country's democratic development. The justice sector has been and continues to be essential to the country's economic development. There were occasional gaps in this support due to political changes and limited human capacities at the MOJ and other counterpart agencies. In 2010, when the project was delayed and not on track to meet its development objective as designed, the MOJ restated its strong support for the project and agreed that a restructuring of the project was necessary. The MOJ continued to support the more holistic justice reform approach instead of a limited focus on construction. In response, the GCC proposed a major change in the Bank-financed construction activities; a difficult and unpopular decision for the SC which delayed the Government's restructuring request for a number of months. To the GOM's credit, the number of stakeholders involved in the public legal education and information component and the law school reforms helped shape the activities, and ultimately their success. The comprehensive legal education reforms involved solid consultations, coordination and engagement of all key stakeholders and their ownership of this process resulted in advances that should be sustained over a considerable period of time. The MOJ recognized and commended the project for allowing them to pilot a number of different means and forms of communication to implement their public information strategy, and more importantly, providing them with costed models that can be replicated. Similarly, the MOJ's willingness to focus on active engagement and strengthening of a stakeholder network created ownership in law school reform within the broad private and public law school community that ultimately lead to almost a grassroots approach to sustainable reform results and capacities at law schools. The creation of law schools standards, which were then enacted into law, helped to further consolidate these gains. The CDEA which was new to Bank projects was actively engaged after restructuring and successfully institutionalized uniform processes, training and IT solutions. The administrative courts have pro-actively advanced their ICT solutions and continue to influence improvements across all courts. (b) Implementing Agency or Agencies Performance Rating: Moderately Unsatisfactory After effectiveness, the implementation arrangements for the project remained much as they had under the previous project, despite the fact that the EJSSP's project activities required a different skill set. This was a factor in the slow project implementation during the first two years. Despite the MOJ's prior project experience, the Bank's processes and procurement procedures did not appear to be well known by the largely new PIU or the relevant MOJ staff, leading to management issues and project delays. Throughout project implementation, the PIU was not properly staffed to manage the dimension or the complexity of the project activities. This impacted particularly the 19 construction and ICT activities. While MOJ and GCC staff were involved to some extent with these activities, there was no staff architect or engineer working day-to-day to handle the ambitious construction activities at any point in implementation. This made handling the shortened construction season (April-October) more delay-prone and difficult. The MOJ and law school community did well in implementing legal education and law school reform activities and the PIU engaged a broad number of stakeholders to this end, but this too stretched the PIU thin - leaving little room to keep the time consuming construction and ICT components and M&E activities on track. By the Mid-Term review, the PSC had begun to meet regularly and provide technical input on project activities, particularly the construction activities. While the PIU could draw upon the PSC when technical issues arose, the PSC did not have the capacities to provide closer supervision to the PIU on a day-to-day basis. The Bank made consistent recommendations to improve the staffing skill set of the PIU through either contracting of qualified staff on a longer term (one year) basis, or through the use of short term consultants. Some short term consultants were brought in, but overall the skills mismatch continued to hamper project implementation. (c) Justification of Rating for Overall Borrower Performance Rating: Moderately Satisfactory The Borrower's performance from project design through implementation was mixed but in the end successful. While the MOJ was highly committed throughout, election cycles and changes in administration during project implementation disrupted the project activities for periods of six months or longer. While this is typical of development projects, the weak staffing of the PIU meant that these changes caused delays and posed more fundamental problems for the project. The PSC started to function as intended beginning in late 2010; the PSC's role was mainly to provide policy advice and take decisions on larger project issues. It was the PIU's role to inform the PSC about project activities and implement and execute the PSC decisions but it remained inadequately staffed to fulfill its many roles and responsibilities. The GOM could have provided closer supervision of certain aspects of implementation, especially procurement, providing technical staff of its own when the PIU staffing was not adequate. At the same time, the MOJ and other stakeholders increased their efforts when it was explained that the project was in danger of failing, agreed to action plans, and contributed funds from their own budget in select areas, leading in the end to full project implementation and almost complete PDO achievement by the time the project closed. 6. Lessons Learned One lesson learned here - and often stressed for successful justice reform projects - is that a more holistic approach tends to be difficult to implement but also more effective in creating sustainable change across the sector. The EJSSP engaged two ministries, the courts and court enforcement agency, a broad range of stakeholders from the public and private law school community as well as national NGOs and national and local government entities that continue to deliver a range of justice sector services. Engaging so many stakeholders successfully to the point that each group takes active 20 ownership and carries the needed changes takes time and requires related capacities in the PIU even when additional consultants are hired to coordinate components. For example, progress made under the law school component demonstrated that the combination of training, course material, and policy changes along with supporting a coordinated approach to changing law school material is a good example for sustainable capacity building, creating local ownership and implementing reforms that can be sustained and continued. But it took several years of engagement and effort to get to this point that often stretched the capacity of the PIU and this is to be expected - and to be reflected in the project design - in most of the Bank's client countries. Good and broad stakeholder involvement throughout implementation ensures not only that the design is a good fit but develops ownership and capacities beyond the targeted activities. Similarly, steering committees have an important function and are a good way to support project implementation but this requires that a clear TOR is developed for the committee, that all members understand their roles and are actively engaged. The PIU has an important role in supporting the steering committee by facilitating their understanding and engagement and not just through coordinating regular meetings and has to have the capacities to provide these. Another general lesson that has been shown here, and in other justice and public sector reform projects, is that actual change in agency operations and justice outcomes are largely dependent on changes in institutional culture, which often take longer than the typical lending project cycle. In recognition of this the PDO Indicators were adjusted at restructuring to reflect the implementation challenges and were consciously chosen to focus on outputs only. The project activities are likely to also produce justice outcomes but for such to materialize the new systems and approaches will need to operate for a year at least. Bank project designs rarely allow for such follow-up period to demonstrate impact. This project pre and post-restructuring demonstrated the great care that needs to be taken when crafting a results monitoring framework during appraisal. The M&E framework needs to clearly reflect the project design, has to be realistic in terms of what can actually be achieved in a given time period and has to reflect availability of data and capacities to collect them. Such a framework also has to consider cost and economic information to allow some efficiency estimates, as difficult as this generally is for justice reform projects. The Bank provided significant technical assistance to the PIU for collecting and analyzing data throughout the project. Lacking a devoted government interlocutor on the project's M&E, Bank missions were often used to focus the sector on M&E and its usefulness. While this is an essential aspect of Bank supervision, a robust M&E framework requires a regular focus from the PIU to collect and monitor the information. For example, the EJSSP financed a wide range of new models and approaches to disseminating public legal information. The approaches chosen were cost-effective, based on needs assessments, well-coordinated and cofounded by the MOJ and other relevant agencies. The multi-method approach largely ensured these activities reached a broad spectrum of the population. All of these design elements may provide models for public education campaigns that are locally driven and implemented. Since the design did not include a way to capture results and measure impact to demonstrate that this was an 21 effective and cost-effective way to reach different population groups across this vast country, the actual value of these project activities is difficult to demonstrate conclusively. Another important lesson learned was that legal covenants are no replacement for project-relevant preparedness. The inclusion of significant construction activities along with the peculiarities of the Mongolian construction cycle should have spurred the Bank team to have completed the site selection, site surveys along with initial procurement planning at appraisal. Issues in site suitability and compliance with the Bank's safeguards policies along with cost development trends need to be fully studied during the preparation and appraisal phases, when the Bank has more budget to hire specialists to appraise the work and the construction environment including conducting some cost- benefit analysis to inform design choices. If disbursement conditions are used, they should be unbundled as much as possible, given these components are prone to delays. Infrastructure components not only require a lot of time for planning, architectural services, site reviews and procurement, they also require both specific expertise for preparation and implementation and a budget to provide this expertise. Given that the weight of the Bank's staff budget allocations is higher during project preparation, project teams are well advised to prepare the construction activities as much as possible at the time of appraisal. Bank project supervision budgets are significantly lower, and the budget for specialists needed for this type of work during implementation is therefore smaller requiring reactive use of off-site assistance and local consultants. The Mongolia- specific short construction period requires particular attention to timing and the vast size of the country along with the difficult road and climate conditions make supervision difficult and can impact timeliness and quality of implementation. A project design heavy with construction requires that a PIU has the expertise and capacities to manage such activities, if not on staff then at least in form of local expert consultants that are contracted for the required period. Adequate staffing needs to be reflected in the design and capacity gaps need to be addressed early on. The Bank expended significant time and effort over the course of many supervision missions remedying many issues related the PIU's lack of capacity, which detracted from discussions of the project's larger objectives. Additional Mongolia-specific lessons learned, particularly related to construction, are outlined in the MTR; these were shared by the CMU with other ongoing projects that had construction components too for cross-learning purpose 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners (a) Borrower/implementing agencies The Ministry of Justice will send comments regarding the project. (b) Cofinanciers Not applicable. (c) Other partners and stakeholders (e.g. NGOs/private sector/civil society) See Annex 6 for comments made during the end-of project stakeholder workshop. 22 Annex 1. Project Costs and Financing (a)Project Costs by Component A E Estimate Revised at Actual/Latest Estimate Percentage of Components Apprail Es restructuring (USD millions) Restructured (USD millions) Estimate CoMponent One: Enhancing Public 500,000.00 CompoEucnt on Eha uic ector (250,000.00 IDA; 500,000.00 490,687.03 98% Legal Education on the Justice Sector 250,000.00 PHRD) Component Two: Increasing 1,410,000.00 Transparency Through Improvements (1,260,000.00 IDA; 550,000.00 510,420.86 93% to Legal Information 150,000.00 PHRD) 4,790,000.00 Component Three: Enhancing (,40,000.00 Judicial Operations, Enforcement, 40,000.00 3,660,000.00 3,186,026.94 87% and Monitoring of Court Decisions '0,000.00 PH M) 1,000,000.00 GOM) Component Four: Project 250,000.00 Management (150,000.00 IDA; 280,0000.00 366,821.69 135% 100,000.00 PHRD) Total Baseline Cost 6,950,000.00 4,990,000.004 Physical Contingencies5 383,400.00 0.00 0.00 Price ContingencieS6 176,060.00 0.00 0.00 Total Project Costs 0.00 5,000,000.00 4,553,956.52 91.08%7 Front-end fee IBRD8 0.00 0.00 0.00 Total Financing Required 5,000,000.00 0.00 (b) Financing Appraisal Restructuring Actual/Latest Source of Funds Type of Estimate Estimate Estimate Percentage of Cofinancing (USD (USD (USD Restructuring millions) millions) millions) Borrower Counterpart 1.00 0 0.67 -- International Development Credit 3.35 3.35 2.99 89.25 A ssociation (IDA ) _99_89_25 IDA Grant Grant 1.65 1.65 1.54 97.02 PHRD Grant Co-finance 0.95 0.00 0.00 -- 4 This amount reflects differences in exchange rate between XDR and USD. 5 The physical contingencies were front-loaded into the original project estimates for Component 3 (Enhancing Judicial Operations, Enforcement and Monitoring of Court Decisions). 6 In the original project cost estimates, the price contingencies were front-loaded and split mainly between Component 2 and Component 3. This reduced amount reflects fluctuations in the exchange rate between XDR and USD. Table 1 (b) shows that the overall disbursement rate of the IDA Credit and Grant were 94.41% and 97.02%, respectively. 8No Front-End Fee was charged on this loan. 23 (c) Project Cost by Component (in USD equivalent) Appraisal Restructuring Actual/Last Estimate Project Component Estimate (USD Estimate (USD (USD millions) millions) millions) Component I: Enhancing Public Legal 0.50 0.50 0.49 Education in the Justice Sector Component II: Increasing Transparency through Improved 1.4 0.55 0.51 Access to Legal Information Component III: Enhancing Judicial Operations, Enforcement and 4,79 3.66 3.17 Monitoring of Court Decisions Component IV: Project Management 0.25 0.28 0.38 (d) Actual Costs of Revised Components and Activities Project Activities Accumulated Disbursed Amount (total) Component I: Enhancing Public Legal Education on the Justice Sector 490,97.03 Comprehensive Legal education reform in law schools 252,344.75 National Public Information Strategy 238,352.28 Consultancy on Strategy 17,682.32 Development & IP of NIPS 201,622.86 Kazakh Region Strategy 19,047.10 Component II: Increasing Transparency through Improved Access to Legal 510,421.30 Information National Justice Sector Information Networking Strategy and Systems 510,421.30 Fiber optic connections to Local Court Decision Enforcement Offices, Courts 218,299.15 and Prosecutor's Offices Supervision cost over Fiber optic connections 5,624.62 Consultant on Fiber Optics 3,600.00 Procurement of Software to CDEA 107,872.88 IT Consultant on Software 5,000.44 Procurement of ICT equipment to CDEA and GCC 170,024.21 Component III: Enhancing Judicial Operations, Enforcement and Monitoring 3,186,026.94 of Court Decisions Justice Sector Support Infrastructure 2,076,090.44 Architectural Services-GCC 9,950.07 Construction Works-Courts 2,056,445.62 Supervision cost over GCC Constructions /PIU 9,694.75 Consultant service of Supervision Engineer over Construction 2,475.35 Court Strategic Planning and Administrative Improvement 163,078.36 Procurement of Software to Administrative Courts 132,742.25 Workshops with Administrative Courts and CDEA's Software Working Groups 16,353.97 Purchase of equipment and furniture for the MOJ unit: Unified Justice System 13,982.14 Court Enforcement Capacity Enhancement 935,858.14 Architectural Services-CDEA 57,736.11 Construction Works-CDEA 794,165.33 Supervision cost over CDEA constructions /PIU 7,116.30 Supervision Cost over Construction /CDEA and GCC Buildings/ -LACUDA 8,131.25 Capacity Building for CDEA Staff 0.00 1. Training (Consultant) 0.00 24 2. Training of Trainers 0.00 3. Training of CDE Officers 50,562.93 4. Publication of Handouts for CDE Officers 0.00 5. Publication of Training Handouts for CDE Officers 1,707.32 6. Purchase of equipment and furniture for the CDEA's Legal Clinic 24,570.15 Component IV: Project Management 367821.69 PlU Consultant Services and Training 250,281.32 Incremental Operating Cost 46,255.85 M&E Consultant /short term-4 months 4,000.00 Roundtables: Justice sector reform Mongolia 5,798.77 Purchase of Equipment and Computers for PlU 29,483.51 Audits 16,184.44 Meetings and Workshop of PIU 15,211.20 Total Project Expenditures 4,553,966.96 25 Annex 2. Outputs by Component When the project was restructured in April 2011, the original components overall remained. The restructuring reduced the scope of the construction activities to bring them better in line with cost realities, government strategies and priorities for the development of the justice sector (see section 1.3 and section 1.6 of the main text). This annex will discuss the outputs for the project's components and their revised activities. Component I: Enhancing Public Legal Education in the Justice Sector (Appraisal estimate: US$ 0.50 million; estimate revised at restructuring: US$ 0.50 million; actual costs: US$ 0.49 million) The original activities of both subcomponents were largely maintained, even slightly expanded during restructuring. The assistance to law schools was expanded to support the implementation of new national standards for legal clinics and support for the public legal education campaigns was expanded beyond delivery of information and education programs to include training of community paralegals to enhance access to legal assistance to the poor in the ger districts in Ulaanbaatar and in rural areas across Mongolia, which further stakeholder consultation conducted by the MOJ had identified as a special need. The assistance to Kazakh minorities in Bayan-Ulgii Aimag was expanded to include support for legal information and assistance for Mongolian minorities in the region since stakeholder feedback indicated particular access to justice issues for those groups, especially for poor women of all minority groups. a. Continuation of Legal Education Reform (actual costs: US$ 0.25 million) Updating legal education syllabi and developing handbooks for law professors and students. As Mongolian law schools move from a lecture-based method of teaching to more Socratic and practice-based methods, aimed at legal skills instead of memorizing laws and theory, the creation of new course material that supports modern teaching methods provided important tools for standardizing legal education, as well as delivering quality course materials for the many new areas of law that gained importance as the country continues to develop into a modern market-based society. The project built upon earlier efforts under the previous project and supported the development of standards syllabi, teacher and student reading material for 23 required and elective law school courses. This is the first time such standard teaching and reading material is available and is now used at all law schools. For the 2013/14 school year, this translates into a total of 322 standardized courses, attended by a total of 6,505 students at 13 law schools. Combined with the law school teacher capacity building described below, this greatly contributes to the overall quality of law school education in Mongolia, while taking into account evolving issues such as business law, international private law and banking, where a high demand was identified. An average of 1,100 pieces of each teaching and reading manual were printed and distributed to all law schools for free. The full list of syllabi and handbooks developed is shown in Annex 11 Table I a-1; the detailed enrollment numbers are listed in Annex 11 Table I a-2. 26 The course material has been uploaded to the NLI website (www.legainstitute.mn) and with the beginning of the new school year on September 1, 2013, all handbooks and syllabi developed can be downloaded for free. The demand for this material was demonstrated by a significant increase in website visitors. In September 2013 the website was 'clicked' 1,248,421 times. An increase of 137% compared to September 2012, when only 527,307 'clicks' were counted. Surveys of the use of the materials show that it is highly valued by law professors and students. Out of 38 professors interviewed 24 professors rate the handbooks' main contents as excellent, 26 professors rate the handbooks methodology as excellent, and 29 professors rate the handbooks' design as excellent. Preliminary survey results for students' satisfaction draw a similar picture: out of 175 students surveyed 117 rated the handbooks' main content as either excellent or good, and 123 rated the case studies used as excellent or good. The satisfaction survey details are listed in Annex 11 Table I a-3. The syllabi and handbooks were developed after considerable consultations with all core stakeholders, in specific the Law School Association, representatives and teachers from the most active public and private law schools, the MOJ and the MOE, and a strong focus was given to current and future demands and quality. Special attention was paid to didactic and greater provision of skills instead just of knowledge of the law; modem adult learning techniques were applied to ensure best teaching and learning results. The multi- faceted support of the project has been a significant contributor to developing better quality and more standardized course content, teaching approaches, and opportunities for practical learning. The Mongolian MOE considers the approach taken as best practice model and education/teaching experts have begun to apply a similar approach for enhancing other education programs. Stakeholders stressed the contribution the project made to develop a more cohesive approach among law schools to work together on systemic improvements. Collaboration between law schools and professors started and deepened as result of their ongoing engagement by the project, creating a more cooperative professional network to continue reforms on their own. The changes supported by the project are considered by counterparts as transformational - the coming years will show if this assessment is justified. Improving capacities of law professors. To develop the capacities of law professors and teachers a range of training activities were conducted to assist them in moving from a lecture-based system of teaching to a more didactic approach, using practical examples and adult learning techniques. The teacher handbooks are designed to assist professors in using modem teaching styles. A total of 633 law professors and teachers participated in 8 training courses covering a broad range of topics from research methodology to ethics. The full list of trainings provided can be found in Annex 11 Table I 1-4; the according satisfaction survey in Annex 11 Table I a-5. Supporting legal clinic education at law schools. In a country like Mongolia, where market demands require that legal education is rapidly evolving to be more skills- and practice-oriented, legal clinics provide students an opportunity to gain practical experience that is otherwise still very limited. Under the supervision of experienced law professors, students provide free legal aid to those who cannot afford a lawyer. Beyond 27 allowing students to learn practical skills, these clinics also serve the dual purpose of providing cost-effective and potentially sustainable models for free legal assistance to the poor - an area of high need in Mongolia. At the start of the project only 2 law schools operated legal clinics. The project supported the development of standards that require all law schools to provide quality legal clinic training to law students. Today all 13 law school clinics are following the new standards, building upon the training and teaching material developed with support of the project, and actively providing services to the poor. In addition, the project supported the creation of legal clinic sub-stations in the poor ger districts in Ulaanbaatar. As a result, 1,131 people were served at legal clinics in 2012 and 1,631 until September 2013. Of these 591, respectively 966 were women. In addition, the two clinics created in the ger strict in 2012 have since served 456 clients (314 women), the legal clinic at the Narantuul market provided legal advice to 2,266 (1,534 women). Details about the types of cases in which assistance was provided are listed in Annex 11 Table I a-6; case details for the legal clinic substations in the poor ger districts in Ulaanbaatar are listed in Annex 11 Table 1 a-7. Further, 10 law schools received assistance to implement legal rights awareness and crime prevention programs at high schools. This provides for another opportunity for students to sharpen their legal communication skill. Law school representatives reported that students developed further practical skills and these programs reportedly increased rights and responsibility awareness among high school students. Mongolian law schools are in the process to develop plans to continue and extend these projects in the future. A total of 85 law school students (35 male, 50 female) and 1,130 (472 male, 658 female) high school students participated. Details for these tailored (high school) projects are listed in Annex 11 Table I a-8. Creation of Standards for Law Degrees in Mongolia. While a standard law degree exam is still not in place, the government's own accreditation efforts and support for them have increase and a range of their measures to ensure more uniform quality requirements are in place today. Related discussions began under the previous Ministry of Justice and Home Affairs (MOJHA), and continued with the current MoJ. The project supported the creation and activities of a Working Group, comprising members of the MOJ(HA), law schools and the MOE to advance these efforts. Based on the findings of a study presented in a two volume "Report on Bringing International Standards to Mongolian Legal Education" produced under the previous project, and lessons learned during a study tour to Germany, co-funded by the Hanns-Seidel-Foundation, the Working Group developed a set of recommendations for law school education standards in line with good international practice. Additionally, an international conference was held in Ulaanbaatar, bringing experts from the US, Germany, South Korea, Japan and Taiwan, China to discuss their experiences and how they can inform the reform process in Mongolia. Using this combined information, the Working Group submitted its recommendations to the MOJ in March 2013. The MOJ reviewed the recommendations and started to plan for implementation of the following: 28 (a) Introduce a 'legal profession qualifying program' similar to the legal profession education system in Germany, i.e. a law degree plus a two-year-on-the-job- training period; (b) Introduce an education system, including related standards, for (future) law teachers/professors to ensure quality educational services; and (c) Reform current teaching methods. The project activities supported the respective counterparts in setting the foundations to implement these recommendations and to build up a modern and high-standard law school education in Mongolia. b. Development and Implementation of a National Public Information Strategy (actual costs: US$ 0.23 million) Public Legal Education Campaigns. The public education program implemented from 2010 to 2012 by the MOJ was focused on 9 social topics important to Mongolia. The campaigns were based on significant stakeholder consultations and well-coordinated with other relevant government agencies across all Aimags and the Capital City. During 4 waves of public education campaigns people received information through TV spots, leaflets & pamphlets, workshops and trainings, and special events. Though the previous and new GOM viewed public education related to understanding of the law and access to justice as important, all national level public information activities have been on hold since May 2012, right before the parliamentary elections. Based on the good experiences and driven by the need for public legal education, the MOJ is now looking again into options for streamlined national public education campaigns. A range of outputs can be listed that indicate a wide reach of the public education activities supported by the project beyond a series of TV spots (411 in total) and other public events. This includes 50,000 leaflets on 25 select legal topics and 4,000 information posters providing legal information on 3 different legal subjects distributed to the legal departments in the Governor's offices of all Aimags; animated legal booklets and posters were provided to public schools across the country, along with posters announcing the legal website, its services and information. As a results, legal information was distributed directly to the public across the country or has been posted well-visible in agency offices and waiting halls. Another publication, "Your Legal Guide" developed around 7 subjects with corresponding DVDs was also submitted to all legal departments in the Governor's offices in the Aimags. In addition, 7 TV programs were developed based on the Guide's 7 sub-subjects, distributed to local TV stations across the country and broadcasted. As more and more people in the countryside have access to electricity, television is increasingly the source of information in comparison to radio broadcasts, one reason for the focus on video material and TV programs. The public education campaigns conducted are listed in more detail Annex 11 Table I b-I. Details of the public education materials distributed are listed in Annex 11 Table I b-2. The project supported upgrading of two MOJ sponsored websites that provide access to all current and enforceable laws, judicial decisions, government decrees and other related legal documents (www.legalinfo.mn - www.e-hutuch.gov.mn). The National Legal 29 Institute (NLI), currently the caretaker of the legal website, noted anecdotally that the website is used considerably by Mongolian lawyers - when the website goes down for maintenance, public complaints calls instantly increase. The upgraded website is also the second most popular not-for-profit site in Mongolia. The costs for the website's maintenance and services will now be paid under the state budget. To promote this easy and free access to laws, regulations, and decisions the project printed over 4,000 informational posters about the websites and distributed them across all Aimag and District Courts. Anecdotal evidence indicates that the publication of all judicial decisions resulted in judges more carefully drafting their decisions, as published decisions are attributed to specific judges; furthermore law schools are beginning to use the actual cases and decisions as practical teaching tools. Targeted legal assistance and public education for Kazakh and Mongolian minorities. The project supported legal education campaigns and a legal aid center for the Kazakh national minority residing in Bayan-Ulgii Aimag, as well as one for the ethnic Urianhai, Durvud and Tuva people to meet the needs and demands of all Ethnic groups present. Until this project provided assistance, no legal information was available in the Kazakh language, impeding access to justice for this minority. Since the ethnic Mongolians living in that region face other challenges related to access to justice service, a special center to assist them was equally essential. Both centers continue to show positive results and their work will now be supported from the MOJ's budget. This very important source of legal information and legal aid for these minority populations was provided at very low cost. Less than US$20,000.00 were spend over the course of 2 years and 7 months for outfitting and operating these centers and for providing targeted public education activities. The services included provision of legal information, legal aid and legal outreach campaigns tailored to the Ethnic needs. A total of 14 key laws were translated into Kazakh and a range of legal information pamphlets, and radio announcements along with other information activities have been delivered providing for the first time comprehensive information to Kazakhs in Mongolia. This information was also posted on a Kazakh language website. The data collected shows that both centers supported not only important information campaigns but provided much needed legal aid to the poor. A total of 15,015 Kazakhs either received legal aid or were informed through specific campaigns, plus a total of total 828 ethnic Mongolian minorities received legal aid and a total of 860 ethnic people were addressed via specific campaigns through the centers, which makes up for nearly 20% of the Aimag's population (89,191). This service is especially essential for poor women in the region; for the mainly Muslim Kazakh women this is generally the only legal assistance available to them. More than half of the legal aid recipients since November 2010 were women. In total 1,421 Kazakh women 1,399 men received legal aid; 424 Mongolian ethnic minority women obtained legal aid compared to 404 men. Further details for the Kazakh Legal Aid Center and the Legal Aid Center for the Ethnic Minorities can be found in Annex 11 Table I b-3, respectively Table I b-4. Further details for paralegal activities across the country can be found in Annex 11 Table I b-5. 30 Component II: Increasing Transparency through Improved Access to Le$al Information (Appraisal estimate: US$ 1.4 million; estimate revised at restructuring: US$ 0.59 million; actual costs: US$ 0.36 million) Under the previous Legal and Judicial Reform, a Unified Legal Information system was created in collaboration with other donors. Under the prior project, 29 ministry and justice sector agencies in Ulaanbaatar had been connected via fiber optic cable with the aim to increase their efficiency and transparency once related data tracking and exchange software were available and by providing them with access to all legal acts, laws, regulations, and resolutions passed by the Supreme Court and the Constitutional Court. The original design of the project correctly identified the need for connectivity and integrated system across all courts and related agencies to enable proper data sharing as at the start of this project only few agencies were able to take full advantage of the Unified Legal Information system. However, the original project design underestimated the significant software enhancements that were needed before adequate data sharing could be implemented to actually address the shortcomings in agency-level data tracking and exchange software and the lack of linages to agencies outside of the capital. Thus, the restructured project activities also included the development of case management software for the administrative court to complement parallel activities for the civil courts supported by the GIZ. a. Connecting Aimag Courts and Court Decision Enforcement Agency (CDEA) Offices to the National Justice Sector Information Network (actual costs: US$ 0.16 million) After significant procurement delays, a contract to establish the needed fiber optic connections across all Aimags was signed with the successful bidder in November 2012. Implementation activities then faced winter weather conditions and work started again only in April 2013. Nevertheless, by the project's end, i.e. within 8 months from contract signing to project closure, a total of 79 locations, including all Aimag and Inter-Soum courts, Administrative Courts, and CDEA offices were connected via fiber optic cable, covering all 21 Aimags as well as the Capital City. The connections for all sites were approved by ICN, the state-owned company and data transfer on a daily basis has started. The full list of sites connected can be found in Annex 11 Table II a-1. To make the most of it, the capacities of the fiber optic network have been developed with future increased data capacities in mind and have been identified by other justice sector agencies, including the police, as a valuable resource to link their own data exchange needs. As the technical infrastructure and equipment is in place, data sharing agreements between the courts and CDEA - and ideally with the prosecutor offices - have to be in place to ensure effective use of this newly established connectivity. Now that the data are available from all enforcement agencies at least on a weekly basis and at least the administrative courts will have full regular data reporting capacities by November 2013 (see below) a meaningful agreement about data exchanges can be made. A MoU for data exchanges was signed on May 29, 2013 and a working group has been created to develop 31 the data sharing details, i.e. data definitions, exchange schedules, etc. Several new laws related to the governance and responsibilities of the GCC and all courts will come into effect on November 1, 2013, which require more evidence-based management and greater reporting transparency of the courts. Among others, the judges will also assume monitoring responsibilities for court decision enforcement outcomes. Data sharing agreements have been designed to provide judges with the needed information to take on this new responsibility and are being tested. The availability of data will also mean that the GCC and CDEA will need to develop their capacities to analyze data and draw conclusion for policy and management purposes. This is one of several key areas were technical assistance would be helpful. b. Developing Case Management Software for the Court Decision Enforcement Agency (actual costs: US$ 0.21 million) In order to improve the efficiency and effectiveness of the CDEA, case management software was developed and installed in all offices by project closure. Currently, all 21 CDEA Aimag offices submit data to the CDEA headquarters once a week; in Ulaanbaatar this is done on a daily basis (see also Annex II Table II a-I). The 1-year-warranty on the software will provide for ongoing testing and improvements until mid-2014. This was included in the contract with the software company to allow the CDEA to proactively identify what works well and what does not, what is needed and what should be changed over the coming year. This will help to ensure that the software is exactly tailored to the needs and demands of the CDEA officers. First feedback responses show the overall high satisfaction of users with the new software; details for this first survey on user satisfaction can be found in Annex II Table 1I b-I. The recommendation that the user committee continues to work actively with the software developers and surveys the experiences across the country to reflect them in their further updates was discussed with the MoJ and the CDEA leadership. Lead stakeholders agreed that collecting such information is highly important not only for internal management, but also for communicating performance improvements and requesting needed budget support. Component III: Enhancing Judicial Operations, Enforcement and Monitoring of Court Decisions (Appraisal estimate: US$ 4.79 million; estimate revised at restructuring: US$ 3.36 million; actual costs: US$ 3.13 million) a. Justice Sector Support Infrastructure (actual costs: US$ 2.07 million) Four new court buildings and one refurbishment, i.e. renovation and addition to an existing building, were all completed by the end of the project. Court staff report greater work efficiency due to a more conducive work environment and court users reports greater satisfaction with the services in the new facilities. In specific, more than half of people interviewed - both court/CDEA staff and lawyers - responded that they are 'satisfied' or 'very satisfied' with the new buildings as it comes to equipment, furniture, waiting hall, and the location picked. 32 Details on the location of construction sites are listed in Annex 11 Table III a-1; details for the user surveys regarding the new buildings are listed in Annex II Table III a-2. b. Court Strategic Planning and Administrative Improvements (actual costs: US$ 0.16 million) To ensure that the courts actually have electronic data basis that can be effectively used for data sharing via the fiber optic cable network and to provide the means for effective performance monitoring and planning, the restructuring included the development of software to support case management in the administrative courts. The software was successfully piloted in 5 administrative courts, namely Orkhon, Darkhan, Selenge, Gobi- Sumber and Tuv. The complete roll-out of the software to all administrative courts will happen by November 1, 2013 when the new law comes into effect. Members of the administrative court working group and the software users in the pilot courts continue to provide feedback for further improvement. The software company is taking those changes into account and keeps adjusting the software to reflect the needed changes. The feedback from users was overall positive and indicates high user satisfaction, reduced data error and time savings due to reduced paperwork along with a high potential for reducing appeals rates. For example, 27 out of 28 survey participants at the Aimag courts responded that the "Alert Function" of the system helps them to meet critical deadlines, and 34 out of 36 responded reported back that the software prevents them from making possible procedural errors. Details for the user surveys are listed in Annex II Table III b-I for the administrative courts in the Ulaanbaatar, and in Annex II Table III b-2 for the administrative courts in the Aimags. The administrative court software is a significant advancement and not only applies to all three court instances - first, appellate and Supreme Court level - but has significant court management reporting and performance management functions. Despite the very short period of time the software is applied in the (pilot) administrative courts, the positive experience has already led the civil courts to request upgrading of their current software. c. Court Enforcement Capacity Enhancement (actual costs: US$ 0.90 million) The construction of three new CDEA buildings was completed by the end of the project (see also Annex II Table III a-1). As for the court buildings, staff report greater work efficiency due to a more conducive work environment and greater court user satisfaction with the services in the new facilities (see also Annex II Table III a-2). The CDEA office in Uvs that is now operating for over a year in a new facility was rated the second best CDEA office in the country and counterparts credit the better work conditions along with the training that was provided. In addition, the project supported the development of training and equipped a small, state-of-the art training facilities for the CDEA in Ulaanbaatar. The CDEA created a working group supported from its own funds for the development of needed training material. The Bank supported the delivery of the training by financing the equipment of a training center, targeted workshops and the publication of the training material. Comprehensive training for CDEA officers, who have a challenging job, is an issue and 33 the focus of these sessions on practical issues of conflict management and stress management are particularly important. In addition to facilitating a training forum for all CDEA officers in the country, the project supported 19 training courses attended by a total of for 1530 participants (some attended more than one course). Details for the trainings provided are listed in Annex 11 Table III c-I. While data for the first 9 months of 2013 indicate increased collection rates in enforcement actions across the country, expectations for significant improvement of enforcement rates especially outside of the capital should be limited. A major issue for enforcement success is the fact that a large number of cases involve individuals with very limited means and a large portion of the unemployed. The only way to address this in the future is to review judicial decision practices and options available by law. This could include the future introduction of payments in installments, restitution in kind, proportional fine schedules that reflect income levels and similar alternatives that reflect the low income levels of many of the rural population. The detailed case and enforcement data that will be available can be a valuable source to assess such enforcement constraints and inform related policy changes. Details for the current enforcement rates are listed in Annex 11 Table III c-2. 34 Annex 3. Economic and Financial Analysis Economic Analysis Component I-a: Continuation of Legal Education Reform: Approximately US$252,344 was used to continue efforts to reform law school education that had started under the prior Bank project. Complaints about the effectiveness of the justice system and capacities of all legal professionals in Mongolia continued over the years and with increasing demand for qualified lawyers especially those with a good understanding of commercial issues, the problems compounded. While international donors have supported judicial training with significant funds for over a decade with unclear results, training for private lawyers has been scarce. As a result, demands for investments in improving law school education to ensure that those entering the market are better equipped were more than justified. Except early assistance to the creation of a legal clinic at one law school supported by USAID in 2004 and occasional support to law schools in form of select study tours for a few law professors and theoretical law books from the Hans-Seidel Foundation, law school education had not been the focus of international donors and the government's reform activities remained limited. Under the previous Bank project an assessment of the status of law school education and recommendations for changes was developed along with the first standard curricula and course material but their use was voluntary. During the stakeholder workshop conducted for the ICR development, law school representatives stated that the support provided by the project has been transformational. In addition to providing the needed information and coordination support for a successful policy change to require legal clinic education, the project supported the development of standard curricula and teaching material for the majority of required law school courses and the most in demand elective courses In addition, law professors were trained in modern teaching techniques and in the use of the standard material (between 50 and 150 training participants per course). The use of the standard material is now required in every course and in combination with improved teaching skills and access to practical education options at the legal clinics, law school education is now completely changed. At the start of this project only 2 law schools offered legal clinic education, now all 13 law schools offer this form of training. When the project started curricula and teaching material for 4 courses was available, now 23 curricula and handbooks were printed and distributed, an average of 1,100 copies on each topic to each law school. Considering that course enrollment across law schools and topics is from less than 100 to 500, this number is at least sufficient for the next two years and the material is also online to be downloaded for free. The combination of training, course material, and policy changes along with supporting a coordinated approach to changing law school material resulted in a completely different approach to educating lawyers with a greater emphasis on skills, the application of law and practical experience rather than memorizing law and theory. Satisfaction surveys of teachers and students indicate positive results of these combined activities. Considering that in the 2012/2013 school year, 6,505 students were enrolled in the remaining 13 law schools, a number that has not significantly changed from the 6,630 enrolled in 2008, and that the full set of training material is only available with the start of the 2013/2014 school year, the investments made per student enrolled this school year come up to US$38.50 - a small 35 amount considering that the same investments already benefitted earlier semesters and will continue to serve students in the future. Actual impact, however, cannot be shown by the end of this project due to lack of relevant baseline data collected and the overall difficulty to measure a change in skills that young lawyers bring to the profession (currently the standard qualifying exams for lawyers who work in government agencies and those allowed to practice in courts do not assess skills and also do not extend to private legal advisers). Component 1-b: Development and Implementation of a National Public Information Strategy: Approximately $US238,352 were used for various public education campaigns that potentially reached all sectors of the population across the country via printed material, TV, radio spots, special events, websites, and paralegals. Assuming a population of about 2.9 million in 2013, this equates to approximately $USO.08 per person spent over approximately 3.5 years (the public education activities started mainly in 2010), an investment of about US$0.02 annually per person.9 Considering that access to information about the laws, individual rights and how to access the justice system are essential for the daily life of every person, for understanding one's rights to government services, for businesses to operate and for investors to understand the risks and safeguards available for investment, this sounds like a set of cost effective intervention - if these activities indeed resulted in greater public understanding and better access to justice institutions. Unfortunately, the data collected on the public legal education and information activities were largely output data and did not include estimates of actual reach, making it hard to measure the actual impact of the project or their cost effectiveness (or the cost effectiveness of different media used). More concrete information about actual reach of public education efforts was collected for the specific activities in the Kazakh region. A total of 16,703 people were reached by the different public education and legal aid activities provided for the Kazakh population and the ethnic minority Mongolian population over the 3.5 years of activities funded (7,729 of which were women). Total funding provided was US$19,047 translating into US$0.77 per person for all years. No cost distinction can be made by activity type (i.e. public education or legal aid) and actual results data were not collected (i.e., any satisfaction data, understanding gained, etc.). Component II: Increasing Transparency through Improved Access to Legal Information: The development of case management software and fiber optic networks within the CDEA and the judiciary was to provide for greater optimization of human resources, quicker times to disposition, fewer procedural errors, as well as an increased ability to collect and monitor case data. Total investment in developing a nation-wide information system for the justice sector, improving data sharing and public information websites, and IT infrastructure and capacities at all courts, court enforcement agency, the Ministry of Justice was US$657,145 (this includes the software developed for the administrative courts under Component 111-b). The total cost for connecting 29 CDEA facilities in all Aimags and UB, developing and providing the case management software 9 In addition, the MOJ and other relevant ministries provided staff and resources from their own budget. None of these costs were, however, captured and can therefore not be included in this analysis. 36 and some IT equipment was US$515,178 or US$17,178 per location. Estimating the return of this investment in form of operational cost saved should be relatively easy for the CDEA, so far it was reported that the adoption of a fiber optic network had an immediate effect of reducing overhead costs on postage, telephone calls and travel for both the courts and the CDEA offices. The videoconferencing capability, not envisioned under the original design, will further allow justice sector officials to obtain advice and information quickly, speeding up the decision-making process. It can also be imagined that there will be quicker times to disposition. Furthermore, the fiber optic system allows for distance learning activities, which will reduce costs, and increase overall effectiveness. Assessing the impact of IT investments on improved enforcement outcomes will be more difficult, if not impossible since investments in facility infrastructure and most importantly in human capacities would also need to be considered. Considering that the system only came online at the end of the project, no observations can be made as to its impact at the current time. At the same time, the training that has been provided as well as the agencies own investments and efforts to improve services seem to have some initial positive impact. Overall caseload, enforcement rate data and amounts collected from 2010 to 2012 indicate a trend of significant fluctuation and generally declining numbers cases received per year for enforcement across different Aimags (i.e. from 849 to 701 in Darkhan, 680, 907, and 585 in Uvurkhangai) but at the same time enforcement rates (i.e., percentage of successful enforcements) improved either in cases enforced or amounts collected or both. The initial data reported by the CDEA for 2013 seem to indicate even greater improvements. By the end of August, the agency had received over 20,000 cases and enforced almost 50% of these. The total amount to be collected (MMT694,325,555 or $US 406,161) was almost triple of those in prior years and the amount already collected (MNT409,159,410 or $US 239,358) almost four times what had been collected each year since 2010, largely due to significant increases in Ulanbaatar. Enforcement in the two Aimags where a new building was financed by the project was also already significantly higher than in prior years in terms of cases received, amounts to be enforced and at 50% of cases successfully enforced while the total enforcement rate for 2012 only reached 70%. It remains to be seen if these increases continue in the coming years and especially if enforcement processes across the country become more efficient. Enforcement success, especially in rural areas but also in many UB districts will remain a challenge as unemployment and poverty rates continue to be high. Component 111-a: Justice Sector Support Infrastructure: Investment in 4 new court facilities in soums and 1 Aimag court refurbishment amounted to US$2,068,366, with new construction prices ranging from about US$400,000 to almost $US550,000. As mentioned before, these amounts were comparable to other similar buildings funded by the Bank but they are higher than what was quoted by client counterparts for government funded buildings. The value of such investments in terms of operational benefits or cost- effectiveness is difficult to establish. There is no doubt that these facilities were needed. Before, the courts were housed in small, often dilapidated buildings that were not originally designed as court buildings and were shared with other agencies (the space available to the courts ranged from 50 to 100 square meters for up to 18 staff members). All selected courts were experiencing increasing caseloads. Initial surveys conducted shortly after the buildings were completed and again at the end of the project indicate 37 high staff and user satisfaction. Estimating economic returns of these investments (i.e. in form of increased court fees collected, value of cases brought to the courts, etc.) is even more difficult since court performance data would need to be available for a few years after the courts have been operating in these new environments and comparison to court operations in locations were the buildings were not replaced may not be conclusive since other factors, such as difference in overall economic situations, caseloads and staffing can impact the results. Overall, considering the increasing but still relatively low caseloads at these courts, a measurable increase in economic returns is not very likely and trying to estimate the price of greater trust in the courts is not feasible. The question if a court building is necessary at all at the soum level is also not just a question of what is economically feasible but if access to justice can be ensured in other ways and has been debated for years. These soums are each several hundred kilometers away from the Aimag center and road conditions are difficult year round. The option of holding court only one or two days per week, as it is practices in rural areas in other countries, still requires a functioning facility (possibly a smaller one or co-located with other agencies), would mean that judges - and possibly judges assistants and some court staff - loose at least a full day, sometimes two travelling to the rural location, and have a decent place to stay, which is a problem in most soums. The difficult road conditions and harsh climate also prohibits the use of mobile courts using fully equipped buses. As technology advances, however, e-filing options, the possibility to conduct most hearings via video become available. For future considerations of increasing access to the courts at the soum or even bagh level may be well served considering these options since they seem provide even greater access to the courts at lower cost. Component 111-b: Court Strategic Plannins and Administrative Improvements: At approximately US$190,000 for the development of software, fiber optic connectivity and select IT equipment the investment made to improve the operations of the administrative courts appears that it will pay off in the near future. The system covers all 26 administrative courts in the Aimags and UB on all court levels. The system is designed to reduce the time cases take, automate notifications, and is likely result in a reduction of appeals cases. Such outcomes not only translate into cost savings for the courts but cost savings for the parties, one of which is the state in administrative cases. Since the system has only come online at the end of the project no related impact information can be expected until at least a year of full operations. The greatest pay off, however, would be increased trust in the administrative court process which may be reflected in case data and user satisfaction but is difficult to collect and reflect in economic measures since money saved by the parties in attorney cost or amounts tied up in court processes for less time can only be estimated in comparison to other years, if at all, since other reasons may have contributed to changes the length of proceedings or appeals rates. As mentioned earlier, the investment in the fiber optic network is also the first step in creating e-court options that have the potential for providing greater access to justice in rural areas in an economically feasible manner. Component 111-c: Court Enforcement Capacity Enhancement: Estimating the cost benefits of investments in CDEA buildings faces similar issues as the ones outlined for the court facilities. The total cost for two facilities in Aimag centers and a smaller one in 38 a large soum was US$922,858. Again, the need to replace these buildings was clearly visible. The building in Uvs Aimag, for example was built in the late 1930, had not seen significant renovations and had been condemned by the State Property Agency in 2002. The number of cases that had to be enforced was, however, relatively stable. Different from court operations, enforcement cannot completely be relegated to e-options. Unless a bank account can be seized, a certain amount of the salary or other income encumbered, someone has to physically locate property that can serve as security or ultimately be seized. There is also the need to securely store items until payments are made or until they are sold by the state. Serving the large Aimags just from the Aimag center is already challenging and investing in functioning space, secure storage and ICT there to facilitate electronic notifications and requests to the extent possible is an essential investment. Financial Analysis All civil works contracts were procured locally under National Competitive Bidding (NCB) due to the relatively limited cost estimates for the small buildings and their location in distant rural areas that tend to be difficult to cover by local companies. The number of bidders was acceptable but at the low end and in one instance no qualifying bidder participated resulting in another round of calls for bids that delayed the construction by a year due to the short construction season. In at least one instance the capacity of the local bidder was insufficient to deliver quality works and the company went bankrupt before the work was completed. In comparison to similar construction in rural areas supported by the Bank (i.e. kindergartens) the costs where reasonably in the same range. In comparison to the cost of construction of such facilities by the government, the costs were high and the time required significantly longer. Government counterparts consider the buildings of high quality and good design; nevertheless, the value for money remains questionable. This, plus significant procurement delays, meant that this particular aspect of the project could not be considered efficient. All goods were procured locally also under NCB but largely under shopping guidelines, a reasonable approach considering the mostly smaller quantities of relatively low cost items. Procurement of the larger amount of IT equipment had been planned to be conducted under ICB but time constraints prohibited this choice. It does not appear that this had a negative cost impact. The majority of consultants were procured locally; limited technical assistance, i.e. for construction and fiber optic cabling, was provided by Bank staff in addition to ongoing feedback from the project team. While the original project design had envisioned significant technical assistance to be provided by international consultants the funding shortfall led the government to opt for local consultants instead. The overall quality of technical assistance procured appears adequate but in some areas too limited (or too late), especially in construction supervision and M&E. Where international consultants may have made a difference is in TA for the development of procurement documents, architectural services, and M&E. 39 Annex 4. Bank Lending and Implementation Support/Supervision Processes (a) Task Team members Responsibility/ Names Title Unit specilty Specialty Lending/grant Preparation Carlos Ricardo Escudero Lead Counsel LEGEA Task Team Leader Robert Buergenthal Sr Counsel LEGJR Task Team Leader Rowena Margaret S. Gorospe Sr Counsel LEGOP Operational Analyst Ramesh Sivapathasundram Lead Information Officer TWICT Information Technology Martin M. Serrano Sr Counsel LEGES Country Lawyer Beth Anne Hoffman Operations Analyst LEGEN Operations Analyst Nina Bhatt Sr Social Development Specialist MNSSU Social Safeguards Bakhtigul Musaiv Safeguards Consultant EACMF Safeguards Halsey L. Beemer Consultant HDGPE Education Specialist Philip Gerald Thacker Consultant ECSP4 Infrastructure Andres M. McAlister Consultant LEGJR Records Management David I Sr Financial Management Specialist LCSFM Financial Management Junxue Chu Senior Finance Officer CTRLN Financial Management Yi Dong Sr Financial Management Specia EASFM Financial Management Daniel R. Gibson Consultant EASDE Safeguards Jinan Shi Sr Procurement Specialist EASR1 Procurement Lhagvasuren Ochir [Operations Officer EACMF Operations Analyst Supervision/ICR Robert Buergenthal Sr Counsel LEGJR Task Team Leader Beth Anne Hoffman Operations Analyst LEGEN Operations Analyst Martin M. Serrano Sr Counsel LEGES Country Lawyer Carlos Ricardo Escudero Lead Counsel LEGEA Task Team Leader Halsey L. Beemer Consultant HDGPE Education Specialist Frances M. Allen Operations Analyst LEGLE Operations Support Heike P. Gramckow Lead Counsel LEGJR Task Team Leader Sylvester Kofi Awanyo Lead Procurement Specialist EASR2 Procurement Gerelgua Tserendagva Procurement Specialist EASR2 Procurement Dulguun Byambatsoo Financial Management Consultant EASFM Financial Management Badamchimeg Dondog Financial Management Analyst EASFM Financial Management Gantuya Paniga Program Assistant EACMF Project Assistance Eva Maria Melis Counsel LEGJR Task Team Member Feng Ji Sr. Environmental Specialist EASCS Environment Zhuo Yu Finance Analyst CTRLN Controlling Xiaoke Zhai Sr Transport Specialist EASCS Transport [Minneh M. Kane Lead Counsel LEGES Country Lawyer Paul Scott Prettitore Sr Public Sector Specialist MNSPS Justice Reform Richard George Andrew Nash Counsel LEGJR Justice Reform [Ramesh Sivapathasundram Lead Information Officer TWICT InformationTechnology 40 (b) Staff Time and Cost Staff Time and Cost (Bank Budget Only) Stage of Project Cycle USD Thousands (including F No.of saff eeks travel and consultant costs) [Lending I FY07 5.1 48.25 FY08 19.7 198.35 Total: 24.8 246.6 Supervision/ICR FY09 5.8 66.91 FY10 8.4 96.66 FY11 8.9 56.75 FY12 16.8 93.07 FY13 11.5 101.64 FY14 4.4 39.87 Total: 55.8 454.90 41 Annex 5. Beneficiary Survey Results (ifany) Not applicable. 42 Annex 6. Stakeholder Workshop Report and Results In preparation for the ICR, a Stakeholder Workshop was held on June 12, 2013 in Ulaanbaatar to ascertain the stakeholders' views on the EJSSP's relevance to Mongolia's justice sector reform agenda, progress made, challenges faced, outcomes and impact. Approximately 50 individuals representing all core stakeholders attended the Workshop. The Workshop was designed to address six main questions for each of the project components and the project overall, namely: (a) Was the project design relevant to reform Mongolia's justice system? (b) Did the restructuring in 2012 bring about necessary changes? (c) How can the project successes be measured? (d) Are there any examples of efficiency improvements under the project or any examples of greater public awareness that can be gleaned from the project? (e) Were the project costs incurred reasonable and sufficient to meet the project's objective ("value for money")? The stakeholders' comments generally focused on those components they were involved with more closely. Particularly towards the end of the discussions, when the audience felt more comfortable, comments were made as to the overall project impact, challenges and areas that could have been handled more efficiently. In particular this included: * Overall great satisfaction with how well the project and individual activities were linked to overall reform/change strategies, the flexible approach taken, being open to testing and innovation and bringing stakeholders together; * The project made important contributions to moving the sector forward; * Recognition that initial delays impeded stakeholders ability to engage early enough; earlier mobilization of stakeholders would have resulted in greater impact; * Recognition that procurement processes took too long; * Recognition that Bank processes and their impact on activities were not well understood; a feeling that Bank processes are too complex and lengthy, especially related to construction and ICT; * Recognition that an understanding of the importance of data collection for assessing success was lacking; * Great disappointment with the low rating of the project, limited understanding why this is justified since good progress has been made in many areas and good support for developing the needed data by the end of the ICR grace period. Public Legal Education and Law School Education: A representative from the MOJ noted that the Ministry is responsible, by law, to provide legal education to all people. While this "mandate" has existed since 1993, a lack of funding has made fulfilling this mandate difficult despite intermittent support from other donors. At the beginning of the project, the Ministry undertook a media reach survey to analyze the use of information by media type and demography to better target public education efforts. Subsequently, the 43 Ministry produced and targeted information on the internet for the population in their 20s (and younger); information in newspapers for the age group 25-35, and information on TV for those over 35 years old. The MOJ representative noted that the Bank financed a number of pilot outreach activities, targeted to different groups in the country. In undertaking these pilots, the Bank project provided the Ministry with actual cost information. The Ministry has stated that they will use the lessons learned for the public education activities for the public education activities, as well as the cost data over the longer terms to continue this important effort. A number of speakers highlighted the important contribution the project made toward creating standards for law schools in Mongolia and improving law school education overall. When the country began its transition to a market economy in the 1990s, the National University Law School (NUL) was the only law school in existence and law school education followed the socialist approach, the only approach all professors were educated in. In the coming decades, privatization triggered an explosion of new law schools being launched. Without any standards or comprehensive accreditation and testing programs in place, the new law schools provided legal education of varying quality - leading to a growing sense that certain standards needed to be created. The law school representatives noted that the project contributed to the development of the new standards, as well as modern curricula and syllabi, as well as accompanying handbooks for both students and teachers, none of which had been available before. This material also greatly contributes to quality courses and more uniform content. The material and accompanying training provided law professors with an understanding of the minimum standards expected of them; in addition, a number of new, relevant courses have been added to the curricula of many law schools. As for the legal clinics, it was noted that in the past only two legal clinics existed, initially created with support of USAID. The stakeholders recognized and appreciated the Bank's foresight to work with all law schools that wished to establish a legal clinic even before the law required clinical legal education, thereby preparing them for accreditation. As part of the legal clinic activities supported by the Bank, one law school created a substation legal clinic in the ger districts. Given the need for legal assistance in these poor districts and the perceived overall success, the substation legal clinic will be continued without further assistance. This substation, as well as the one created in the Narantuul market area, were seen as particularly helpful for introducing new laws to the often less educated, poorer populations living there, as well as in providing general legal advice to the poor. It was widely noted that also the students appreciate the practical experience the clinics provide them. Specifically related to the work with the Kazakh and Mongolian minorities, a representative from the Kazakh minority population noted that for over 20 years, there was little understanding of Mongolian law by the Kazakh minorities. Many were and are herders living in rural areas, and did not even have a basic understanding of their rights and responsibilities. Most of these population groups live 180 km and more away from 44 the Aimag centers; many are reluctant or lack the means to travel to the Aimag centers to obtain services, and many had not obtained an ID card. This is one important service facilitated by the legal aid centers supported by the Bank. Under the project, 14 laws were fully translated and additional legal information was provided in Kazakh. Furthermore, a handbook on how to use the law (titled "Your Legal Guide") was translated into Kazakh, and included phone numbers for any follow-up advice. Now, with the growth of cell phone technology across the country, these populations too got increasingly more proactive in calling a lawyer when they need legal advice; this is attributable, in large part, to the existence of the Kazakh legal aid centers. Considerable outreach took place under the project to improve these minorities' legal knowledge using a variety of means - print materials, radio, television, etc., as well as personal visits from the legal aid center to the soums. There are about 10 magazines/newspapers in Kazakh in circulation in Mongolia, and the legal clinics used them for dissemination of the laws in Kazakh and other information. There are also three different Kazakh websites in Mongolia and the translated version of the Mongolian laws are now posted on these websites. This component also included considerable outreach activities for Mongolia's blind population. The National Association of the Blind noted that they had taken a rights-based approach in the application/use of the project activities. As the project financed the publication of a number of 7 laws in braille, the project was seen as an important step towards providing Mongolia's blind population with access to the laws and international conventions regarding their rights. In the past, laws were not published in braille due to their relatively prohibitive costs for local organizations to produce. The braille version of the UN Convention on the Rights of the Child was viewed as particularly important. Many of Mongolia's blind can now demand the rights they are entitled to, particularly as they relate to the Law on Social Welfare and Insurance. The Association has branches in each Aimag, making dissemination easy. As a result, the impact of these particular activities was quite large. Fiber optic connections: A representative of the MOJ noted that the full deployment of the fiber optic network will allow for on-line videoconferencing and VolP (Voice over Internet Protocol) telephone. This will mean that the sector will save considerable money on phone and postage, and enjoy considerable efficiency gains. In particular, the MOJ noted that the development of the Uniform Information Network (UIN) initially only for UB-based agencies, that was supported by the Bank and other donors during the previous projects and extended to all Aimags under the current project, provides the basis and needed infrastructure for the current MOJ's plan to create a sector-wide data exchange network. Similarly, the network, along with software developed, now forms the backbone of the CDEA's 2012-2016 IT policy. Furthermore, there has been considerable attention paid to the governance of this network. CDEA: Prior to the World Bank project, no donor had worked with the Court Decision Enforcement Agency. According to the law on civil service, all civil servants are to be given proper (physical) working conditions. A representative from the CDEA's 45 management noted that this was not the case for years since all CDEA offices, especially in the Aimags, were located in old dilapidated buildings. The government used its own funds to build new facilities in UB but investments in the Aimags were lacking. The project has provided improved working conditions (offices) for the CDEA agents. The CDEA noted that there is low staff retention within the CDEA, which puts pressure on the agency to provide (basic) training on a continuing basis. There is a considerable need for a CDEA training program and center to ensure consistency of processes and decisions, despite high levels of turnover. For instance, the CDEA representative noted that in the past, the public had complained about the agent's attitude and professionalism. During the last phase of the project, all CDEA agents were brought together for a 2-day training event in Ulaanbaatar in April 2013. Covered in the training was information regarding the relevant laws, as well as specific sessions on ethics, stress management, and conflict resolution. Court and CDEA Construction: A representative from Intersoum Court # 2 spoke about the improved working conditions of his court. He mentioned that prior to the project, the court was in poor working condition and had not been physically modified since socialist times. As a result, there were many people that were afraid to come to the court or use the court. Now that the court is located in a new and modern facility, the users' attitude has changed. Furthermore, he noted that the court staff was more motivated to work. At the end of the Bank workshop, the Bank team asked the stakeholders in hindsight, what they would have done differently. For the most part, the stakeholders felt that the project had accomplished a great deal, and were disheartened to hear that the overall project ratings were low. They applauded the project's flexibility in meeting the evolving policy and needs of the sector. While many responded that the project responded well to the needs of a sector growing and adapting to new circumstances, a few ideas for improvement were provided. There was some consensus around the fact that all the project stakeholders should have been more efficient in implementation, to avoid the delays in seeing real results under the project. The stakeholders also posited that the Bank needed a better understanding of the problems the sector was facing when beginning to design the project to arrive at more realistic concepts for project activities. Much of what had been envisioned at the start of the project is only possible now that a good foundation is laid - now is the time that good external support can make a real difference. The CDEA noted that since this was their first time working with a donor, they did not fully understand how to engage. They stated that they would have begun their efforts much earlier, realizing the benefits of the project. In particular, they would have liked to see other models and examples from other countries to better understand how to reform their own system. 46 The stakeholders representing the Law Schools noted that the project's greatest contribution has been its intellectual investment. They noted that there had been many societal changes within Mongolia since the advent of the project. The scope of the project expanded with time, as professors needed more knowledge to keep up with larger socio-economic changes taking place in Mongolia. The GCC noted the problems with the building construction. As per the original plans, the design was first undertaken, and then the cost estimates were developed. With the delays in the construction, the raw material costs rose quickly. Similarly, a number of speakers felt that the budget for the outreach activities to the Kazakh and other ethnic minorities, as well as the rural populations, was not sufficient. 47 Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR To be delivered by the MOJ 48 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders Not applicable 49 Annex 9. List of Supporting Documents Mongolia - Enhanced Justice Sector Services Project: Environmental Report (Vol. 1 of 2) ((Mongolian), January 30, 2004 (Report No. E1876) Mongolia - Enhanced Justice Sector Services Project: Project Information Document, February 8, 2007 (Report No. AB28 71) P. Gerald Thacker, Mongolia Justice Reform Project: Next Phase - Recommendations for Facilities Component, October 2007 Mongolia - Enhanced Justice Sector Services Project: Integrated Safeguards Data Sheet, April 16, 2008 (Report No. AC3564) Mongolia - Enhanced Justice Sector Services Project: Indigenous Peoples Plan: Draft Ethnic Minority Development Plan, April 22, 2008 (Report No. IPP288) Mongolia - Enhanced Justice Sector Services Project: Project Appraisal Document, June 6, 2008 (Report No. 44059) Financing Agreement, C4493-MN and Grant H413-MN Conformed, October 7, 2008 Mongolia - Enhanced Justice Sector Services Project - Mongolia: P101446 - Implementation Status Results Report: Sequence 03, February 26, 2011. Mongolia - Enhanced Justice Sector Services Project: Environmental Report (Vol. 2 of 2): Environmental Safeguard Guidelines for Small Civil Works, June 22, 2011 (Report No. E1876) Mongolia - Enhanced Justice Sector Services Project - Mongolia: P101446 - Implementation Status Results Report: Sequence 04, December 19, 2011 (Report No. ISR4047) Mongolia - Enhanced Justice Sector Services Project: Restructuring, April 10, 2012 (Report No. 67799) Mongolia - Enhanced Justice Sector Services Project: Amendment to the Financing Agreement for Credit 4493-MN and Grant H413-MN, April 30, 2012 Mongolia - Enhanced Justice Sector Services Project: Mid-Term Review, May 2012 Mongolia - Enhanced Justice Sector Services Project - Mongolia: P101446 - Implementation Status Results Report: Sequence 05, July 1, 2012 (Report No. ISR6993) Mongolia - Enhanced Justice Sector Services Project - Mongolia: P101446 - Implementation Status Results Report: Sequence 06, January 2, 2013 (Report No. ISR8409) Mongolia - Enhanced Justice Sector Services Project: Integrated Safeguards Data Sheet, May 3, 2013 (Report No. ISDSR3000) Mongolia - Enhanced Justice Sector Services Project - Mongolia: P101446 - Implementation Status Results Report: Sequence 07, June 25, 2013 (Report No. ISR10657) 50 Annex 10. Project Activities per Component (original and revised) This annex depicts the changes made to the original project with the April 2012 restructuring. The text depicted in the right column in red italicized text represents activities added as part of the restructuring. Text depicted in strikethrough and yellow highlight (shaded) represents activities that were dropped as a result of the restructuring. PDO (Original): PDO (Revised): The Project's Development Objective is to The Objective of the Project is to Enhance Enhance the Efficiency, Transparency and Access to Justice Sector Information and Accountability of Justice Sector Legal Advice, and Increase the Efficiency Institutions. and Transparency of the Courts and Court Enforcement Agencies. Enhancing Public Legal Education on the Justice Sector 0.25m IDA 0.25m IDA 0.25 PHRD a) Formal Legal Education a) Formal Legal Education b) Public Legal Education b) Public Legal Education c) Bilingual Legal Information Program for the c) Bilingual Legal Information Program for the Kazakh (Bayan-Ulgii Aimag) Kazakh (Bayan-Ulgii Aimag) Increasing Transparency through Improved Access to Legal Information 1.26m IDA 0.55m IDA 0.15m PHRD a) Joint National Justice Sector Information a) Joint National Justice Sector Information Strategy Strategy b) Information sharing amongst justice sector b) Information sharing amongst justice sector institutions and with the general public institutions and with the general public c) IT Networking and Capacity Building for the c) IT Networking and Capacity Building for the Judiciary and the MOJHA Judiciary and the MOJ1A d) Automating and linking all regional Aimag d) Automating and linking all regional Aimag courts courts e) Central Judicial Archive and Public Information e) Cenflal Judieial Arehive and Public Information System System Enhancing Judicial Operations, Enforcement and Monitoring of Court Decisions 3.34m IDA 3.66m IDA 0.45m PHRD 1.0m GOM a) Refurbishment or Construction of select rural a) Refurbishment or Construction of select rural Aimag CDEA offices (11) Aimag CDEA offices (3) b) Nationwide CDEA IT network b) Construction of GCC court buildings (4) and c) Training for CDEA staff extension and refurbishment of GCC court d) Construction of a new Supreme Court facility building (1) c) Fiber Optic Connections to the local court offices, GCC, CDEA buildings, PGO, and Prosecutor Offices d) Supply and Installation of Administrative Case Management and Monitoring Software e) Training for CDEA staff f) CefnSttfin ef a neW SuMremfe Ceut facilitl, Annex 11: Supporting Documentation Table I a-1: List of Syllabi and Handbooks Developed ItrainlPiaeLw 150 160 140 50 110 150 50 50 50 40 50 50 50 10 150 160 140 50 110 150 50 50 50 40 50 50 50 10 BuiesLw150 160 140 50 110 150 50 50 50 40 50 50 50 10 Scooy150 160 140 50 110 150 50 50 50 40 50 50 50 10 Aplcto fLw150 340 310 100 220 310 50 100 100 80 90 100 90 10 LwesEtis150 180 150 50 110 160 40 40 50 40 40 50 40 10 SoilWlaeLw150 180 150 50 110 160 40 40 50 40 40 50 40 10 SeuiyLw150 180 150 50 110 160 40 40 50 40 40 50 40 10 BnigLw150 180 150 50 110 160 40 40 50 40 40 50 40 10 Thoyo vdne150 180 150 50 110 160 40 40 50 40 40 50 40 10 150 180 150 50 110 160 40 40 50 40 40 50 40 10 CmaaieLw150 180 150 50 110 160 40 40 50 40 40 50 40 10 FoescSine150 180 150 50 110 160 40 40 50 40 40 50 40 10 Ciiooy150 180 150 50 110 160 40 40 50 40 40 50 40 10 Noay150 180 150 50 110 160 40 40 50 40 40 50 40 10 Cor eiin150 180 170 50 110 160 0 50 50 40 40 50 40 10 52 10 10 10 5 11nt Law 3eneral) 150 180 170 50 110 160 0 50 50 40 40 50 40 10 aw 150 180 170 50 110 160 0 50 50 40 40 50 40 10 s 150 180 150 50 110 160 40 40 50 40 40 50 40 10 Ciiesi and Migration 150 180 150 50 110 160 40 40 50 40 40 50 40 10 sh 85 95 87 42 45 83 23 22 25 22 23 23 23 0 Administrative 150 180 150 50 110 160 40 40 50 40 40 50 40 10 Table I a-2: Enrollment numbers in Standardized Courses at Law Schools NUMBER OF STUDENTS ENROLLED in School Year of 2013-2014 #of #of Stude Profe nts ssors/ 3 33 enroll Teach 3 E ed ers å-"3 y W. -tn eyi cz (total) (total) ä! j 3 rE -t . lo u j . : a r . 5 = o > M .- c - r- .o 2 00 uj.~< E 0 CL 0 F C0 0. > National University of 774 51 6461 Mongolia 53 Law w, < a, a,> .> . > c, c, _ . > _ _ . 0 . .>; _4 _ >4 Enforcemen 1503 142 .o n o n f, c n n 7l ri 0000 6779 <> <> <> Lnl w Lnl mf cu w eJ mf cu> w w2 w> ai mf en w Lnl <* -i t University i i a e iu- > > Otgontenge 0D 04 CD4 0 0 1-4 0D 1- 'ý- r, o 0 0:: 1 38 nn 392 21 m a m . m m 00 m m m n * * 0 C Ln3081 University 0 0 m m 00 Orkhon , s u604 Ere me m 16 Ln Ln Ln f l 23 Institute khZsg00 t.0 00 (D0 0 00 (D0 kD* * . 00 00 00 00 0 0 t.0 0 00 00 *? 0 154 Zasa riJ 0 C,4 0 On -i 0o 0 rIq C4J C114 1-4 m~ m 0 a) r14 1-4 0 8682 University (fl w ai rnf > <> . f f en > fn efl ai efn N (D0 000 Ln0 4389 t ( Un1vers6ty-4 -1f 0~ cn - 0 -1> 0- - Biignsitu te - 11 15 1284 Ulaanbaatar- 31 142>238 Erdem 0 . 0Ln r 0 0 0 * ~ ~ ~ 23 173 152> -45< Institute5 Bilig Institute 151 15 00 e :1m e n m en m e :1 e t m L 1284 Ulaabaatr Ln o Ln o -i> > Ulaabaatr 00co0 00 Ln~'0 ~ '0W 0 . 2381 University -41 <2> 0i 00 N 0 00 r- >< i ' r,. T173e 15< - 1015 Institute - 54 0000000Go 0 D0 00 00 00 0k0( . 0 0 Sutal Institute 100 7 r 1 596 Naioa 529 22 0)0 00 0 0. D U D Mongolian > National 529 22 2958 University W c w w Trade School q 0 rq Ln r i -4 r J CN q CIJ 0 0 q IN Ln 4-' of NUM 98 7 5 7 42 Khovd School 105 9 j 0 0s j q U LA722 W ~ ~ .; V4WTrMq*M 00 4DN4-q NO4N W N r- 000 ND t0) V4 L) P) M~ V0) n r% r r % a 6505 402 N T 0) m oN 0 42689 rq0 V40 "4 N4 rq V4 " 00 3 L V4 rq -4 "4 N- 0q V-0- "I 4 N 0Rr468 Table I a-3: Satisfaction Survey for Syllabi and Handbooks Developed (Teachers, respectively Students) # of Survey Participants Total: 38 Thereof Female: 27 Thereof Male: 11 # of Law Schools participating 10 Select Topic _How often do you use the handbook? Often Usually Sometimes Constitutional Procedure Law 6 3 International Private Law 6 5 Business Law 6 2 Legal English 10 __ 55 When do you use the handbook? (multiple choice) In the classroom For my assignments Other Constitutional Procedure Law 8 7 2 International Private Law 9 8 2 Business Law 7 7 1 Legal English 8 9 1 How would you rate the following aspects of the handbook? Main contents Methodology Design 0000 0 0 5 0 0 u 0) 0 = -- o --o =~ o r- LLi- LL x Constitutional Procedure Law 5 4 -- -- 5 3 1 -- 7 2 -- -- International Private Law 8 3 -- -- 10 1 -- -- 9 2 -- -- Business Law 5 3 -- -- 4 3 1 -- 5 3 -- -- LegalEnglish 6 4 -- -- 7 3 -- -- 8 2 -- -- # of Survey Participants Total: 175 Thereof Female: 120 Thereof Male: 55 Constitutional Procedure Law International Private Law (55/40/15) Business Law (54/34/20) Legal English (10/8/2) (56/38/18) How often do you use the handbook? Often Usually Sometimes Constitutional Procedure Law 4 18 36 International Private Law 14 18 26 Business Law 3 13 40 Legal English 2 4 4 56 When do you use the handbook? (multiple choice) In the classroom For my assignments Other Constitutional Procedure Law 31 26 18 International Private Law 37 24 15 Business Law 34 12 24 Legal English 2 5 4 How would you rate the following aspects of the handbook? Main content Case studies References Contents Design = 0 - = o - = 0 - = 0 - = 0 0 (U 0 0 () 0 0 a) 0 0 a) 0 Cu 0 a) 0 L.L 0 U(D LL L9 LL U L9 LL U (D U- U )a xxx x x U oo U oU U U Constitutional Procedure Law 11 30 12 1 14 24 15 1 10 28 15 -- 1 1 -- International Private Law 23 21 11 1 19 26 10 -- 18 26 7 2 -- 1 1 -- Business Law 7 25 20 2 7 33 10 4 8 28 12 3 -- 1 -- -- 1 -- LegallEnglish -- -- -- -- -- -- -- -- -- -- -- -- 3 4 2 -- -- 7 2 Table I a-4: Trainings Provided for Law School Professors and Teachers Trainng o Traners2011 30 10 20 Stud Tou-Gerany2011 19 7 12 Lega EngishTeacing2011 24 2 22 Methodology Training 2013 100 43 57 Research Methodology 2013 130 58 72 Application of Law /Constitutional Law, Administration Law, Civil Law, Criminal and Criminal Procedural Law 2013 150 67 83 57 Junior Lawyers Training /Organized with MOJ, NLI/ 2013 50 26 24 Table I a-5: Satisfaction Survey for Trainings Provided for Law School Professors and Teachers 1 I- L- L.L.M A1 1 2 1 0 0 0 ?4.1 4.. 4.1 4.1 4.1 4.1-4-4 Law, Criminal Law, L 'E E EEh E twj to Training~~ of Trinr 30 22 19 3 0 2 1 3 1 W9 3 8 4 1 UI( >. .-M Stud TorGemn 19 4- 4- 4- 00 YogPofor Trainng 1746 AdnsrarMtodowvl 130 113 110 3 108 12 109 11 100 13 101 12 106 13 105 15 AplatingofTawr 150 130 123 3 120 2 124 3 115 15 124 3 125 4 119 11 58 /Constitutional Law, Administration Law, Civil Law, Criminal and Criminal Procedural Law Junior Lawyers Training 50 50 48 2 49 1 44 6 42 8 39 11 48 2 38 12 /Organized with MOJ, NLI Table I a-6: Case Details for Law School Clinics Total Number of people served by Legal Clinics Case Type of Legal Aid Application List of Law Schools ofC lial 2013 (until Sept 30, Social iis Standards Family Land Welfar Labor trai Insur Other Legal Clinic Femalbaata Male FenlTotal Male Fenl Total e n ance e e Legal Clinic, School of Commerce V 8 2 9 3 8 9 9 1 2 7 85 4 of NUM Lea Ciic awSholofV 33 17 so 25 20 45 21 10 6 18 0 5 35 Otgontenger University Legal Clinic, Ikh Zasag University V/ 165 200 35 175 325 50 305 221 131 86 25 34 63 Legal Clinic, Ulaanbaatar Erdem28 5 79 4 75 17 6 298 17 4 293 University Legal Clinic, Orkhon University V/ 30 21 51 35 32 67 25 18 11 13 9 17 25 Lea .lnc UlabatrV 35 14 49 43 18 61 45 8 17 18 12 8 2 UniversityI IIIIIIII Legal Clinic, Sutai University / 24 38 39 65 41 25 19 17 19 21 24 Legal Clinic, National Institute of v 45 18 27 13 23 8 13 26 1 18 14 59 Mongolia Legal Clinic, National University of 85 115 110 165 Mongolia, Law School Legal Clinic, Shihikhutug 35 29 64 80 100 180 53 106 29 34 11 1 10 University Bilig Institute 27 46 73 31 57 88 51 27 17 21 9 16 20 Tushee Institute V 15 21 36 26 38 64 33 13 9 15 8 9 13 Law Enforcement University V Doesn't serve the public Table I a-7: Case Details for the Legal Clinic Substations in the poor ger districts in Ulaanbaatar "Justice Bridge" Ger District Project Legal Clinic, National Sukhbaatar . . District 13, 15, 1 2 6 6 212 49 163 31 113 18 50 University of Mongolia 1t oo 18th Horoo Chingeltel Legal Clinic, Shibikhutug Cigle .eg .iic District 11, 15, 1 2 28 32 244 93 151 66 109 27 45 18th Horoo "Born under the Sky, Live under the Law" Narantuul Market Project Legal Clinic, Ulaanbaatar Narantuul2 .nier.t Market 2 1 5 7 1810 590 1220 -- - -- Universitymecat merchants 60 Numbe of eopl whoLegal Aid Received per Case Type received Legal Aid Legal Clinic Civil Social Total Male Female Family Land Wlae Labor Registra Insuranc Contract al Other tion e University of Mongolia 68 15029985433 Legal Clinic, Shihikhutug 72 27 45 21 11 6 11 7 3 2 8 3 University Table I a-8: Tailored Projects "Legal Rights Awareness and Crime Prevention" at High Schools Unvibert Oct 01, 2012os Nubro awSuet NmefHihSho Legal Clinic, School of Commerce of theSchool u project4262 National University of Mongolia Legal Clinic, Law School of Otgontenger School Unviersity Oct 01, 2012 Legal Clinic, Ikh Zasag University 97 th School until project 2 1 7 8 48 72 Legal Clinic, Ulaanbaatar Erdem 5 h Scolclosure1348673 Unviersity th closure 1 3 Legal Clinic, Orkhon Unviersity 18th School 1 2 5 7 64 84 Legal Clinic, Ulaanbaatar University 84 School 2 1 7 6 79 121 Legal Clinic, Sutai University 33th School Dropped Oct 01, 2012 Legal Clinic, National Institute of th S ot poe 3491 Mongolia11 School until project 0 3 4 6 49 161 Mongoliaclur closure 61 Table I b-1: Public Education Campaigns "Law and Integrity" - October/November 2010 MoJHA Arkhangai 90,986 +,nor further information Bayan-Ulgii 89,191 2 Bayankhongor 80,858 +,nor further information Bulgan 58,369 +,nor further information Govi-Altai 56,172 +,nor further information Govi-Sumber 14,508 +,nor further information Darkhan-Uul 95,043 +,nor further information Dornogovi 59,772 Dornod 74,473 +,nor further information Dundgovi 45,914 2, total of 5,000 participants Zavkhan 71,877 +,nor further information Orkhon 88,705 Umnugovi 52,306 +,nor further information Uvurkhangai 112,120 +,nor further information Sukhbaatar 54,852 +,nor further information Selenge 101,752 +,nor further information Tuv 87,909 16 Uvs 74,733 4 11 Khovd 82,204 Khentii 70,143 365 5 Khuvsgul 124,613 +,nor further information Ulaanbaatar 8 7000 62 "Peaceful Environment" - January/February 2011 MoJHA, N LC and Globe International (NGO) Arkhangai 90,986 Bayan-Ulgii 89,191 4(100) Bayankhongor 80,858 3 6(40) Bulgan 58,369 Govi-Altai 56,172 Govi-Sumber 14,508 3 2,000 7 Darkhan-Uul 95,043 18 20 Dornogovi 59,772 27 Dornod 74,473 Dundgovi 45,914 5 2(61) Zavkhan 71,877 Orkhon 88,705 Umnugovi 52,306 Uvurkhangai 112,120 5 Sukhbaatar 54,852 7 5 Selenge 101,752 Tuv 87,909 12 21,000 1 (2000) Uvs 74,733 Khovd 82,204 Khentii 70,143 15 Khuvsgul 124,613 1,000 Ulaanbaatar "Environment Protection" - May/June 2012 63 MoJHA Arkhangai 90,986 Bayan-Ulgii 89,191 Bayankhongor 80,858 Bulgan 58,369 Govi-Altai 56,172 Govi-Sumber 14,508 2 1 (300) 1,647 757 Darkhan-Uul 95,043 Dornogovi 59,772 30 29(5,655) 33,024 11,410 Dornod 74,473 Dundgovi 45,914 7(4,800) 19,300 Zavkhan 71,877 Orkhon 88,705 Umnugovi 52,306 Uvurkhangai 112,120 3 12 (7,865) 57,436 Sukhbaatar 54,852 Selenge 101,752 Tuv 87,909 Uvs 74,733 Khovd 82,204 Khentii 70,143 60 1 (200) 2,250 Khuvsgul 124,613 7 15,762 Ulaanbaatar "Family & Labor" - April/May 2012 MoJHA Arkhangai 90,986 1 3 Bayan-Ulgii 89,191 6 1 (345) Bayankhongor 80,858 2 2 (245) 64 Bulgan 58,369 14 (2,800) Govi-Altai 56,172 Govi-Sumber 14,508 1 300 6 Darkhan-Uul 95,043 Dornogovi 59,772 Dornod 74,473 2 3 (600) Dundgovi 45,914 2 3 Zavkhan 71,877 Orkhon 88,705 Umnugovi 52,306 80 2,000 152 (3,055) Uvurkhangai 112,120 8 (872) Sukhbaatar 54,852 120 2 (564) Selenge 101,752 6 4 (2,020) Tuv 87,909 17 600 63 (25,400) Uvs 74,733 Khovd 82,204 5 500 5 (500) Khentii 70,143 9 5 (607) Khuvsgul 124,613 5 200 Ulaanbaatar Table I b-2: Public Education Materials (Distribution) # of # of # of Handout #oPstr# of # of Posters # of Posters # of Posters Soums Khoroos "Your Legal Animated "Anti- "Anti- "Constitutional Aimg opuaton in in UIB Guide" mnad Schoolbooks Tobacco" Corruption" Rights" Aimag Districts Distributed ww.huhm Distributed Distributed Distributed Distributed n distributed Arkhangai 90,986 19 476 238 50 50 50 Bayan-Ulgii 89,191 14 392 196 50 50 50 Bayankhongor 80,858 20 504 252 50 50 50 65 Bulgan 58,369 16 368 184 50 50 50 Govi-Altai 56,172 18 404 202 50 50 50 Govi-Sumber 14,508 3 56 28 50 50 50 Darkhan-Uui 95,043 4 124 62 50 50 50 Dornogovi 59,772 14 284 142 50 50 50 Dornod 74,473 14 312 156 50 50 50 Dundgovi 45,914 15 328 164 50 50 50 Zavkhan 71,877 24 556 278 50 50 50 Orkhon 88,705 2 88 44 50 50 50 Umnugovi 52,306 15 292 146 50 50 50 Uvurkhangai 112,120 19 304 252 50 50 50 Sukhbaatar 54,852 13 320 160 50 50 50 Selenge 101,752 17 296 148 50 50 50 Tuv 87,909 27 496 248 50 50 50 Uvs 74,733 19 248 224 50 50 50 Khovd 82,204 17 436 218 50 50 50 Khentii 70,143 20 436 218 50 50 50 Khuvsgul 124,613 24 596 298 50 50 50 Legal Aid Centers in 6 Soums 300 300 300 Ulaanbaatar Bayangol District 192,100 46 96 48 50 50 50 Bayazurkh District 283,300 56 116 58 50 50 50 Khan-Uul District 119,900 32 68 34 50 50 50 Chingeltei District 153,100 38 80 40 50 50 50 Songino-Khairkhan 257,200 64 132 66 50 50 50 District Sukhbaatar District 137,800 40 84 42 50 50 50 Baganuur District 27,000 10 24 12 50 50 50 Bagakhangai 3,700 4 12 6 50 50 50 District Nalaikh District 32,500 14 32 16 50 50 50 66 14 Law Schools 130 28 14 Ministry of Justice 486 100 100 100 National Legal Institute 100 100 100 11-11 Information Center 100 100 100 Secondary Schools of Ulaanbaatar 2,180 Orphanages and Homes for Street and Unsupervised 390 Children (39) Training & Public Awareness Unit of the Municipal Police 20 Department Protection of Public Order Unit of the Municipal Police 320 Department The Child Protection & Address and Identification Center 20 (state-run center under the police department) Child & Family Support Department of Ulaanbaatar 20 National Center for Children 20 153 Khoroos (administrative units of Ulaanbaatar) 1,530 Table I b-3: Kazakh Legal Aid Center (Bayan-Ulgii Aimag) Legal Aid received by Participants in Legal Total Number of PeoplePepeAds/RahdbAg Term Gender Outreach Campaigns Addressed/Reached Male Female Male Female 18-35 36-55 Above 55 Noeme 210- 109 113 395 132 749 225 395 129 December 2011 Deeme 211- 713 794 1,925 1,867 5,299 2,048 1,823 1,428 December 2012 January 2013 - Ju577 514 4,374 3,502 8,967 734 5,098 3,135 June 2013 67 Table I b-4: Legal Aid Center for Ethnic Minorities (Bayan-Ulgii Aimag) Ja ernuar 201 - un November2010-144 168 57 193 62 312 November 2011 December 2011 -85614 Deebr 01-187 175 88 180 94 362 December 2012 January 2013 - June 73 81 40 50 64 154 2013 Tablel Feal b-:PaatglAciite November 2010 - No aie uen November 2011 December 2011 -85614 December 2012 January 2013 - June 146 117 263 20131111 Table I b-5: Paralegal Activities Numbe r of Number of trained Number of trainings Number of Aimags Population paraleg paralegals provided for trainings & als providing services the paralegals outreach activities 21 02 21 oa trained since 2011 in 2011 Bayan-Vt2 89,191 41 4 no data No data 2,365 No data BaNankhonoor 80,858 51 23 no data No data 4,869 No data 68 36 (The Aimag's Citizens Representatives Bulgan 58,486 52 Hural approved no data 150 1,791 6,234 8,025 No data 36,902,400 tugrugs as paralegals' fee for 2014-2016 years) Govi-Altai 56,172 60 24 4 No data No data No data Govisumber 14,508 9 No activities undertaken. no data Darkhan-Uul 95,043 26 No activities undertaken. no data Dornogovi 59,772 50 23 no data 38 4,869 no data Dornod 74,473 31 18 no data no data 27,613 no data Dundgovi 45,914 65 5 no data no data 150 no data 792 (27 946(39 . mio mio. Zavkhan 71,877 99 29 no data no data Tugrugs for tugrugs (No 1,738 no data paralegals' for budget) es paralegals fees) (48 mio tugrugs Orkhon 88,705 51 36 13 17 approved 13,376 30520 for paralegal s' fees) Umnugovi 52,306 53 41 no data no data 9,584 no data Uvurkhangai 112,120 56 55 no data 502 45,978 no data Sukhbaatar 54,852 40 No activities undertaken. no data Selenge 101,752 32 32 no data no data 7,282 no data Tuv 88,000 99 318 10 58 8744 27587 1005 37,336 no data Uvs 74,733 77 No activities undertaken. no data Khovd 82,204 61 No activities undertaken. no data Khentii 70,143 74 19 no data no data 3,848 no data Khuvsgul 12,4613 76 12 no data no data 634 no data 69 Ulaanbaatar Bayangol 192100 36 36 no data 1 150 150 no data District Bayanzurkh 283300 50 50 no data 7 370 370 no data District Khan-Uul 119900 42 42 no data 1 120 120 no data District Chingeltei 153100 58 58 no data 25 480 480 no data District Songino- Khairkhan 257200 63 63 no data 1 160 160 no data District baatar 137800 72 72 no data 2 100 100 no data District Baganuur 27000 22 22 no data 1 64 64 no data District Bagakhangai 3700 12 12 no data 2 57 57 no data District Nalaikh 32500 16 16 192 192 no data District Table II a-1: Sites Connected and Data Sharing Activities Fiber Data entry Aimag EniyOptics (as applicable: CDEAReak Entity . Reak (Province) Connecti software/Administrative Court on Software) Administrative Court Not regularly (Nov 1, 2013) 70 Aimag Court / Bayan-Ulgii CDEA (Aimag) / At least once a week Administrative Court / Not regularly (Nov 1, 2013) Aimag Court / Bayankhongor CDEA (Aimag) / At least once a week Administrative Court / Not regularly (Nov 1, 2013) Aimag Court / Bulgan Administrative Court / Not regularly (Nov 1, 2013) Aimag Court / Govi-Altai CDEA (Aimag) / At least once a week Administrative Court / Not regularly (Nov 1, 2013) Aimag Court / Govi-Sumber CDEA (Aimag) / At least once a week Administrative Court / Regularly Software Pilot Court Aimag Court / Darkhan-Uul CDEA (Aimag) V/ At least once a week Administrative Court / Regularly Software Pilot Court Aimag Court / Dornogovi CDEA (Aimag) / At least once a week Administrative Court / Not regularly (Nov 1, 2013) Aimag Court / Dornod CDEA (Aimag) / At least once a week Administrative Court / Not regularly (Nov 1, 2013) Aimag Court / Dundgovi CDEA (Aimag) / At least once a week Administrative Court / Not regularly (Nov 1, 2013) Aimag Court / Zavkhan Tonsontsengel Inter-Soum Court / Newly constructed CDEA (Aimag) / At least once a week Administrative Court / Not regularly (Nov 1, 2013) Orkhon Aimag Court / 71 CDEA (Aimag) / At least once a week Administrative Court Regularly Software Pilot Court Aimag Court / Khorkhorin Inter-Soum Court / Newly constructed Uvurkhangai CDEA (Aimag) / At least once a week Khorkhorin Soum CDEA / At least once a week Administrative Court / Not regularly (Nov 1, 2013) Aimag Court / Refurbished Umnugovi CDEA (Aimag) / At least once a week Administrative Court / Not regularly (Nov 1, 2013) Aimag Court / Sukhbaatar CDEA (Aimag) / At least once a week Administrative Court / Not regularly (Nov 1, 2013) Aimag Court / CDEA (Aimag)/ At least once a week Mandal Sourm CDEA / At least once a week Administrative Court / Regularly Software Pilot Court Aimag Court / Tuv CDEA (Aimag) / At least once a week Administrative Court / Regularly Software Pilot Court Aimag Court / Uvs Administrative Court / Not regularly (Nov 1, 2013) Aimag Court / Khovd Bulgan Inter-Soum Court / Newly constructed CDEA (Aimag) / At least once a week Administrative Court / Not regularly (Nov 1, 2013) Aimag Court / Khuvsgul CDEA (Aimag) / At least once a week Administrative Court / Not regularly (Nov 1, 2013) Aimag Court / Khentii Bor-Undur (Darkhan) Inter-Soum Court / Newly constructed CDEA (Aimag) / At least once a week 72 Administrative Court / Not regularly (Nov 1, 2013) Administrative Court of the Capital City / Not regularly (Nov 1, 2013) Administrative Appellate Court / Not regularly (Nov 1, 2013) Administrative Cassation Court / Not regularly (Nov 1, 2013) CDEA of the Capital City / Daily Baganuur District of Ulaanbaatar / At least once a week Debt Extinguishing Office of the Banks and Savings and Credit / At least once a week Cooperatives CDEA Headquarter / Daily Building #2 of the Court Decision Enforcement Agency / Daily GCC / Not regularly Table II b-1: User Satisfaction Survey on Court Enforcement Agency Software (24 respondents) Did the training achieve its purpose? 15 - 1) No (0) 10 -2) Somewhat (0) 3) Moderately (2) 5 - 4) Almost (15) 5) Completely (7) 0 1 xyparyAI 2 6ara sapar 3 AyHA 33ar 4 6apar XypC3H 5 6yp3H XypC3H YHanlfa7 73 Was the training content consistent with the training's purpose? 20 1) No 15 2) Somewhat 3) Moderately 10 4) Almost (7) 5) Completely (17) 0 1 xyp9aryA 1 2 6ara sapqr 3 AYHA 3ar 4 6apar XypC9H 5 6yp9H XypC9H HaraYHI How would you rank the impact of following factors on the training's success? 4.6 - 4.4 1) Training methodology (4) 4.2 2) Training preparation (4) 4 3) Trainer's ability (5) 4) Training tools (4) CypranTblH X9B 3arBap CypranTblH 6anTr3 ammn Cyprari 6arwlH yp ,aABap cypralTbIH 6arai xqpqrc9.n 74 How would you rate the immediate practicality of the knowledge you obtained through this training? 15 - 1) Poor 10 2) Somewhat poor 3) Moderate (2) 4) Good (9) 5) Very good (13) 0 1 xypearyI 2 6ara sapgr 3 AYHA 3apgr 4 6apar 5 6yp9H XypCOH XypC3H Possible barriers to implementation 1 respondent: No barriers (5%) 2 respondents: Lack of experience (13%) 6 respondents: Lack of high-speed internet (38%) 7 respondents: Capacity of technical equipment (44%) 75 P0 ular res onses Indv. Total What did you enjoy during the training? 01 02 3 04,5 1 13 1 3 2 1% 1 7 1 37% 01 2 Tr r 4AauN ,3 0ted system can no parse ows and studioS [C t anIme itese odirct inqumres. I Alucions are now\ easier anl with proper monitoring.1 Itsveygod that we are implementing this new systenm that allows Inertdsystemn implementation 1 4 Table III a-1: List of Construction of CDEA offices and Court Buildings Court Buildings Bulgan Soumn 546,793 'u '1 o 0 Nov 30, 2012 Ganzoson Construction LLC ConConstruction LLC November 2012 2011 2011 Aug 22, Nov 30, ac ,21 gu hsi [ Tonsontsengel Soum 517,118 April30,2012 Asar Urguu LLC ConConstruction LLC April 2012 2011 2011 Khorkhorin Soum 452,901 'u '9 o 0 Oct 30, 2012 Inteko Engineering LLC ConConstruction LLC September 2012 2011 2011 Bor-Undur Soum 394,970 'u '2 o 0 March 5, 2012 Uguuj Khishig LLC ConConstruction LLC March 2012 2011 2011 Umnugovi Aimag 156,584 Aug 24, Nov 30, Jan 15, 2012 Batjin Construction LLC ConConstruction LLC June 2013 76 (Refurbishment) 2011 2011; extended to June 28, 2013 CDEA Offices June 27, November Nov 30, 2012; Bulgan Aimag 391,400 ' extended to June Machine Mechanism JSC OSIB-ZT LLC June 2013 2012 30, 2012 2,21 28, 2013 August November Jn30212 Tsagaan Khushuu Trade Uvs Aimag 386,087 June 30, 2012 OSIB-ZT LLC June 2012 24, 2011 30, 2011 LLC Zamyin-Uud Sou 145,371 30, Nov 30, 2011 Mongol Deveer LLC OSIB-ZT LLC November 2011 16, 2011 2011 77 Table III a-2: Survey on User Satisfaction with Newly Constructed/Refurbished Buildingsio Court building in Bor-Undur Soum (completed in March 2012) - Staff Equipment 15 19 Furniture -16 Waiting hail 11 12 02013 Administrative address 17 02012 17 Location 16 0 2 4 6 8 10 12 14 16 18 20 Court building in Bor-Undur Soum (completed in March 2012) --Lawyers Equipment 10 13 Furniture - 16 Waiting hall 13 02013 Administrative address 12 18 .2 012 Location 14 15 0 2 4 6 8 10 12 14 16 18 20 10 At each site, a total of 40 people were randomly selected and interviewed, out of which 20 were clients and 20 were lawyers. The numbers comprise the responses 'very satisfied' and 'satisfied'. 78 Court building in Bulgan Soum, Khovd Province (completed in November 2012) - Staff Equipment 1 13 Furniture 9 20 Waiting hall 16 19 02013 Administrative address 12 13 02012 20 Location -2 0 5 10 15 20 25 Court building in Bulgan Soum, Khovd Province (completed in November 2012) - Lawyers Equipment 15 15 Furniture 20 Waiting hall 18 19 M 2013 Administrative address - M2012 Location 18 19 0 5 10 15 20 25 79 Court building in Tonsontsengel Soum, Zavkhan Province (completed in April 2012) - Staff Equipment 15 Furniture 13 13 Waiting hall 15 N16 02013 Administrative address 12 14 02012 Location 1 19 0 2 4 6 8 10 12 14 16 18 20 Court building in Tonsontsengel Soum, Zavkhan Province (completed in April 2012) - Lawyers Equipment 12 17 Furniture 14 16 Waiting hall 16 18 m 2013 Administrative address 15 m 2012 Location 20 0 5 10 15 20 25 80 Court building in Khorkhorin Soum, Uvurkhangai Province (completed in September 2012) - Staff Equipment 18 Furniture 18 20 21 Waiting hall 20 Administrative address W 14 Location 15 0 5 10 15 20 25 12 Court building in Khorkhorin Soum, Uvurkhangai Province (completed in September 2012) - Lawyers Equipment 12 1 Furniture 20 20 Waiting hall - 18 19 m2013 Administrative address 20 Location 20 0 5 10 15 20 25 81 CDEA Office in Uvs Aimag (completed in June 2012) - Staff Equipment -- Furniture 418 Waiting hall 12 18 02013 Administrative address 18 02012 18 Location 1 13 0 2 4 6 8 10 12 14 16 18 20 CDEA Office in Uvs Aimag (completed in June 2012) - Lawyers Equipment 20 20 Furniture 20 20 Waiting hail 12 14 02013 Administrative address 20 02012 20 Location 3 15 0 5 10 15 20 25 Table III b-1: User Satisfaction Survey on Administrative Court Management Software (Capital City Courts) Survey Question # of Responses Answers collected 3 It's easy. 82 casc, complaint, and casc 12 It's relatively manageable. procedure information 1 It's difficult to enter data due to incomprehensible UI elements. into the software? Recommendation: Lack of edit function makes difficulties. Overall user feedback positive. Some complaints include overlong and complicated data entry procedures, repetitive processes, and unnecessary automation of certain elements. 10 Yes. Dos heALRTsyt1 No. ctlp dourmeadie? Recommendation: The system is invaluable in certain situations. For instance, declaring court meeting dates, improving the decision announcement process. Doesn't work sometimes. Is it possible to reduce 12 The "reminder" function of the software is useful to avoid procedural errors. procedural crrors 2 The "reminder" function prevents all possible procedural errors. than ks to thcALERT 1 The system fails to provide assistance that would result in a reduction of procedural errors. system? Recommendation: Possible to achieve with optimal programming. Recommendation: Include a direct link to the error in the alert message so it is easier to track. The information contained in the alert message is too comprehensive. to the ALERTI system? Docs the software help 4 Inclusion of court decisions in the software makes it easier and faster for the end user. facilitate the process of 10 Auto filled data fields such as case parties and nature of claim etc. saves time for the user. issuing decrees, orders' 4 Inclusion of example court decisions in the software is counter-productive. The judge must make their own and other official decision. documents? 2 While auto filled data fields save time, if the data gets changed, wrong data would get entered. 8 It's both concise and easy to understand 1 It violates section 72 of the Administrative Procedure Code; it should therefore be reverted. 2 While it's comprehensible, prefer another design. Recommendation: It may be best to keep the old format for entered data. Be advised that the new format is too design-centered. A function to import data from MS Word is needed. Is it possible to add a grammar proofing function? 11 We are able to retrieve wanted news and studies from the software easily and quickly. Is it fast and easy to There are problems in retrieving information from the software. retrieve necessary 1 While fast, it's sometimes impossible to retrieve all necessary data from the software. Generally positive. information and reports Recommendation: While the software is fast, it may be helpful if it was able to check if its registry has an error. from the software? Sometimes, reports come incomplete. Also, make certain admin rights available to judges, which will make it easier to conduct studies. Make it so that all judges and judges' assistants are able to access reports. Does the software 4 Yes shorten the time 2 No 83 d for the case 7 While additional time is needed to feed data into the software, it also enables quicker access to information, which ement process? has an overall positive effect. Recommendation: Once people learn to use the software better, it will get more efficient. Judges' assistants 2 currently have to use the software every hour, which is inefficient. Do technical adjustments so the software doesn't freeze, or slow down. Currently the software involves tons of user input. 9 Systems handbook is easy to understand and use. handbook casy to 4 The handbook is incomprehensible, difficult to use, and lacks necessary information Recommendation: Basically positive feedback received. Didn't have a chance to study the handbook in detail. Recommendation: Add courtrooms. It's currently impossible to know which courtroom has a court in session. re ended changes Also, color printing would make it easier to understand. tthhand book? Unnecessary (1) 2 3 4 Invaluable (5) Registration of complaints and new cases 1 3 5 8 Complaint resolution 1 2 1 4 11 Case entry 1 1 2 6 5 Announcement 2 1 3 6 3 Third party summons 1 1 2 5 5 Printing of decrees, decisions, and orders 1 1 5 7 7 Scheduling court sessions (Court calendar) 2 1 2 3 8 Compiling court and judge reports 3 3 14 Collecting evidence 1 2 5 4 4 Case transfer 1 5 8 2 Court proceedings 1 6 2 3 4 Others: case lifetime 2 Table III b-2: User Satisfaction Survey on Administrative Court Management Software (Aimag Courts) Survey Question # of Responses Answers collected Is t as t ete apel, 12 It's easy It's difficult to enter data due to incomprehensible U elements. 84 into tesoftware? Recommendation: Generally positive. No edit function makes things difficult. Does th ALEIRT system 27 Yes help yu remember 1 No critic deadlines? Recommendation: The system is invaluable. But it doesn't work sometimes. Is it sible to reduce 21 The "reminder" function of the software is useful to avoid procedural errors. proceural crrors 13 The "reminder" function prevents all possible procedural errors. thank to the ALERT 2 The system fails to provide assistance that results in reduction of procedural errors. system? Recommendation: Keep it simple. Believe it is possible. Remember to fix the error so it stops not working. Is there anything you Recommendation: Add plaintiff s deadline to give response. Make it possible to edit later, if decision number is wou like to see added entered incorrectly. If the case has clerical errors, make it so that upper level court gets a warning. Make warnings to the ALERT system? repeatable. Does e software help 23 Inclusion of court decisions in the software makes it easier and faster for the end user. facilie the process of 17 Auto filled data fields such as case parties and nature of claim etc. saves time for the user. issuin decrees, orders, Inclusion of example court decisions in the software is counter-productive. The judge must make their own and or official decision. documents? 4 While auto filled data fields save time, if the data gets changed, wrong data would get entered. Is the w and changed 35 It's both concise and easy to understand intcrfae to the court 4 It violates s.72 of the Administrative Procedure Code. Should be reverted. case, dge's order, and While it's comprehensible, prefer another design. decre section Recommendation: The fact that it's possible to pick an empty template is very useful. comprhensible? Is it t and easy to 31 We are able to retrieve wanted news and studies from the software easily and quickly. retrie necessary There are problems in retrieving information from the software. information and reports 5 While fast, it's sometimes impossible to retrieve all necessary data from the software. Generally positive. from che software? Recommendation: It's better than the previous version. Dos hesotwre23 Yes shorte the time IN 18 While additional time is needed to feed data into the software, it also enables quicker access to information, which has an overall positive effect. Recommendation: Once people learn to use the software better, it will get more efficient. 26 Systems handbook is easy to understand and use. 8 The handbook is incomprehensible, difficult to use, and lacks necessary information Recommendation: The handbook is missing some software functions. Add role responsibilities. Wha are the Recommendation: Haven't studied it thoroughly yet. Generally the handbook is easy to understand, but wouldn't recommcnded changes hurt if it were better. Add role responsibilities. Would have been better if the tasks assigned to secretaries and to the handbook? assistants were defined separately. It was helpful that the handbook was included on the help menu of the software 85 in file format Add more ontabulatd mfomation. Include all sofare fumctionselIf it's impossible edit later, make Table III c- 1: List of Trainings Provided to the Court Decision Enforcement Officer How to change the Legal Ethics Attitude of Jnay2,21 2 L Lawyers On Pension February 1, 2013 123 CDEA Protection of State Employees' Rights February 8, 2013 79 Law Enforcement University On Corruption February 15, 2013 91 Independent Authority Against Corruption On nfrcmet o AmiisratveCort February 18, 2013 80 Capital City Administrative Court Decisions Company Law February 25, 2013 79 Law Enforcement University Coercive Auction of Immovable Assets March 1, 2013 81 Representing at the Court, Gathering of Mrh1,2381Capital City Court Evilence Noncustodial Sanctions Procedure March 29, 2013 79 Capital City Prosecutor Office Reform in the Criminal Law April 5, 2013 79 Advocacy office Civil law April 12, 2013 81 Law School of Mongolian National University The Psychology of Communication April 19, 2013 123 NLI Verbal Communication April 19, 2013 123 NLI StrssMaagmet:HowtoPrvet ndApril 22, 2013 79 Lady Center, financed by EJSSP Reduce Stress Organizational Psychology April 26, 2013 122 Social Science School of Mongolian National University Archiving May 10, 2013 79 General Authority for Archive Introduction of Newly Adopted Regulations May 17, 2013 129 Capital City CDE Office Methodology of Entering Data into the My3,21 9CE Registration System Forum on Intensification of Noncustodial Spebr1,2321CDEA, Prosecutor's and Police's Office of Bayangol Sanctions Procedure District Table III c-2: Enforcement Rates 86 2010 2011 2012 As of August 31, 2013 CDEA Offices e/s 366 252 68.9 358 286 79.9 375 299 79.7 284 145 51.1 Arkhangai Aimag am 469983.5 303036.5 64.5 365039.6 278090.8 76.2 384583.2 284317.3 73.9 427883.3 195609.8 45.7 e/s 574 398 69.3 468 364 77.8 367 266 72.5 462 245 53.0 Bayan-UIgii Aimag am 722736.2 461892.8 63.9 566590.8 362950.7 64.1 1124124.1 440666.3 39.2 922601.4 138820.9 15.0 . e/s 546 360 65.9 458 307 67.0 396 310 78.3 330 187 56.7 am 759339.4 572063.3 75.3 560849.6 424236.3 75.6 387475.0 319915.0 82.6 776062.1 592819.1 76.4 e/s 427 300 403 290 352 247 426 210 573306.8 325306.2 543649.6 333167.3 750607.0 496799.4 809787.4 260933.2 e/s 469 389 82.9 365 302 82.7 342 287 83.9 241 86 35.7 Govi-Altai Almag _______ am 533085.9 426499.7 80.0 317512.9 260254.8 82.0 395750.9 326886.7 82.6 583495.9 206692.6 35.4 e/s 454 272 59.9 294 192 65.3 223 146 65.5 162 81 50.0 Govi-Sumber Almag _ am 532719.1 193396.8 36.3 351362.3 178416.5 50.8 274693.9 139583.1 50.8 236919.4 62543.3 26.4 e/s 849 561 66.1 691 458 66.3 701 467 66.6 742 272 36.7 Darkhan-Uui Aimag am 2287513.0 1662134.8 72.7 2021261.4 1475388.4 73.0 2060647.8 1526076.0 74.1 4300008.8 1493916.5 34.7 . . e/s 793 447 56.4 689 396 57.5 606 409 67.5 478 220 46.0 Dornogovi Almag am 599825.9 441115.5 73.5 510562.1 367687.3 72.0 841497.8 674772.9 80.2 1441838.3 474529.0 32.9 e/s 1475 815 55.3 1099 716 65.2 905 683 75.5 1179 767 65.1 Dornod Aimag am 611186.3 292602.7 47.9 360056.4 252608.2 70.2 660829.8 552024.9 83.5 838624.8 312105.1 37.2 e/s 482 386 80.1 379 304 80.2 359 288 80.2 244 142 58.2 Dundgovi Almag am 146742.4 106275.9 72.4 204471.6 154904.2 75.8 419419.3 354542.4 84.5 285188.5 136465.0 47.9 e/s 315 267 84.8 212 190 89.6 261 232 88.9 229 156 68.1 Zavkhan Almag _______ am 604852.4 529690.9 87.6 377409.1 333358.7 88.3 464788.6 387765.1 83.4 466525.2 226069.5 48.5 e/s 1795 850 47.4 1656 681 41.1 1725 869 50.4 1239 340 27.4 Orkhon Aimag am 5362984.4 3019240.4 56.3 5924652.0 3442914.7 58.1 5872678.3 2667306.6 45.4 13727463.5 9043654.5 65.9 e/s 680 613 90.1 907 847 93.4 585 551 94.2 473 331 70.0 Uvurkhangai Almag am 495857.4 461969.6 93.2 351015.9 329244.8 93.8 273552.1 258577.5 94.5 429163.5 186593.3 43.5 Umnugovi Aimag e/s 523 300 57.4 525 381 72.6 442 330 74.7 332 189 56.9 87 arn 1133105.0 663150.0 58.5 828907.0 483905.0 58.4 1374662.0 1145077.0 83.3 2441217.0 1882493.0 77.1 15 e/s 456 343 75.2 324 230 71.0 289 216 74.7 314 184 58.6 Sukhbaatar Aimag am 486985.1 362783.5 74.5 349436.9 265578.6 76.0 370001.9 272917.7 73.8 345210.9 161812.0 46.9 e/s 1069 811 75.9 909 569 62.6 923 558 60.5 801 331 41.3 Selenge Aimag am 4053543.6 1486562.6 36.7 3451268.1 2316968.0 67.1 1842211.3 606545.8 32.9 2548830.3 784565.6 30.8 e/s 741 451 60.9 811 589 72.6 773 539 69.7 756 346 45.8 Tuv Aimag am 1233891.1 884013.7 71.6 1161278.5 844186.6 72.7 1396819.3 1094421.0 78.4 1366167.3 351220.5 25.7 e/s 789 454 681 351 787 542727 366 1055017.3 866781.8 1221096.4 1079123.4 556421.4 467070.3 798774.2 567473.8 e/s 419 366 87.4 336 275 81.8 350 297 84.9 181 104 57.5 Khovd Aimnag ___ arn 1139406.5 1042051.4 91.5 1042084.8 936747 89.9 655728.1 580180 88.5 949836.3 557318 58.7 e/s 462 378 81.8 445 372 83.6 474 408 86.1 418 279 66.7 ar n 482561.4 396064.4 82.1 726174.8 647467.1 89.2 805304.7 732502.0 91.0 1265148.7 718276.5 56.8 e/s 403 289 71.7 335 241 71.9 323 246 76.2 339 168 49.6 Khentii Aimnag ar 538134.4 265626.5 49.4 922437.1 510804.8 55.4 511500.7 378651.4 74.0 672428.8 254563.8 37.9 CDEA of the Capital e/s 10288 5403 52.5 8980 5287 58.9 9735 5736 58.9 9437 3594 38.1 City ar 168692360.8 67647622.5 40.1 166157936.1 67656056.6 40.7 201427909.3 75421572.4 37.4 657540942.7 390147399.5 59.3 Baganuur District of e/s 474 345 72.8 467 363 77.7 333 255 76.6 299 115 38.5 UB ar 589068.0 436360.2 74.1 480308.5 334050.8 69.5 537382.5 386651.5 72.0 706188.3 240063.3 34.0 Khorkhorin Soum of e/s 204 180 88.2 170 143 84.1 175 158 90.3 109 82 75.2 Uvurkhangai Aimag ar 151964.5 139682.7 91.9 195211.8 188072.2 96.3 78566.6 73921.6 94.1 44010.9 27101.3 61.6 Mandal Soum of e/s 235 181 77.0 193 158 81.9 249 206 82.7 210 122 58.1 Selenge Aimag ar 169446.3 119004.7 70.2 183287.2 117454.0 64.1 287780.6 217833.5 75.7 401237.7 136371.7 34.0 e/s 25,288 15,411 22,155 14,292 22,050 14,545 20,412 9,062 Total a 193,425,616.7 83,104,929.1 189,173,860.5 83,573,636.3 223,754,936.2 89,806,577.2 694,325,555.2 409,159,410.8 88 90/E 95 E 10°E 101E MONGOLIA ENHANCED JUSTICE SECTOR SERVICES PROJECT PROJECT WORKS: COURT DECISIONS ENFORCEMENT AGENCY NATIONAL CAPITAL R U S S l A N F E D E A T l 0 N AIMAG (PROVINCE) CAPITALS MONGOLIAINTER-SOUM COURT BUILDINGS SOUM (DISTRICT) BOUNDARIES AIMAG COURT BUILlDING AIMAG (PROVINCE) BOUNDARIES LEGAL AID CENTERS FOR MINORITIES INTERNATIONAL BOUNDARIES PROJECT SITES 11°E 11°'E 12°'E 50o SOP lö rnom -- - - -5 4 ---he -- ia Is I Uni ofTh A -sou Sukhbaatar laantigoM dn KHnVSGL- TvS,¯ SELEN0E BAYAop eOR KHg DAR ~ D VORNtD oso gel hdybalsan ZAKANBLAN ULAANB ATARk- KherlenSoum' (Jr läntS A RUL)NG ANBAATAR KHIENP4 orsele un undns r\ooo TsetserofÖndörkhaan X( dT Darkhan Baruun Urt Bayankhlongore/ Gov S &MERs y u aig' Arvaykýheer ,Cor' o u SKB Altay) V R - Mand.a'gv9°-------- GOV'ALTAY, BAYAN-, "/ 14--------- KHN0 R KHNA DUNGOV Saynshand. DORN060V' Zammyn-uud, SouM Dalanzadgad ThisMap Des n Un-f The World Bank. Tebunais any othr in m t a shownter on this map go not imply, on 1 2 50 75 10Klmtr the part of he World Bank A40°* på.~ ~ ~ ~ I. cI N A 5----M_ 75 1Mie -T