Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TZ-2ndary Educ. Dev. Program II (FY10) (P114866) Report Number : ICRR0020863 1. Project Data Project ID Project Name P114866 TZ-2ndary Educ. Dev. Program II (FY10) Country Practice Area(Lead) Tanzania Education L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IDA-47480 31-Dec-2015 469,300,000.00 Bank Approval Date Closing Date (Actual) 27-May-2010 31-Dec-2016 IBRD/IDA (USD) Grants (USD) Original Commitment 150,000,000.00 0.00 Revised Commitment 127,429,766.13 0.00 Actual 121,661,960.26 0.00 Prepared by Reviewed by ICR Review Coordinator Group Katharina Ferl Judyth L. Twigg Joy Behrens IEGHC (Unit 2) 2. Project Objectives and Components a. Objectives According to the Project Appraisal Document (PAD, p. x) and the Financing Agreement (p. 4) of June 11, 2010, the objective of the project was “to support the recipient’s effort to improve the quality of the secondary education with a focus on underserved areas.” The PAD (p. 9) defines underserved areas as “rural areas, where public schools have inadequate infrastructure and unfavorable student-qualified teacher ratios." For the APL I period, unfavorable student-qualified teacher ratio is defined as less than 1:40. The project was the first phase of a three phase Adaptable Program Loan (APL) to be implemented between 2010 and 2020. The overall objective of the APL was “to assist the Government of Tanzania (GoT) in Page 1 of 14 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TZ-2ndary Educ. Dev. Program II (FY10) (P114866) implementing secondary education reforms to improve learning outcomes among secondary students.” During the project restructurings in June and November 2015, some outcome targets were revised in response to changes in government policy. However, since activities were not changed (ICR, paragraphs 12- 13) and the scope (and level of ambition) of the project stayed the same, no split rating is required. b. Were the project objectives/key associated outcome targets revised during implementation? No PHEVALUNDERTAKENLBL c. Will a split evaluation be undertaken? No d. Components The project included four components: Component 1: Upgrading existing school infrastructure (appraisal estimate IDA US$78.3 million, actual US57 million, 72.8% of appraisal estimate): This component was to finance an increase in the number of secondary schools meeting the minimum infrastructure requirements to make them fully functional. Also, this component was to finance support for the rehabilitation and completion of existing structures, and enhance the teaching and learning environment in schools through improving classrooms, science laboratories, teacher residences, latrines, and water and electricity supply. Component 2: Improving the equitable provision of teachers and the quality of teaching in mathematics, sciences, and languages: (appraisal estimate IDA US$27.1 million, actual US$ 17.6 million, 64.9% of appraisal estimate): This component was to finance improving equity in the distribution of teachers by providing a one-off settling-in allowance and housing for new teachers taking up posts in “hard to reach” locations. Also, this component was to finance the monitoring and regulation of teacher supply, providing science teaching facilities at teacher-education institutions as well as teacher training and professional development. Component 3: Ensuring adequate financing to secondary schools and improving utilization of resources (appraisal estimate IDA US$37.8 million, actual US$37.8 million, 100% of appraisal estimate): This component was to ensure that financing for purchase of teaching and learning materials was disbursed on time and monitored effectively. Component 4: Providing capacity building and technical assistance to implement reforms (appraisal estimate IDA US$6.8 million, actual US$6.7 million, 98.5% of appraisal estimate): This component was to finance the strengthening of the Education Management Information System (EMIS) to collect more accurate secondary education data. Also, this component was to finance assessments of capacity of critical institutions in the secondary school sector including the Tanzania Institute of Education (TIE), Agency for Development of Education Management (ADEM), and the National Examination Council of Tanzania (NECTA). Page 2 of 14 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TZ-2ndary Educ. Dev. Program II (FY10) (P114866) e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project Cost: The project was estimated to cost US$469.3 million. Actual project cost was US$215.5 million. Cancellation of Phase 3 school construction and low levels of government payments to capitation grants until 2015 represent a significant amount of the discrepancy between appraised project cost and actual project cost. Financing: The project was to be financed through a US$150 million IDA loan. The actual amount was US$119.1 million. US$30.9 million was cancelled. Borrower Contribution: The borrower was to contribute US$319.30 million. The actual contribution was US$96.4 million. Dates: The project was restructured three times: • On June 22, 2015 the project was restructured to clarify the language and targets of some key performance indicators in order to better align the Results Framework with the PDO. Also, the project’s closing date was extended by one year from December 31, 2015 to December 31, 2016 due to delays in school construction-related activities. • On November 23, 2015 the project was restructured to align the project’s Results Framework with Form II examination pass mark policy changes introduced in 2012. • On December 20, 2016 the project was restructured to change the approach to construction in the sector based on relatively better performance using other mechanisms. This led to switching funds allocated to the last phase of construction under the project to the Education Program for Results (EP4R) project. 3. Relevance of Objectives & Design a. Relevance of Objectives High: The project’s objective was highly relevant for Tanzanian country context, given that 34% of the working age population is in the age range of 15 to 24 years, of which only 15.7% have secondary education and only 3% have some tertiary education, allowing for an unmet demand for skilled workers. Also, even though enrollment in primary education had increased over time at project appraisal, a large share of children from low income families did not transition from primary to secondary education. The main challenges of the secondary education sector were misalignments in teacher availability and incentives for performance, inadequate infrastructure, and fluctuating intra-sectoral financing and insufficient capitation grants. The objective of the project was in line with the government’s Secondary Education Development Program (2004- 2020), which aims to meet the demand for better skilled citizens. Page 3 of 14 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TZ-2ndary Educ. Dev. Program II (FY10) (P114866) The objective of the project was also in line with the Bank’s Joint Assistance Strategy (FY2007-FY2010), which had a strong focus on improving the functioning of the education sector. The project’s objective was also aligned with the Bank’s strategy for 2012-2015 which had, among others, the aim to build infrastructure and deliver services, strengthen human capital and safety nets, and promote accountability and governance. The objective is highly relevant to the Bank’s current Country Partnership Framework (2018- 2022), which aims to diversify growth and enhance productivity, boost human capital and social inclusion, and make institutions more efficient and accountable. Rating High b. Relevance of Design Substantial: The APL instrument was appropriate, given the long-term time frame for implementing planned changes and the necessary sequencing of interventions. The planned activities were logically and plausibly linked to the achievement of the project’s objectives. Activities to improve the quality of secondary education included the upgrading of infrastructure of secondary schools and improve teaching in mathematics, sciences, and languages. Also, activities included teacher training and professional development of existing teachers, and monitoring of teacher absences. The project also provided capitation grants for the purchase of teaching and learning materials. Furthermore, the project provided technical assistance to build capacity for grant management and to implement reforms in the education sector. Activities to improve the quality of the secondary education with a focus on underserved areas included improving the equity of teacher distribution by increasing deployment of teachers to rural areas and constructing multi-unit teacher residences in hard-to- reach areas. The project design identified rural public schools to be rehabilitated through student-classroom ratio and student-teacher ratio. A needs assessment of the selected schools was to be conducted to identify infrastructure requirements and to develop adequate procurement packages. The underlying assumptions about how program actions would lead to intended outcomes were logical and properly laid out. However, the project design included some politically challenging interventions, such as the provision of a settling-in allowance to teachers based in hard-to-reach areas; this provision could not be implemented as planned due to opposition from teacher groups. Also, the project design could have benefitted from a deeper analysis of the construction modality of schools, as well as of coordination between, and capacity of, the different implementation entities. Rating Substantial 4. Achievement of Objectives (Efficacy) PHEFFICACYTBL Page 4 of 14 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TZ-2ndary Educ. Dev. Program II (FY10) (P114866) Objective 1 Objective To support the recipient’s effort to improve the quality of the secondary education Rationale Outputs: • The percentage of public schools meeting government-approved minimum standards for infrastructure and a student–teacher ratio of 40:1 increased from 4% in 2010 to 26% in 2016, not achieving the target of 37%. • 2,764 secondary classrooms were built, not achieving the target of 4,800 classrooms. • The student-teacher enrollment in science and mathematics teaching courses at teacher training colleges increased from 2,650 teachers in 2010 to 7,896 teachers in 2013, surpassing the target of 3,000 teachers. • 28,060 teachers were trained in service, almost achieving the target of 30,000 teachers trained. • 8,233 heads of public schools received management training, not achieving the target of 12,300. • 12,600 teachers were enrolled in pre-service diploma math and science programs under the project, of whom 3,991 graduated. • The percentage of qualified government secondary teachers increased from 57% in 2012 to 95% in 2016. • A semi-autonomous Teachers’ Service Commission to manage teacher affairs and a Teachers’ Professional Board to set professional standards and accreditation of pre-service teacher training programs were established. • The percentage of schools receiving full resources increased from zero in 2010 to 71% in 2016, not reaching the target of all schools. • All schools were (as of the time of the ICR) receiving allocations on a timely basis, achieving the target. • The textbook-student ratio in mathematics, sciences, and languages increased from 1:5 in 2010 to 1:1 in 2016, achieving the target. • The number of textbooks procured by schools in mathematics, biology, chemistry, physics, English, and Kiswahili increased from 1.7 million books in 2010 to 7.17 million books in 2016, surpassing the target of 6.8 million books. • EMIS integration with data on secondary education was completed and is operational in local government authorities, achieving the target. • An assessment of capacity for institutional development of the Tanzania Institute of Education, the Agency for the Development of Education Management, the National Examination Council of Tanzania, and the Education Materials Approval Committee, with linkages to the Ministry of Education, Science, and Technology (MoEST) and the Prime Minister's Office for Regional Administration and Local Government, were completed, and a plan for phased institutional strengthening was developed, achieving the target. Outcomes: Page 5 of 14 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TZ-2ndary Educ. Dev. Program II (FY10) (P114866) • Completion rates at the O level increased from an overall 22% (24% for male and 20% for female) in 2009 to 36.8% for male and 37% for female in 2016, surpassing the revised target of 30% and almost achieving the original target of 39%. • Completion rates at the A level increased from an overall of 2.6% (3.2% for male and 2.0% for female) to an overall of 5.4% (6.4% for male and 4.5% for female), surpassing the revised target of 3.5% and the original target of 4.1%. • The examination pass rates in math, science, and language were not used by the ICR to assess project performance, as the methodology for calculating the pass rates was changed during project implementation, making scores non-comparable across project years. Rating Substantial PHREVDELTBL PHEFFICACYTBL Objective 2 Objective To support the recipient’s effort to improve the quality of the secondary education with a focus on underserved areas Rationale Outputs: • The number of hard-to-reach secondary schools with teacher residences increased from zero in 2010 to 375 residences in 2016, surpassing the revised target of 367 residences and the original target of 28 residences. • The qualified teacher-student ratio for 500 hard-to-reach schools increased from 1:59 in 2010 to 1:17 in 2016, surpassing the revised target of 1:25 and the original target of 1:48. The ratio in rural areas improved more than it did at the national level, where the ratio was 1:20. • In the five regions with the highest proportion of rural population, student-teacher ratios improved from an average of over 46:1 in 2010 to 17:1 in 2016. • Student-classroom ratios only slightly improved, from over 39:1 in 2013 to 37:1 in 2016. Outcomes: • Completion data for rural areas is only available through 2012. Between 2009 and 2012 completion rates in rural areas increased from 21.1% to 38.8%. Page 6 of 14 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TZ-2ndary Educ. Dev. Program II (FY10) (P114866) Rating Modest PHREVDELTBL PHEFFICACYTBL Objective 3 Objective Overall APL objective: to assist the Government of Tanzania (GoT) in implementing secondary education reforms to improve learning outcomes among secondary students Rationale The ICR demonstrates progress toward achievement of the overall APL objective. Rating Not Rated/Not Applicable PHREVDELTBL PHREVISEDTBL 5. Efficiency Substantial: The PAD (p. 80) includes an economic analysis that assesses the economic return of investments in schooling. Mincerian analyses of returns to schooling were used to estimate earnings functions from a combined 2000-01 and 2007 household budget dataset. The analysis estimated that returns to secondary education were approximately 13.5%, higher than the returns to primary education of approximately 5%. On the margin, public spending on secondary education was considered to provide the greater return. The ICR (p. 22) estimates the gains in terms of additional enrollment in contrast with what would have occurred if the annual increase would have been half of what was observed. The analysis assumes that additional enrollment will be observed during the seven-year duration of the program, and that benefits will stop at the time the project closed. The analysis took into account lifetime earnings premiums of additional schooling, labor force participation, and employment rates. The analysis applies a six percent discount rate on lifetime earnings. A Net Present Value (NPV) of US$53.9 million, an internal rate of return of 5%, and a benefit cost ratio of 1.5 are estimated, indicating a worthwhile investment. The project experienced implementation delays due to the government’s fiscal constraints and lack of coordination between different departments within the government. Also, the cost for construction of classrooms was 13% higher than the initial estimate, due to delays. However, considerable costs were saved under the construction of teacher residences. While the initial design had planned for single two-bedroom houses, it was decided during implementation to build low-cost multi-unit teacher residences, decreasing the estimated cost of US$37,000 per house to US$14,000 per flat. Page 7 of 14 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TZ-2ndary Educ. Dev. Program II (FY10) (P114866) Taking everything together, efficiency is rated Substantial. Efficiency Rating Substantial a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal 0 Not Applicable 0 ICR Estimate 0 Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome Relevance of objectives was high given Tanzania’s need to reform its secondary schools and alignment with government and Bank strategies. Relevance of design was Substantial, as the underlying assumptions about how program actions would lead to intended outcomes were logical and properly laid out. While there was clearly substantial achievement of the objective to improve the quality of secondary education, there was not yet sufficient evidence to demonstrate those achievements specifically in underserved areas. Efficiency is rated substantial. Taken together, these ratings are indicative of moderate shortcomings in the project's preparation and implementation, and therefore an Outcome rating of Moderately Satisfactory. a. Outcome Rating Moderately Satisfactory 7. Rationale for Risk to Development Outcome Rating The government continues to be committed to reform the country’s education sector and develops relevant policies such as the Free Basic Education Policy, which aims to abolish fees and other costs involved in secondary education and all direct costs in primary education. Furthermore, the government aims to make lower secondary education compulsory starting in 2021 in order to increase enrollment rates. Other aspects of the education sector, such as teacher training and distribution and monitoring and release of capitation grants, will continue to be supported by the Bank’s ongoing education performance for results project. Furthermore, a Page 8 of 14 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TZ-2ndary Educ. Dev. Program II (FY10) (P114866) new Bank project is being designed ensuring the sustainability of achieved results. However, the government’s ongoing fiscal constraints might continue to have a negative impact on the reform of the education sector. The risk to development outcome is therefore Modest. a. Risk to Development Outcome Rating Modest 8. Assessment of Bank Performance a. Quality-at-Entry The project was built on lessons learned from previous Bank involvement in the sector. The Bank conducted appropriate analytical work and consulted with other stakeholders. The team identified relevant risks as Substantial. These risks included weak implementation capacity within the Ministry of Education and Vocational Training and the Prime Minister’s Office of Public Service Management. Also, the challenge of increasing access to and quality of secondary education at the same time, non-compliance by local government authorities with agreed selection of schools for upgrading, inadequacy of incentives to attract teachers to designated “hard-to-reach” schools, risk of incentive abuse, increased resource needs due to increased enrollment over time, and insufficient capacity at the district and school level to use and monitor resource use, were all identified as Substantial risks. In order to mitigate the financing risk, the Bank agreed with the government on an annual dated covenant for government contributions to the SEDP II program. Mitigation measures regarding low implementation capacity included capacity building and the provision of training activities. The Results Framework had several shortcomings, such as not including an indicator for measuring a critical aspect of the project’s objective: underserved areas. Also, the Results Framework included pass rates as an outcome indicator even though these were not reliable measures due to the annual changes in criteria for passing. Furthermore, data sources and collection mechanisms had shortcomings such as rural/urban completion data not being available (see Section 10a for more details). Quality-at-Entry Rating Moderately Satisfactory b. Quality of supervision The Bank had a strong working relationship with the government and other development partners. Also, the Bank worked closely with the government to identify implementation challenges and find solutions to address them. The Bank built capacity in critical areas such as financial management, procurement, and safeguards. However, the project was restructured to address shortcomings 18 months after they had been identified during the mid-term review. Since the restructuring did not include major changes, implementation could continue during this period. The Bank team ensured during the restructuring that the project’s design Page 9 of 14 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TZ-2ndary Educ. Dev. Program II (FY10) (P114866) continued to remain relevant and aligned with the government’s priorities. Implementation status reports and aides-memoire provided Bank management with detailed assessments of project implementation. Quality of Supervision Rating Moderately Satisfactory Overall Bank Performance Rating Moderately Satisfactory 9. Assessment of Borrower Performance a. Government Performance The government was committed to achieving the development objective. During project identification the Ministry of Education, Science and Technology worked closely with the Bank on the project’s design and identified priority areas for project activities. However, the government faced fiscal constraints that precluded continuous financial contribution, which resulted in implementation delays. Furthermore, lack of coordination between different departments within the government, particularly at the level of local governmental authorities, led to additional implementation delays. Also, the project was affected by weak capacity in financial management, procurement, and M&E. At the later stage of project implementation (2014/2015), progress picked up and the government’s financial support became more regular. Furthermore, the functionality of the EMIS system improved, allowing for better M&E. Government Performance Rating Moderately Satisfactory b. Implementing Agency Performance The Ministry of Education, Science, and Technology was responsible for overall project coordination and management and was to implement the project in collaboration with the Prime Minister’s Office for Regional Administration and Local Government. In order to ensure the coordination between different government agencies, the project team also included staff from the central level and local government authorities’ level. However, the project faced several issues in regards to communication and clarity of responsibilities between the different agencies, resulting in implementation delays. Also, the project team had weak capacity in critical areas such as procurement, financial management, and M&E. Relevant documents such as interim financial reports and progress reports were consistently submitted late, resulting in implementation delays. The implementing agency worked closely with the Bank to address these issues by building capacity for reporting on program activities, issuing guidelines to local government authorities on training staff in reporting on resources and M&E, and improving coordination between government agencies. Also, the accuracy and reliability of EMIS data was improved, and training was provided to staff in areas such as financial management, procurement, and safeguards. Page 10 of 14 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TZ-2ndary Educ. Dev. Program II (FY10) (P114866) Implementing Agency Performance Rating Moderately Satisfactory Overall Borrower Performance Rating Moderately Satisfactory 10. M&E Design, Implementation, & Utilization a. M&E Design The original Results Framework included four APL outcome indicators, six PDO outcome indicators, and 14 intermediate outcome indicators. The objective was clearly specified and the indicators mostly reflected the objective. The selected indicators were measurable in terms of numbers, timing, and location. The project built on already existing M&E mechanisms in the country, such as EMIS monitoring instruments. These EMIS monitoring instruments were further expanded to allow for the collection of data related to secondary schools, incorporation of new indicators, and conducting of impact evaluations. Also, two new set of instruments were developed, piloted, and included in the EMIS -- one for collecting data on the use of resources for the purchase of teaching and learning materials, and the other for collecting and reporting information on teacher attendance. However, the Results Framework had several shortcomings, such as not including a critical PDO indicator to measure the project’s performance in underserved areas. This indicator was added only during the May 2015 restructuring. Also, EMIS did not disaggregate data on project outcomes in rural versus urban areas, which would have been beneficial. The Results Framework included pass rates as an outcome indicator even though pass rates were not reliable measures due to annual changes in criteria for passing. Furthermore, the Results Framework did not include adequate indicators for measuring outcomes in 500 hard-to-reach schools. b. M&E Implementation An Impact Evaluation was designed under the project in order to evaluate the impact of performance-based incentives for teachers and schools. During project implementation, shortcomings of the Results Framework, such as the use of pass rates to measure student learning outcomes, were corrected. Also, the project faced delays in regard to hiring data management consultants. Furthermore, some reporting delays resulted from data management issues (for example, data stored in consultant's systems rather than in the government's own systems). The project addressed these issues throughout implementation. c. M&E Utilization M&E data and analysis were used to inform decision making, including to target interventions to lagging schools and to identify vulnerable populations in need of support. The education ministry used comparative data on regions for resource planning. Page 11 of 14 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TZ-2ndary Educ. Dev. Program II (FY10) (P114866) M&E Quality Rating Modest 11. Other Issues a. Safeguards The project was classified as Environmental Screening Category B and triggered OP/BP 4.01 (Environmental Assessment) due to the construction and rehabilitation of facilities such as classrooms, laboratories, offices, and teaching blocks within compounds of existing educational facilities. In order to address any negative impacts related to the construction and rehabilitation activities, the government prepared an Environmental and Social Management Framework (ESMF) and an Environmental and Social Assessment consistent with Tanzanian environmental laws and the Bank’s safeguard policies. The ICR (paragraph 45) reports that project’s compliance with the environmental safeguard was rated satisfactory throughout project implementation. b. Fiduciary Compliance Financial Management: The project used country systems for its financial management. Budgets were prepared by the project team and approved by the Parliament as part of the ministry’s budget. Funds were transferred directly to school bank accounts, and monthly reports were prepared by each school and submitted to the local government authority. Project activities were reviewed through an internal audit, including all activities of the ministry. Also, an external audit was required by law and conducted by the controller and auditor general. The project experienced financial management issues in regards to a delay in transfer of funds to schools in the early part of project implementation. This was resolved by switching to direct transfers of capitation grants to schools. Other issues included delayed submissions of interim financial reports, audit reports, and ineligible expenditures. These issues were addressed through constant follow-up, capacity building, and training. Ineligible expenditures were refunded by the government. In addition, audit recommendations of previous years had been implemented by the time the project closed. The external auditor’s opinion was not qualified. The ICR (p. 12) states that the financial management of the project was satisfactory. Procurement: Bank procurement and consultants’ guidelines were followed. The project faced several procurement challenges resulting in implementation delays. These challenges included insufficient procurement staff at the ministerial level with adequate knowledge of Bank procurement. Furthermore, local government authorities had limited procurement and technical capacity and did not receive sufficient support from technical departments in the provision of technical specifications, terms of references, and evaluation of bids and proposals. These issues were addressed through one-on-one interactions during the various stages of the procurement process, assistance with customizing bidding documents for use by local government authorities, and provision Page 12 of 14 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TZ-2ndary Educ. Dev. Program II (FY10) (P114866) of training for the procurement staff at the ministry. c. Unintended impacts (Positive or Negative) None reported. d. Other --- 12. Ratings Reason for Ratings ICR IEG Disagreements/Comment Moderately Moderately Outcome --- Satisfactory Satisfactory The government's fiscal constraints present risk, but Risk to Development policy priorities and continued Substantial Modest Outcome Bank support should somewhat mitigate these risks. Moderately Moderately Bank Performance --- Satisfactory Satisfactory Moderately Moderately Borrower Performance --- Satisfactory Satisfactory Quality of ICR Substantial --- Note When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 13. Lessons The ICR (p. 29) provides useful lessons learned: 1 . A results-based approach can be beneficial in a decentralized context to ensure an alignment of incentives between the national and local levels with a strong emphasis on accountability. In this project, communication and coordination issues between implementing partners at different government Page 13 of 14 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TZ-2ndary Educ. Dev. Program II (FY10) (P114866) levels led to implementation challenges and a focus on inputs/outputs rather than on results. 2 . Ensuring sufficient technical capacity for implementation and clear definition of roles of staff in different implementing agencies at the outset is critical to avoiding implementation delays. This project experienced implementation delays from the beginning due to the need to hire key staff and due to the lack of clarity between different roles. Ensuring that key staff is already being put in place during project preparation, and formalizing implementing arrangements with clear accountabilities during project implementation, will have a positive impact on project implementation. 14. Assessment Recommended? No 15. Comments on Quality of ICR The ICR provides a good overview of project preparation and a useful economic analysis. The ICR is concise and consistent. However, the ICR does not provide any information on procurement and how procurement issues were resolved. Also, the ICR includes only limited information on financial management. a. Quality of ICR Rating Substantial Page 14 of 14