ICRR 12820 Report Number : ICRR12820 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 02/07/2008 PROJ ID : P090346 Appraisal Actual Project Name : Second Education US$M ): Project Costs (US$M): 100 100 Sector Development Policy Credit Country : Pakistan Loan/ US$M): Loan /Credit (US$M): 100 100 Sector Board : ED Cofinancing (US$M): US$M ): 0 0 Sector (s): General education sector (60%) Sub-national government administration (40%) Theme (s): Education for all (40% - P) Decentralization (20% - S) Public expenditure financial management and procurement (20% - S) Participation and civic engagement (20% - S) L/C Number : C4046 Board Approval Date : 07/01/2005 Partners involved : Closing Date : 10/31/2006 10/31/2006 Evaluator : Panel Reviewer : Group Manager : Group : Howard Nial White Kris Hallberg Alain A. Barbu IEGSG 2. Project Objectives and Components: a. Objectives: This credit was the second of a series of three DPLs with the same, shared, objectives . The ICR under review is a simplified ICR, with the full ICR being prepared on completion of the third credit in the series . The Second Education Sector Development Policy Credit Project (Punjab Province) was to support: a) public finance reforms to increase public spending for education (and other pro-poor services) and to ensure fiscal sustainability; b ) reforms that strengthen devolution and improve the fiduciary environment and governance; and, c ) education sector reforms to improve quality, access and sector governance . b.Were the project objectives/key associated outcome targets revised during implementation? No c. Components (or Key Conditions in the case of DPLs, as appropriate): Improving Public Sector Financing Letter of Sector Development Policy issued by the Government of Punjab A medium-term provincial budget framework (MTBF) which gives priority to increasing provincial expenditures on education by more than 50% over a 3-year period Support Devolution and Public Sector Management Reforms Terms of Partnership signed between provincial and district governments with agreed annual performance indicators, and conditional grants under PESRP fully transferred to district accounts upon signature of TOPs Developed accounting procedures to monitor and track program expenditures Fortnightly DAC meetings to review and settle all arising audit observations , as well as progressively clear backlogs Procurement reforms initiated to increase transparency Improving Quality, Access and Governance in the Education Sector Announced free schooling up to matriculation level Policy of free textbooks announced for primary school students. Introduction of a stipend program for all girls in grades 6-8, targeted initially to 15 disadvantaged districts with low literacy levels Textbook printing and publishing opened to competition Credible monitoring & evaluation system established Independent third party surveys commissioned to assess availability of stipends and free textbooks at the school level. School Councils re-established and roles clarified Established school-based budgets Established transparent criteria for districts to recruit new contract school-specific teachers, favoring recruitment of women and local candidates d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: As a DPL the credit was fully disbursed on effectiveness . 3. Relevance of Objectives & Design: The objectives support the three-year education sector reform program of the Government of Punjab a to enhance access and improve quality of education and improve gender parity in Punjab, which is consistent with both Pakistan's overall development strategy and the MDGs . Relevance is also demonstrated by continuing governments support for these reforms at all levels . Program design responded to perceived weaknesses in the education system . 4. Achievement of Objectives (Efficacy): Pillar 1: Improving Public Sector Financing (substantial ) The levels and composition of the education budget are increasingly aligned to meeting the needs of the sector. The Government has maintained its commitments under the Medium-Term Budget Framework (MTBF). Education allocations in the FY05 provincial budget (supported through PEDPC II) as well as in FY06 have increased by 15% and 12.6%, respectively. Districts have allocated 15% more non salary budget for education from their own budgets. The budget increases financed through the provincial budget are filling infrastructure gaps; supporting targeted incentives such as stipends and free textbooks; financing quality improvement such as teacher training and non-salary budgets for schools; and supporting capacity building programs . Pillar 2: Supporting Devolution and Public Sector Management Reform (substantial ) Annual conditional grants for education are being provided by the provincial government through the TOP agreements between the provincial and district governments on the basis of needs-cum-performance formula. While the transfer of resources is timely and sufficient, there are issues that continue to constrain district level performance. First, capacity constraints lead to under-spending and delays in implementation. Further, there are limited capacities for undertaking civil works which also affect quality of school construction. District level monitoring of the use of development funds needs to be strengthened to ensure improvements in service delivery. Further, after the election of new district governments in 2005, the district budget approvals have been behind schedule this fiscal year. This has led to delays in implementation of district financed programs . Proper accounting procedures to track and monitor sector expenditures have been developed by the Program Monitoring & Implementation Unit (PMIU). The Department’s large backlog of pending audit paragraphs are being progressively reduced. Satisfactory progress has been made in the area of procurement. Reforms achieved include contracts for civil works above a reasonable threshold are now based on open competition and transparent procedures; the requirements for pre-registration of contractors and of composite schedule of rates have been removed and replaced with realistic cost estimates based on prevailing market rates; School Councils (SCs) are now authorized to carry out small civil works at the school level up to a threshold of Rs. 400,000; and annual independent third party ex-post procurement reviews of a sample of procurements are being conducted. While such actions have laid the foundation for increased transparency and efficiency, there are areas that require further strengthening to institutionalize procurement reforms. For example, although SCs are authorized to carry out small civil works, this is only taking place in a few districts and is not yet a systematic practice . Pillar 3: Improving Quality, Access and Governance in the Education Sector (substantial ) Objective: To strengthen education sector management: This new objective was included under PEDPC II to re-orient the education department’s operational structure at the provincial and district levels to align it with its evolving responsibilities and functions under a devolved framework. Initially progress in this area was slower than anticipated, but restructuring of the provincial and district education departments has now begun. . The Education Department has created a unified Planning Wing by merging the two wings for college and school education to focus on planning and monitoring functions for the entire sector, including mainstreaming the management and information systems developed by the PMIU within the core functions of the Planning Wing. Capacity support programs are under implementation. A plan for restructuring of the district education management system has been approved . Objective: To improve teacher management:Progress has been made by revising the teacher recruitment policy including the policy of hiring all new teachers on school-specific contract and enhancing the requirement for educational qualification. From September 2005 all teacher recruitment is being undertaken using MIS data and related need analysis to determine school-by-school needs based on student-teacher ratios. Teacher recruitment policy has been reviewed and amendments incorporated including hiring of only female teachers for primary schools.Comprehensive system for monitoring absenteeism is yet to be developed, monitoring through District Monitoring Officers reported 3,000 absenteeism cases and action has been taken on 1,600. There have been delays in developing a focus program to improve teacher management by the district education set up. This is a core priority area under the next and future credits . Objective: To Revitalize School Councils: The program for supporting 2,400 SCs in six pilot districts in partnership with two NGOs using two different models (contracting out management to the NGO and a capacity building approach) is now under implementation. The Education Department’s analysis shows that SCs are functioning in 53,741 schools (or 84% of total schools).There are two deficiencies: capacity constraints of SCs; and lack of sufficient community mobilization to enable SCs to better understand their roles and functions. School-based recurrent budgets have been provided to SCs in 19 districts. A performance indicator regarding release and utilization of school based budgets has been included in the TOPs. A review of quality of civil works undertaken by SCs in districts where they have been provided funds for school civil works has also been launched . Objective: To Improve Equitable Access to Education: This objective continues to show good performance, with steady increases in public sector enrollments and progress in moving towards gender parity. Total enrollments (Pre-Primary – Grade 10) have increased at a respectable rate of 7% between October 2004 and October 2005 (or 20% since the start of the reform program). Enrollment increases have been registered for both boys and girls, although at a higher rate for girls as a result of which Punjab is seeing a narrowing of the gender gap. The stipend program targeting government Grade 6 – 8 girls in fifteen low literacy districts has continued to show enrolment increases and improved attendance. The number of girls receiving the stipend has increased from 214,000 in October 2004 to 252,000 in October 2005 (almost 18% increase), with attendance now at over 99% (up from 90% last year). Free textbooks have been extended to cover all government Grade Katchi – Grade 8 students under which more than 20 million books were provided to 9.7 million students in April 2005. Objective: To Ensure Provision of Quality Education through Improved Teaching and Materials" Several steps are being taken to improve the quality of teaching and learning. Recruitment of teachers with higher qualifications is continuing to fill the vacancy gap. The process of textbook improvement is also underway with opening up of textbook printing and publishing to private competition. A most significant development has been the establishment of baseline for monitoring the quality of learning through assessments and examination through the National Education Assessment System (NEAS). The first pilot assessment of Grade IV students in language and mathematics provides the baseline of student learning outcomes at the provincial and national levels . Objective: To Expand Access to Education through Innovative Public Private Partnerships : The Punjab Education Foundation (PEF) is now fully functional and staffed . It has initiated several innovative programs to support low cost private schools, which will be evaluated to consider future scaling up . 5. Efficiency (not applicable to DPLs): N.a. ERR )/Financial Rate of Return (FRR) a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the re -estimated value at evaluation : re- Rate Available? Point Value Coverage/Scope* Appraisal % % ICR estimate % % * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: The outcome is rated as satisfactory . PESRP continues to show steady progress . Achievements of the first operation have been sustained, including the continued increases in public sector enrollments and sector financing . Progress has picked up in areas that showed slower movement under the first Credit, such as the quality agenda and public private partnerships. a. Outcome Rating : Satisfactory 7. Rationale for Risk to Development Outcome Rating: The main benefits will be reversing the slide in education indicators, bringing more children into school up to grade 8 (particularly poor children and girls ) sooner than previously expected, providing better quality education . The program would contribute to closing the social gap and helping the Country progress toward the Millennium Development Goals (MDGs) related to poverty reduction and Education for All (EFA). Another indirect, but important, benefit is that the program is strengthening decentralization by increasing the role of districts, and promoting accountability between service users (parents and students) and service providers (schools and teachers). Overall, given the success of the first year program, the risk rating is moderate for the second year program, proposed to be supported under this project . Nevertheless, there is a need to maintain continued vigilance in case of a change of priorities and focus at the political level, both at the province and districts . a. Risk to Development Outcome Rating : Moderate 8. Assessment of Bank Performance: Bank performance is rated as satisfactory. Quality at entry followed from staying on course with the overall reform package, directing focus to areas needing greater attention. Continuity of the task team has been one major factor for ensuring the consistency in policy dialogue with the senior provincial leadership and in working with the Education Department. The team regularly engages with civil society to understand different stakeholder perspectives. At the operational level, consistent engagement with the PESRP team has enabled effective review of progress and discussion on areas needing acceleration. Coordination with donors has been effective and has enabled better alignment between available technical expertise and resources and with the needs of the Program. Finally, the Bank’s performance has benefited from regular field visits to districts and schools leading to better understanding of local level stakeholder issues, including those of district officials, teachers, and parents. at -Entry :Satisfactory a. Ensuring Quality -at- b. Quality of Supervision :Satisfactory c. Overall Bank Performance :Satisfactory 9. Assessment of Borrower Performance: Borrower performance is rated as satisfactory at all levels . At the provincial level, progress has been based on strong political championship, alignment of vision and strategy among the key set of policy makers, and intensive monitoring and oversight by the dedicated operational teams . The Chief Minister’s keen interest and monthly reviews of implementation progress have helped keep the Program on track . The PESRP Provincial Steering Committee continues to monitor progress, address bottlenecks, and provide overall guidance on policy issues . The PMIU’s intensive monitoring and analysis has ensured that the Program momentum is maintained, especially in timely implementation and delivery of inputs, and that policy decisions are based on information and analysis . At the district level, the commitment of district governments is reflected in progress achieved in the indicators agreed to in the TOP agreements. a. Government Performance :Satisfactory b. Implementing Agency Performance :Satisfactory c. Overall Borrower Performance :Satisfactory 10. M&E Design, Implementation, & Utilization: A strong monitoring system has been established which is now recognized by other departments in Punjab, other provincial governments and by the federal Ministry of Education . PMIU’s work has generated interest in other provinces, and the Punjab team has started to provide technical assistance to education reforms in Sindh and to the irrigation sector in Punjab. Similar requests have been made by the NWFP health and education departments . MIS data is being used for the purposes of analysis and reform program planning and policy decisions (e.g., data-based needs analysis for teacher recruitment ). This ongoing monitoring is supported by a regular third party validation (TPV) system to verify delivery of program inputs . TPVs conducted in 2005 for the delivery of stipends and textbooks showed more than 90% and 99% delivery, respectively. While there has been good progress in the area of monitoring, capacity of the Government to undertake evaluation is weak. Such capacity is also limited in national research organizations and among private consultants . The Government is working with the Bank and other donors to strengthen this capacity . Further, while the revamped monitoring system is in place and datasets developed by the PMIU placed with the districts, the capacity of district officials to analyze and use this information needs to be strengthened . Additionally, EMIS has to be strengthened to enable monitoring of outcomes, and needs to include full sector data of both public and private A National Education Census is currently being carried out by the Federal Bureau of Statistics, and will provide information on all public and private educational institutes in Pakistan . Once finalized, Punjab EMIS will use the data to update private sector information and to make it part of the routine EMIS census exercises., and requires technical improvements to enable computation of drop out, retention and completion rates . Wider dissemination of the information generated through its M&E system, including through Punjab Government ’s official website, is required. Finally, the monitoring system has been overly focused on enrolments with insufficient attention to learning outcomes, but also to other process indicators related to decentralization and feduciary performance . a. M&E Quality Rating : 11. Other Issues (Safeguards, Fiduciary, Unintended Positive and Negative Impacts): None. 12. 12. Ratings : ICR IEG Review Reason for Disagreement /Comments Outcome : Satisfactory Satisfactory Risk to Development Moderate Moderate ICR uses old rating system, rating Outcome : sustainability as "Likely" Bank Performance : Satisfactory Satisfactory Borrower Performance : Satisfactory Satisfactory Quality of ICR : Satisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate . 13. Lessons: 1. Commitment at all levels of government helps implementation of reforms, and can institutitonalize them so they are sustained through changes in givernment . 2. Commitment is also maintained by ensuring there are visible improvements in the quality of service delivery . 3. Continuity amongst Bank staff helps maintain the quality of the program, and is especially important to ensure consistency within a DPL. 4. District capacity development is important for social sector reforms which include decentralization . 14. Assessment Recommended? Yes No Why? PPAR of the three DPLs. 15. Comments on Quality of ICR: The ICR presents detailed information on the program with a balanced analysis of its achievements and shortcomings. However, it should be noted that the ICR was prepared sometime before the credit closed, that is May and October 2006 respectively, thus limiting the ability of the report to comment comprehensively on outputs and outcomes. a.Quality of ICR Rating : Satisfactory