Document of The World Bank Report No: ICR00001284 IMPLEMENTATION COMPLETION AND RESULTS REPORT (IDA-40890) ON A CREDIT IN THE AMOUNT OF SDR 33.0 MILLION (US$ 50.0 MILLION EQUIVALENT) TO THE SOCIALIST REPUBLIC OF VIETNAM FOR A TARGETED BUDGET SUPPORT FOR NATIONAL EDUCATION FOR ALL PLAN IMPLEMENTATION PROGRAM December 8, 2009 Human Development Sector Unit East Asia and Pacific Region CURRENCY EQUIVALENTS (Exchange Rate Effective 11/13/2009) Currency Unit = Vietnamese Dong (VND) VND 1.0= US$0.00005596 US$1.00 = VND17,869.50 FISCAL YEAR January 1 ­ December 31 ABBREVIATIONS AND ACRONYMS BCEP Building Capacity for Education Planning M&E Monitoring and Evaluation BoET Bureau of Education and Training MoET Ministry of Education and Training BoF Bureau of Finance MoF Ministry of Finance CAS Country Assistance Strategy MTEF Medium Term Expenditure Framework CIDA Canadian International Development Agency MOU Memorandum of Understanding CPAR Country Procurement Assessment Report MPI Ministry of Planning and Investment CPRG Comprehensive Poverty Reduction and Growth Strategy NGO Non-Governmental Organization S DCA Development Credit Agreement NTP National Targeted Program DFA District FSQL Audit ODA Official Development Assistance DFID Department for International Development PAD Project Appraisal Document DoET Department of Education and Training PC People's Committee DoF Department of Finance PEDC Primary Education for Disadvantaged Children Project DPI Department of Planning and Investment PEP Primary Education Project EC European Commission PRSC Poverty Reduction Support Credit EFA Education for All PTDP Primary Teacher Development Project EMIS Education Management Information System PTR Pupil-Teacher Ratio FMIS Financial Management Information System SBV State Bank of Vietnam FMR Financial Management Review SEDS Socio-Economic Development Strategy FMT Financial Management Technical Review SEQAP School Education Quality R Assurance Program FSQL Fundamental School Quality Level SREM Support to the Renovation of Education Management FSQL Fundamental School Quality Level Input Indicator SRV Socialist Republic of Vietnam II FTI Fast Track Initiative TBS Targeted Budget Support IDA International Development Association UPE Universal Primary Education GOV Government of Vietnam WB World Bank VJ CV SE PE IE VIETNAM Targeted Budget Support for National Education for All Plan Implementation Program Contents Data Sheet A. Basic Information B. Key Dates C. Ratings Summary D. Sector and Theme Codes E. Bank Staff F. Results Framework Analysis G. Ratings of Project Performance in ISRs H. Restructuring I. Disbursement Graph 1. Project Context, Development Objectives and Design ...................................................... 13 2. Key Factors Affecting Implementation and Outcomes........................................................ 7 3. Assessment of Outcomes ................................................................................................... 22 4. Assessment of Risk to Development Outcome .................................................................. 38 5. Assessment of Bank and Borrower Performance............................................................... 39 6. Lessons Learned................................................................................................................. 42 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners.................... 45 Annex 1. Project Costs and Financing .......................................................................................... 50 Annex 2: Outputs by Components/Sub-Components ................................................................... 51 Annex 3: Economic and Financial Analysis ................................................................................. 58 Annex 4. Bank Lending and Implementation Support/Supervision Processes............................. 63 Annex 5: Beneficiary Survey Results [Not Applicable] ............................................................... 66 Annex 6. Stakeholder Workshop Report and Results [Not Applicable]....................................... 67 Annex 7. Summary of Borrower's ICR ......................................................................................... 68 Annex 8. Joint Partners External Evaluation ................................................................................ 79 Annex 9: List of Supporting Documents....................................................................................... 87 Map IBRD 33511R ....................................................................................................................... 89 A. Basic Information Support for National Education-for-All Plan Country: Vietnam Project Name: Implementation Program Project ID: P085260 L/C/TF Number(s): IDA-40890 ICR Date: 12/17/2009 ICR Type: Core ICR SOCIALIST Lending Instrument: SIL Borrower: REPUBLIC OF VIETNAM Original Total XDR 33.0M Disbursed Amount: XDR 33.0M Commitment: Revised Amount: XDR 33.0M Environmental Category: C Implementing Agencies: Ministry of Education and Training Cofinanciers and Other External Partners: B. Key Dates Revised / Actual Process Date Process Original Date Date(s) Concept Review: 09/11/2003 Effectiveness: 01/26/2006 01/26/2006 Appraisal: 04/04/2005 Restructuring(s): Approval: 06/28/2005 Mid-term Review: 09/17/2007 09/17/2007 Closing: 06/30/2009 06/30/2009 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Moderately Satisfactory Risk to Development Outcome: Low or Negligible Bank Performance: Moderately Satisfactory Borrower Performance: Moderately Satisfactory C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Quality at Entry: Moderately Satisfactory Government: Moderately Satisfactory Implementing Quality of Supervision: Satisfactory Moderately Satisfactory Agency/Agencies: Overall Bank Overall Borrower Moderately Satisfactory Moderately Satisfactory Performance: Performance: i C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments Indicators Rating Performance (if any) Potential Problem Project Quality at Entry No None at any time (Yes/No): (QEA): Problem Project at any Quality of No None time (Yes/No): Supervision (QSA): DO rating before Moderately Closing/Inactive status: Satisfactory D. Sector and Theme Codes Original Actual Sector Code (as % of total Bank financing) Central government administration 7 7 Primary education 93 93 Theme Code (as % of total Bank financing) Education for all 50 50 Gender 25 25 Indigenous peoples 25 25 E. Bank Staff Positions At ICR At Approval Vice President: James W. Adams Jemal-ud-din Kassum Country Director: Victoria Kwakwa Klaus Rohland Sector Manager: Eduardo Velez Bustillo Christopher J. Thomas Project Team Leader: Emanuela Di Gropello Luis Benveniste ICR Team Leader: Emanuela Di Gropello ICR Primary Author: Suzana Nagele de Campos Abbott F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) The project will assist the borrower in the implementation of its National Education for All Plan through targeted budgetary support for selected sub-components of the Educational National Targeted Program (NTP) designed to enhance the quality of basic education, and through strengthening of the administration of the Education National Targeted program. Priority will be given to the primary level and the improvement of educational outcomes in this sub-sector. ii Revised Project Development Objectives (as approved by original approving authority) Not applicable (a) PDO Indicator(s) Original Target Formally Actual Value Values (from Revised Achieved at Indicator Baseline Value approval Target Completion or documents) Values Target Years Indicator 1 : Gross Intake Rate in Grade 1 Value quantitative or 100% 106.9% Qualitative) Date achieved 11/30/2005 06/30/2009 Comments (incl. % achievement) Indicator 2 : Net Primary Enrolment 100% (FSQL audit) Value 100% (FSQL audit) quantitative or 96.1% Qualitative) 95% (MOET/EMIS) (MOET/EMIS) Date achieved 11/30/2005 06/30/2009 Baseline NER according to FSQL audit was overestimated. Methodology for Comments assessment of NER was adjusted by completion and therefore, latest NER value (incl. % is likely to reflect an increase. This is also confirmed by the NER generated by achievement) MOET/EMIS data. Indicator 3 : Grade 1 Drop Out Rate Value quantitative or 3.0% 2.3% Qualitative) Date achieved 11/30/2005 06/30/2009 Comments (incl. % achievement) Indicator 4 : Grade 1 Repetition Rate Value quantitative or 2.3% 3.6% Qualitative) Date achieved 11/30/2005 06/30/2009 Comments (incl. % achievement) Indicator 5 : Primary Completion Rate Value quantitative or 89% 98.2% Qualitative) iii Date achieved 11/30/2005 06/30/2009 Comments (incl. % achievement) Indicator 6 : Student Achievement in Grade 5 (Standardized Learning Assessment) Value Math: 79.9 Math: 86.7 quantitative or Language: 51.3 Language: 61.0 Qualitative) Date achieved 12/31/2001 06/30/2009 Comments (incl. % This refers to the indepdendent learner benchmark. achievement) Indicator 7 : Pupil-Teacher Ratio Value quantitative or 22.0 19.4 Qualitative) Date achieved 11/30/2005 06/30/2009 Comments (incl. % achievement) Indicator 8 : Per Student spending on Non-Salary items (in VND, 2003) Value quantitative or 66,000 136,080 Qualitative) Date achieved 11/30/2004 06/30/2009 Comments (incl. % Still to be updated to 2007/08 achievement) Indicator 9 : Gender Parity Index (added in last ISR as part of IDA core indicators) Value quantitative or 1.01 (FSQL audit) Qualitative) Date achieved 06/30/2009 Comments (incl. % achievement) (b) Intermediate Outcome Indicator(s) Original Target Actual Value Formally Values (from Achieved at Indicator Baseline Value Revised approval Completion or Target Values documents) Target Years Indicator 1 : Average district FSQL index Value (quantitative 65.0 70.9 or Qualitative) Date achieved 11/30/2005 06/30/2009 iv Comments Baseline value was slightly readjusted at completion to reflect consistency in (incl. % methodology achievement) Indicator 2 : Number and % of schools that score >80 in FSQL index score Value 2,443 3,804 (quantitative 16% 24% or Qualitative) Date achieved 11/30/2004 06/30/2009 Comments (incl. % achievement) Indicator 3 : Number and % of teachers to have received 5 or more days of in-service training Value 237,738 222,303 (quantitative 66% 64% or Qualitative) Date achieved 11/30/2005 06/30/2009 Comments (incl. % This is not a cumulative indicator but an indicator of teachers trained very year achievement) Indicator 4 : Number and % of grades to have teaching aids for reading Value 127,095 235,739 (quantitative 81% 88% or Qualitative) Date achieved 11/30/2005 06/30/2009 Comments (incl. % achievement) Indicator 5 : Number and % of schools to have potable water Value 11,248 18,129 (quantitative 28% 48% or Qualitative) Date achieved 11/30/2005 06/30/2009 Comments (incl. % achievement) Indicator 6 : Number and % of classrooms to have a good blackboard Value 110,137 217,113 (quantitative 49% 96% or Qualitative) Date achieved 11/30/2005 06/30/2009 Comments (incl. % achievement) Indicator 7 : Number and % of classrooms to have achieved school construction FSQL Value 136,909 163,501 (quantitative 61% 73% or Qualitative) Date achieved 11/30/2005 06/30/2009 v Comments (incl. % achievement) 1) Central, Provincial and District financial and performance reporting mechanisms and management capacity improved Indicator 8 : 2) education budget increases in % of national budget 3) formula funding improved 1) Fully functional 1) New monitoring monitoring and and planning tools reporting 1) Reporting mechanisms developed and Joint mechanisms and weak Circular issued Value satisfactory 2) 18.6% (2004) 2) 20.1% (2008) (quantitative management 3) Funds not sufficiently 3) Needs-based or Qualitative) capacity driven by needs and transparent 2) 19.1% performance allocation formula 3) Equitable and applied to 2009 efficient formula ENTP funding Date achieved 12/31/2004 06/30/2009 06/30/2009 Comments (incl. % See policy matrix benchmarks for full set of intermediate results indicators achievement) Decline in shortfall of qualified teachers (added in last ISR as part of IDA core Indicator 9 : indicators) Shortfall in teachers Shortfall in teachers with with 12+2=15% N 12+2=25% N and % of Value and % of teachers teachers having received (quantitative having received 5 5 or more days of in- or Qualitative) or more days of in- service training: 228,849 service training: - 64% 166,236 - 52% Date achieved 11/30/2005 06/30/2009 Comments (incl. % achievement) Decline in shortfall of classrooms (added in last ISR as part of IDA core Indicator 10 : indicators) Shortfall in quality Shortfall in quality (FSQL) classrooms: (FSQL) classrooms: 39% Value 30% N and % of N and % of classrooms to (quantitative classrooms to have have achieved school or Qualitative) achieved school construction FSQL = consctruction FSQL 136,909 - 61% = 155,996 - 70% Date achieved 11/30/2005 06/30/2009 Comments (incl. % achievement) vi G. Ratings of Project Performance in ISRs Actual Date ISR No. DO IP Disbursements Archived (USD millions) 1 05/15/2006 Satisfactory Moderately Satisfactory 0.00 2 12/01/2006 Satisfactory Moderately Satisfactory 2.50 3 08/28/2007 Moderately Satisfactory Moderately Satisfactory 17.50 4 01/02/2008 Moderately Satisfactory Moderately Satisfactory 35.00 5 06/12/2008 Moderately Satisfactory Moderately Satisfactory 35.00 6 02/07/2009 Moderately Satisfactory Moderately Satisfactory 35.00 7 06/29/2009 Moderately Satisfactory Moderately Satisfactory 50.19 H. Restructuring (if any) Not Applicable I. Disbursement Profile vii 1. Project Context, Development Objectives and Design 1.1 Context at Appraisal At the time of Appraisal, Vietnam had demonstrated strong commitment to achieving universal basic education as a foundation to social development and economic growth, as reflected in the Education Development Strategic Plan for 2001-2010 and the Comprehensive Poverty Reduction and Growth Strategy (CPRGS). A fully costed National Education for All (EFA) Action Plan, which provided a credible strategic framework, had been approved and endorsed by Vietnam's international partners and Non-Governmental Organizations (NGOs). At the time, Vietnam was the only country in the South East Asia Region that had qualified to take part in the EFA Fast Track Initiative (FTI). Vietnam had made considerable progress in education in the earlier decade. Between 1990 and 2003, the primary net enrolment rate (NER) had significantly increased to reach levels of above 90 percent, the dropout and repetition rates had decreased from 12 percent to about 3 percent and from 9 percent to less than 3 percent, respectively, and the completion rate had increased from 47 percent to about 90 percent. Vietnam had also made significant progress in developing lower secondary education opportunities. The transition rate from primary to low secondary had increased from 78 percent to 88 percent; a majority of Vietnamese youth had gained access to nine years of basic education. However, Vietnam still faced, at the time, significant challenges in educational access and quality. Barriers to education access were concentrated in specific regions and rural areas where schooling conditions were harder due to lack of adequate human resources, prevalence of vernacular languages, high incidence of poverty, lack of adequate financial resources, low resource mobilization ability at the local level and the existence of smaller remote communities that tended to make the unit costs of education higher. An estimated 20 percent of children from some minority groups had no access to education, and it was increasingly apparent that universal basic education in these areas could only be achieved through targeted compensatory programs. The quality of education was low and variable, and it was estimated that a significant proportion of Vietnam's primary schools did not have the minimum infrastructure and instructional resources they needed to deliver quality education. Vietnam was at the time carrying out an ambitious reform of the teaching profession with assistance from Department for International Development (DFID) and the World Bank (WB) under the Primary Teacher Development Project (PTDP). The reform addressed the introduction of professional teaching standards and accreditation of teacher training, review of teachers' work and terms of service, creation of options for credentialing, performance/remuneration at different levels and right-sizing the teaching force. It was also implementing the Bank and donor- financed Primary Education for Disadvantaged Children (PEDC) project that aimed to improve the access to primary education and the quality of education for disadvantaged children through, inter alia, a component targeted at raising the fundamental school quality level (FSQL) of schools in the country's most disadvantaged districts. There were also important challenges with respect to the sector's financing. Despite a rapid increase in budgetary allocations to the education sector -- the sector's share of total public expenditures in 2003 was 17 percent-- funding was not being allocated effectively, in particular to address the needs of disadvantaged groups. About 95 percent of the primary school budget was spent on teachers' salaries, leaving minimal resources for non-personnel expenditures. As a result, financing of the education system still relied heavily on significant contributions from families for both capital and some recurrent expenses. As part of its National EFA Action Plan, Vietnam had committed to increase the share of its education budget to 20 percent by 2010. Official Development Assistance (ODA) accounted for approximately 10 percent of the total education budget. Still, the existing levels of funding were not considered sufficient to achieve Vietnam's EFA targets by 2015, and an additional $2 billion to $2.5 billion would be required over the period 2003-2015. Furthermore, additional funding to compensate for private contributions was considered fundamental towards relieving the burden of education cost-sharing to poor families. The International Development Authority's (IDA's) support to the Targeted Budget Support (TBS) Program, along with that of other donors, was designed to partially bridge the estimated financing gap at the primary level for the period 2005- 2008. The TBS operation was expected to support the Government's strategic approach to poverty reduction through: (a) prioritizing financial assistance to achieve specific education goals to localities in greatest need; (b) providing top-up funds for non-salary expenditures to improve the quality of basic education; (c) promoting planning, management and service delivery based on evidence-based needs analyses ­ supporting the use of FSQL; (d) encouraging local community demand; (e) strengthening governance as well as accountability, monitoring and evaluation procedures to assess continuously and in a timely fashion the effectiveness of education programming, resource allocation and resource utilization; (f) enhancing necessary administrative and financial management systems, particularly at the local level, in order to complement the ongoing decentralization and public administration reform efforts; and (g) providing technical support in order to respond to the challenges of uneven capacity across provinces and districts. Building on budget support arrangements that had been developed under the Poverty Reduction Support Credit (PRSC) series, the TBS operation aimed to channel ODA more efficiently and reduce transaction costs by using a sector budget support and delivery mechanism. 2 1.2 Original Project Development Objectives (PDO) and Key Indicators (as approved) The TBS operation's development objectives were to assist the Recipient in the implementation of its National EFA Plan through targeted budgetary support for selected sub-components of the Educational National Targeted Program (ENTP) designed to enhance the quality of basic education, and through strengthening the administration of the ENTP. At the Government's request, priority was given to the primary level (in part because this was the only level that incorporated minimum standards of access and quality, as defined by the FSQL regulation) and the improvement of educational outcomes at that level. A key assumption in the design of the TBS was that the provision of FSQL-inputs to schools that served poor children and ethnic minorities would be necessary and sufficient to improve both access and quality outcomes. Because schools below FSQL tended to serve poorer communities, the beneficiary group was expected to contain a higher proportion of poor children and ethnic minorities than the population of the country in general. Initial amounts of ODA financing, including the Bank's assistance, were expected to provide resources needed to cover approximately one-third of the shortfall in FSQL deficiencies in primary schools in Vietnam. Strengthening the administration of the National Targeted Program (NTP) was to be achieved through capacity building strategies and actions that would enhance the Ministry of Education and Training's (MoET's) ability to perform and comply with its existing performance benchmarks such as financial and administrative regulations and procedures, and adapt to emerging demands and requirements. Progress towards the accomplishment of PDOs and outputs was to be monitored yearly based on key performance indicators of education access, quality and efficiency and a policy matrix (see Table 4 and Annex 2). 1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification The Project's development objectives and key indicators were not revised during implementation. 1.4 Main Beneficiaries The TBS Project's main beneficiaries were to be primary school-aged children who were to be provided with the following benefits for primary education: reduced dropout and repetition, higher primary completion rates, enhanced net enrollment rates from inclusionary interventions for out-of-school children, improved transition rates to lower secondary schools and improved quality of education in schools. Primary school-aged children in rural/isolated areas as well as high-poverty districts would especially benefit through targeted funding to elevate local schools in disadvantaged localities to a minimum service delivery standard that would help address existing inequities. 1.5 Original Components (as approved) 3 The Project had two components. Component 1 aimed to support the implementation of FSQL standards by channeling official development assistance into the ENTP. Component 2 aimed to strengthen MoET's institutional capacity to administer and implement the ENTP. Component 1: Achievement of FSQL Standards through TBS to the ENTP (US$123 million total, including IDA and bilateral funding sources). The targeting of resources was to be based on an assessment of the financing gap needed to improve educational outcomes and satisfy minimum standards of access and quality, as defined by the FSQL regulation, in disadvantaged schools and districts. The FSQL standards established minimum service delivery criteria in terms of physical infrastructure, instructional materials, teaching staff qualifications, in-service professional development, school organization and management and school-community linkages. It also included a framework to enhance teacher capacity, school leadership and community participation while promoting inclusive education for traditionally disenfranchised communities. The ENTP mechanism was considered a useful vehicle to enhance pro-poor allocative efficiency and provide additional resources for the accomplishment of agreed sector priorities. The MoET has received technical assistance during preparation of the TBS Project to review and amend the then existing joint MoET-Ministry of Finance (MoF) Inter-Ministerial Circular that regulated the purpose, contents, targeting, budget management and implementation of the ENTP. The TBS was to focus on five of the seven existing ENTP sub-components: · Sub-component 1: Illiteracy elimination, consolidation of Universal Primary Education (UPE) and achievement of Universal Lower Secondary Education · Sub-component 2: Renovation of the curriculum and textbooks/teaching methods · Sub-component 4: Upgrading infrastructure of Teacher Training Institutions and teacher qualifications · Sub-component 5: Support for the education of ethnic minorities and disadvantaged regions · Sub-component 6: Infrastructure development 4 The Inter-Ministerial Circular placed emphasis on innovative ways to encourage school attendance by special needs children, who were frequently out of school. Activities eligible for financing included: · Outreach activities to attract out-of-school children · Renovation and construction of classrooms, latrines and libraries · Provision of teacher aids, equipment, instructional materials or free textbooks for schools servicing disadvantaged children (due to poverty, ethnic background, disability) · Teacher upgrading and in-service training in areas such as Vietnamese as a second language, vernacular language instruction, multi-grade teaching, inclusive education and school readiness for Grade 1 · Principal training in school management, planning and mobilizing communities for UPE · Community participation and Parent/Teacher Association (PTA) activities to encourage enrolment and to retain children in school. The ENTP is a centrally-sponsored scheme in which the MoET indicates to the provinces the expected allocations of central State budget funds to ENTP sub-components. However, provincial authorities have full discretion over planning, investment and resource allocation. Unlike a traditional investment credit, there were to be no legal ex ante financing allocations under the TBS Project. The Project was to support provincial and district ENTP annual work programs as well as technical assistance and capacity building to strengthen program management, allocative efficiency and equity, financial accountability, monitoring and outcome evaluation. ENTP expenditures are financed not only by the central Socialist Republic of Vietnam (SRV) State budget, but also by provincial and district supplements, community contributions, transfers from other special programs and other minor sources. IDA funds were to be pooled with bilateral donor partners' grant funds and channeled directly into the State budget via the State Bank of Vietnam (SBV) and the central Treasury. Donor funds were to be indistinguishable from central SRV funds, and there was therefore to be no intent to track ODA funds separately from those of other State budget expenditures. The ODA contribution to the ENTP was to total on average approximately 24 percent of the central State contributions to the relevant ENTP sub-components or approximately 10 percent of the total ENTP funds available for those sub-components (including, central, provincial, community and other sources). The SRV committed to increase the yearly central State budget counterpart financing for the ENTP by approximately 10 percent per annum from its 2005 level. Component 2: Institutional Strengthening for ENTP Administration and Implementation (US$5 million, parallel financed from bilateral sources, but jointly managed by all TBS partners) This component was to be financed from a parallel capacity building technical assistance fund, that was to be predominantly reserved for support to the central, provincial and district levels on planning, allocation, procurement, financial management and institutional strengthening issues, as well as for enhancing community consultation and empowerment. Funds were also to be allocated to independent implementation progress reviews (including procurement and financial management audits), some local-level monitoring by social organizations and commissioning 5 case studies on ENTP administration, monitoring and impact evaluation. This component included support to the following broad categories of activities: (i) ENTP Management and Administration; (ii) Capacity Building; (iii) Monitoring, Assessment and Evaluation; (iv) Inter- Ministerial Coordination; and (v) Program Administration, TBS-EFA Review and ENTP External Audits. The Project's expected contribution to the overall institutional strengthening of the ENTP management and monitoring system was to include: · Effective needs-based targeting of funds · Educational output and outcome results-based reporting · Enhanced accountability of education administrators · Effective and transparent financial reporting, supervision and audit · Efficient public procurement processes through rigorous benchmarking and capacity building. In addition, the project aimed to strengthen the policy environment within which schools operated through a policy matrix1 detailing supportive government policies and benchmarks. Annual performance monitoring would incorporate regular appraisals of the policy environment in which the project operated. Some of the key areas included: · Implementation of provincial education planning · Strengthening of resource allocation · Student achievement measurement and monitoring · Enhanced pro-poor budgeting and spending · Inter-ministerial coordination · Non-salary expenditure per student Planning and implementation of activities related to ENTP institutional strengthening would be closely coordinated with related parallel donor activities addressing critical capacity building issues. Disbursements. ODA funding for the Component 1 of TBS was to be made against advances and then subsequent ex-post reviews on the basis of expenditure performance for all sources in participating provinces under the five sub-components of the Education NTP, up to a previously agreed maximum and upon demonstrating progress in reducing FSQL deficits. According to the Memorandum of Understanding (MOU) signed by the Government and TBS Partners (Belgium, Canada, New Zealand, the United Kingdom, the European Commission and IDA), disbursements for each of fiscal years 2006, 2007 and 2008 would be made following an annual review meeting to be held no later than September 15 of each year on the basis of: (i) the annual work program for selected Sub-components of the ENTP and Component, provided by MoET; (ii) Financial Monitoring Reports for the previous two semesters indicating sources and uses of funds for each 1 See Annex 2. 6 of the Sub-components, physical progress in the implementation of each Sub-component, and the sources and uses of funds for Component 2; (iii) annual financial statements and financial audit reports of the ENTP for the preceding fiscal year; and (iv) the annual procurement audit report. 1.6 Revised Components The Project's components were not revised during implementation, although there were significant delays in the implementation of Component 2. As planned, the implementation of Component 2 will continue through December 31, 2010. 1.7 Other Significant Changes Not Applicable. 2. Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design and Quality at Entry The Bank had been a long-standing partner of MoET. Beginning with the Primary Education Project (PEP) in 1993, a comprehensive strategy of support for basic education had been adopted through progressive engagement. During the three-year period immediately preceding appraisal, the level of Bank support had increased significantly in terms of the number, size and scope of projects. The Bank was, at the time, leading several co-financed operations, including the Primary Teacher Development project, the Primary Education for Disadvantaged Children project, and a trust-funded activity to support provincial education planning capacity. ODA- supported projects had increasingly provided the operational context for piloting critical elements of Vietnam's wider reform agenda that included: the development of minimum service standards (or "Fundamental School Quality Levels or FSQL"), teacher professional profiles, student achievement testing instruments, a sector investment planning methodology, a Medium Term Expenditure Framework (MTEF), and a computerized Education Management Information System (EMIS). This operation was designed to rely and build upon the achievements and lessons of experience from these reform activities. Alignment with Government, Bank and other donors' priorities. The introduction of a fully- costed EFA Action Plan, the increasing importance of decentralized education planning and delivery, and the support that was being provided to strengthen education management capacity at all levels of Government provided a unique opportunity for developing a coordinated multi- donor approach to funding EFA goals using the Government's administrative and budgetary systems. Taking advantage of this opportunity, the Bank assumed a coordination role with Belgium, Canada, the European Commission, New Zealand and the United Kingdom (the TBS partners) in the development and implementation of the TBS operation. 7 The TBS operation was fully aligned with the then current Country Assistance Strategy (CAS) goal of "making basic social services accessible and affordable for the poor". The Government had recently renewed its commitment to provide Universal Primary Education, and this commitment was reflected in several Government plans and strategies. The National EFA Action Plan provided the strategic framework for long-term education development and brought together the national education goals and targets under the Socio Economic Development Strategy (SEDS) 2001-2010, the Comprehensive Poverty Reduction and Growth Strategy (CPRGS), the Education Development Strategic Plan for 2001-2010 and the Vietnam Development Goals. Lessons learned. The design of the TBS operation incorporated several lessons learned from previous involvement. First, it supported a change in the Government's funding formula for basic education aimed at promoting greater equity in educational outcomes. Previously, the allocation of recurrent funding to education had been determined by provincial and district general population levels which favored schools located in highly populated, developed urban areas at the expense of those in remote and sparsely populated disadvantaged regions. The new funding formula altered the resource allocation formula to consider the size of the school age population while placing greater emphasis on central level targeted compensatory programs to channel resources to disadvantaged groups. Second, the operation supported the concept of FSQL initially developed under the PEDC Project, shifting the emphasis on schooling standards from one traditionally associated with elite schools that fulfilled conditions of excellence to one that focused on a basic set of school resource requirements, taking into account the specific needs for instructional improvements in isolated multi-grade classrooms and communities. Third, in response to the ongoing shift of administrative, planning and management responsibilities to lower governance levels under the then ongoing public administration reform process, the TBS operation supported decentralized planning and service delivery focusing on districts and schools (while recognizing that local level capacity constraints could affect the achievement of project goals). Finally, in order to facilitate donor harmonization and alignment with SRV procedures, reduce transaction costs and strengthen sector management capacity, the TBS operation adopted a funding approach whereby ODA funds were fully merged with domestic funds to support the government's education objectives. Participatory process. Project preparation was built upon an extensive consultation effort during the design of the PEDC Project and the development of FSQL principles. During preparation, eleven field visits were made and discussions were maintained with provincial representatives, district officers, teachers, principals, primary students and community members from urban, rural and remote areas. Focus group discussions were held at school sites, and provincial consultations reviewed preliminary project proposals. Choice of instrument. The preparation and consultation process also included extensive discussions on the choice of instrument. Most donors supported an approach which would provide support to an already established Government program, while strengthening Governments' systems and minimizing transaction costs through donor harmonization, alignment with SRV procedures and ultimately a funding approach pooling all funds into the Treasury. As such, this approach was very much in line with a "budget support" which could have led to the choice of a DPL. At the same time, however, maintaining a closer grip on education sector funding and technical support, together with the need to actively support and 8 accompany the Government in the introduction and use of FSQL in the ENTP, was also considered to be a priority by the World Bank, leading to a preference for a SIL over a DPL. In this context, and after various discussions and consultations among donors and with the Government, an "hybrid" approach was finally chosen whereby a SIL was finally selected but with characteristics tending very much towards a budget support approach ­ leading to the admittedly rather ambiguous appellation of "targeted budget support" (which raised several "eyebrows" in some parts of the World Bank). In fact, as further developed below, this ended up being a very much a Sector Wide Approach (SWAp), in its purest sense, probably ahead of time for the complex and low capacity Vietnamese institutional context. Safeguards. The TBS operation did not trigger any of the Bank's Safeguard Policies. Still, since it focused on disadvantaged areas where ethnic minority people resided, and was expected to enhance the livelihood of indigenous people by improving the access to and quality of primary education, the project utilized the same basic Indigenous Peoples' Policy Framework (IPPF) from the PEDC Project. While recognizing that there were no aggregate gender differences in access and education outcomes in primary schools, preparation identified factors that constrained boys' and girls' educational participation and aimed to support interventions that would benefit them. Risks and mitigation. The TBS operation responded to the Government's priorities, counted on its full commitment, was framed within the context of the Bank's and donors' assistance strategies, took into account lessons learned and incorporated input from extensive stakeholder consultations. Yet, the operation was extremely ambitious. While it focused on the critical subcomponents of the ENTP and set-up a program of donor-financed technical assistance to support their implementation, the geographic reach of the operation was country-wide (and by definition because of its objectives, in more dispersed, less advantaged districts and communities), and it was to be implemented by central, provincial and district levels of government besides the schools themselves. The operation's parallel objective of strengthening the administration of the ENTP was pursued by relying exclusively on existing Socialist Republic of Vietnam (SRV) systems -- administrative, financial management, procurement, monitoring and evaluation -- while providing support to improve those systems during implementation of the project. Project documentation acknowledges concerns about varying institutional capacity among districts, a significant inherent risk in the program arising from the overall resource management context, a risk of obtaining poor value for the money as a result of corrupt practices, and an overall high procurement risk2 based on findings of previous studies and revisions of SRV's procurement legislation and the inherent risks related to the highly decentralized nature of project procurement. In retrospect, perhaps the greatest challenge that the TBS operation faced was its dispersed and decentralized implementation by multiple levels of agencies of somewhat limited but varying capacity, within a fiduciary framework that was in itself cause for some concern. IDA's Project Appraisal Document (PAD) was not explicit about these risks, although they were subsumed under different risk headings. The risk rating for local level implementation capacity was "Modest", and fiduciary risks, although mentioned in the Financial Management (FM) and Procurement Arrangements Annexes, were not brought forward to the Critical Risk Table, and hence not taken into account in the project's overall risk which was rated "Modest". Although these risks would by definition have been difficult to mitigate during preparation, it is not clear that the mitigating measures for Financial Management, for example, which relied on 2 Risk associated with Government procurement procedures and potential impact. 9 an audit to mitigate FM risks, were sufficient. The same applied to procurement where, in spite of the "High" ascertained risk of using Government systems, the only mitigation measures proposed were to provide guidance on "Procurement Best Practices" and the external audit. The design of the TBS operation basically followed a Sector Wide Approach (SWAp), in its purest sense, where there is up-front agreement on a Government program (defined as the five eligible Sub-components), donor coordination, pooling of funds, and use of Government systems for financial management, procurement and monitoring and evaluation. However, during preparation, weaknesses in those very systems that would be used to supervise the project's implementation had already been identified but under-rated in terms of the risk they would impose and with insufficient upfront mitigation measures. Instead of requiring a strengthening of those systems before implementation in order to have them up and running to provide needed inputs, the design of the TBS included strengthening and capacity building activities to strengthen the Government's systems under Component 2, to be carried out in parallel with the implementation of eligible Sub-components. For example, the preparation of FM and Procurement Manuals, which were eventually completed under Component 2 during implementation, could have provided more forceful measures to mitigate fiduciary risks. Along the same line, the completion of an acceptable 2006-2010 Inter-Ministerial Joint Circular before effectiveness, or at least disbursement, regulating the implementation, roles and responsibilities under ENTP would have also helped address the extremely complex institutional setting. 2.2 Implementation Implementation of the TBS operation was impacted enormously by the extremely complex institutional environment in which it was operating, with varying levels of institutional capacity and the resulting need for coordination among different institutions and levels of government, and the resulting difficulties in financial administration and control, that were all the more important given the operation's design as an instrument of budget support. The lack of adequate systems for monitoring, evaluation, and control of expenses, as well as institutional capacity to improve those very systems, made supervision, both by the MoET and the donors all the more cumbersome. Institutional Environment. The institutional environment in which the TBS operation was implemented involved what is basically a matrix structure, undergoing a process of decentralization that was initiated with the passage of the new Budget Law. At the central level, MoET, MoF and Ministry of Planning and Investment (MPI) had different responsibilities; MoET's responsibilities were changing and it was expected that its main functions would center on policy development, and provision of guidelines and framework for the improvement of education quality as well as of a national quality assurance mechanism to ensure the fulfillment of national priorities through locally-based activities; MoF had the primary responsibility for recurrent expenditures (which represent roughly 95% of education expenditures), and for the design and implementation of the MTEF and Management Information Systems; and MPI had responsibility for capital expenditures. This structure was paralleled at the provincial levels, with Departments of Education and Training (DoETs), Departments of Finance (DOFs) and Departments of Public Investment (DPIs) that report jointly to both their respective ministries at the central level, and to the Provincial People's Committees (PCs), which have education units which are involved in the planning and implementation of education. At the district level, Bureaus of Education and Training (BoETs) report to the respective DoET and the local People's 10 Committee (PC). Representatives of these sub-national units work through Steering Committees at each level, but the ultimate control of expenditures rests with the PCs, limiting the authority of MoET over program implementation and reporting. In this already complex basic framework, things were made even more complicated by the inconsistency in implementation of responsibilities across provincial samples, which impacted performance and the ability to address shortcomings in a systemic fashion. Implications for the implementation of Component 1. Funding for the TBS operation's five eligible Sub-components was affected in several ways, both allocation and utilization. Central allocations, based on the ENTP pro-poor allocation formula for eligible Sub-components, are proposed by the MoET to the MoF, but the latter can adjust the proposal not only in aggregate but also reallocating among Sub-components. Not all ENTP expenditures are financed from the central budget, however. Donor funds, which accounted for around 10 percent of ENTP funding, are transferred to provincial accounts, together with the central budget, where they are then complemented with provincial and district supplements and community contributions. ENTP funding is transferred to provinces under seven separate budget sub-headings corresponding to the ENTP's priorities, of which only five were considered eligible sub-components under the TBS operation. Once the funds are transferred, the provinces, especially the PCs, have considerable autonomy over planning, investment and resource allocation across priorities and districts Table 1 below summarizes the State Budget allocations across each sub-component over the 2006-2008 period. From 2006 to 2008, school infrastructure (Sub-component 6 of the ENTP), vocational training (Sub-component 7 ­ not supported by TBS), and textbooks (Sub-component 2 of the ENTP) accounted for the bulk of ENTP funding, followed by support of ethnic minority groups (Sub-component 5 of the ENTP). However, in the absence of fully reliable actual expenditures by sub-component and how these compare province-to-province, we cannot say to what extent the provinces have followed the indicative allocations by sub-component as they were received from MoET.3 Additionally, there are early indications that over time, as universal primary education was being achieved, in response to enrolment demand, resources were allocated to the growing priority sub-sectors of lower and upper secondary education.4 Table 1: Allocation of State Budget Funding by ENTP Sub-component 2006 2007 2008 2006-2008 ENTP Sub-component VND bn. % VND bn. % VND bn. % % Eradication of Illiteracy and 150 5.1 170 5.0 150 4.3 4.8 Universalisation of Primary Renovation of Curriculum 1,120.5 37.7 563.5 16.7 252 7.2 19.7 and Textbooks Training of Informatics 78 2.6 150 4.4 168 4.8 4.0 Engineers * 3 Field work observations indicated that in most but not all cases provincial allocations were consistent with central notional allocations. 4 See also Draft Joint Donors' Final Evaluation Report, October 2009 11 Upgrading Teacher Training 275 9.3 400 11.8 80 2.3 7.7 Institutions and Teacher Training Support for Education of 330 11.1 500 14.8 540 15.5 13.9 Ethnic Minorities and Disadvantaged Groups Infrastructure Development 516.5 17.4 897.0 26.5 1,290.0 37.0 27.5 Improvement of Vocational 500 16.8 700 20.7 1,000 28.7 22.4 Training * TOTAL 2,970.0 100 3,380.0 100 3,480.0 100 100 * = Not included in TBS-EFA Source: MoET - DPF Another issue related to funding and expenditure was a significant under-spending of allocated budgets (estimated at around 26%), over the implementation period. Table 2 below shows the figures available for the period January 1, 2007 - March 31, 2008, using the 2008 allocation including local fund counterpart.5 There is a significant total under spent, with total expenditures of VND 3,164.8 bn. / 4,100.1 bn. = 87.2%, or a total 22.8% under spent; there was a broadly similar under spent for the year 2008. Under-spending was the result mostly of: (i) the late release of budgets to provinces and districts; (ii) complex procedures to agree upon expenditures; and (iii) rigid unit cost forecasts which would later require complex approval procedures to be adjusted. 6 Table 2: Under-spending of Funds for ENTP (Jan 1, 2007 ­ March 31, 2008) Forecast Actual Shortfall as % of as % of as % of Sub-Component VND.m. VND.m. Total VND.m. Forecast Total Primary Universalisation & 179,960 130,244 4.1 49,716 27.6 5.3 illiteracy eradication Reform of curriculum & 1,081,813 823,790 26.0 258,023 23.9 27.6 textbooks Training of Informatics 211,855 152,291 4.8 59,564 28.1 6.4 engineers* 5 This explains the higher budget in relation to the table above 6 However, it appears that this situation is common in Vietnam and that the sums unspent will be carried forward to future years and will still be used for purposes specific to ENTP. In this sense, it can be argued that donor funding towards elements of Component I will continue, at least into 2009. 12 Improvement of pedagogical colleges and teacher 315,716 245,978 7.8 69,738 22.1 7.5 training Strengthening Education in 440,899 354,528 11.2 86,371 19.6 9.2 mountainous areas Improvement of school 1,170,026 965,854 30.5 204,172 17.5 21.8 infrastructure Improvement of vocational 699,814 492,153 15.6 207,661 29.7 22.2 training* TOTAL 4,100,084 3,164,838 100.0 935,246 22.8 100.0 * = Not included in TBS-EFA Source: External Audit Report, Deloitte Vietnam, 2008 Implications for the implementation of Component 2. The extremely complex institutional environment in which the TBS was implemented, the design of the operation's institutional arrangements which involved coordination and implementation by the existing structure of the MoET, MoET's limited capacity to implement a program of this magnitude while trying to strengthen cross-cutting systems that required inter-Ministerial coordination and that were required to provide input and structure to and evaluation of the program all stretched MoET's capacities, and required an extraordinary effort on the part of the donors to keep the operation on track. In this context, implementation of the institutional strengthening activities under Component 2 suffered important delays ­ made worse by differences of opinion among donors as to the intensity of supervision and Technical Assistance (TA) to be provided in a context where country systems applied, to the point that donors were still discussing the work plan of Component 2 late during implementation. Financial reporting by the provinces, which differed substantially in term of institutional capacity, was invariably delayed, yet the project's FM Manual was only finalized in March 2009. Understandably, problems with external financial audits became a central theme in donors' supervision efforts (see Section 2.4). Implications for donors' supervision, working arrangements and disbursement decisions. Donor supervision involved a broad framework of Joint Annual Review (JAR) and Joint Semi-Annual Reviews (JSAR); the JARs were scheduled around September of each year, and JSARS around April. There was an agreed process of rotating responsibility among donors (from now on defined as TBS partners) for taking the lead in formally inter-acting with the Government to reduce transaction costs for both Government and donors and ensure alignment of views and approaches within the program. In the second stage of program implementation (and up to June 30, 2009), when a more "hands-on" approach was finally determined to be necessary to address the program's complexities, TBS partners and relevant Government' counterparts also organized themselves in two working groups (on financial management and procurement, and education quality and standards) to tackle basic implementation issues, but also to try to bring the technical dialogue forward, in these two areas. Scheduled annual disbursements in respect of Component 1 were to take place after each JAR. Yet, invariably TBS partners would have to delay disbursements because of the delays in submission of important documents. These delays together with the need to focus on basic implementation issues as opposed to broader strategic themes affected the program and placed 13 added responsibilities on the donors for follow-up (see above regarding the strengthened working arrangements). The 2006 disbursement was delayed pending receipt of: (i) Procurement Audit and Financial Assessment Reports; and (ii) 2007 Work Plans and Budget for ENTP Support (Component 1) and ENTP Capacity Building (Component 2). The 2007 disbursement was delayed pending receipt of: (i) acceptable finalization of the financial statement audit for 2006 (which highlighted persistent problems in relation to monitoring and oversight of allocation, execution and outputs by the decentralized system and lack of response to the previous year's audit report); (ii) a time-bound action plan for addressing critical financial management issues adopted by MoET and State Treasury; and (iii) a 2008 work plan for Component 2 finalized and agreed by all the partners. Finally, the 2008 disbursement was delayed pending receipt of: (i) a report on the budget allocation to the ENTP disaggregated by sub-components and provinces for the 2009 fiscal year, including an overview of the allocation principles and evidence of dissemination to provinces; (ii) management responses to procurement and financial management audits, including action to be taken in response to findings, etc, (iii) report and update to the 2007-developed FM Action Plan; (iv) 2008 progress and financial reports for Component 2; and (v) a refined and prioritized Project Implementation Document for 2009-2010 Component 27, with clear implementation arrangements. In spite of these delays, however, pro- active donors' management succeeded in at least obtaining all the needed documentation and completing the project on schedule. 2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization M&E Design. SRV was to provide information on implementation progress and impact of NTP- funded activities at different levels of Government based on available databases. Funds were to be controlled, accounted for and reported on using the Government's own financial systems. Key outcome and results indicators were defined to monitor the project's progress towards its objectives, and a baseline established during preparation. Expected targets to be achieved by completion were not set, however, for two main reasons. First, the fully decentralized nature of the program allowed provinces full discretion on how to allocate funds across sub-components (with the exception of Sub-component 2 on curriculum renovation) and districts, subject to spending eligibility. As a result, fixing ex-ante targets was very difficult. The second reason was related to the fact that the ENTP was an existing Government program to which donors' financial contributions would represent only about 10 percent of the overall budget, making it difficult to have enough leverage to "impose" targets on the overall program. An agreement was reached to aim for an overall improvement in all indicators. Two exceptions to this rule were the agreements ­ which were however not put up-front in the results framework ­ that it "was expected that ODA financing will provide additional support to cover approximately one-third of the shortfall in FSQL deficiencies in primary schools" and that the "Government has made a commitment that yearly central State budget counterpart financing for the ENTP will increase approximately 10 percent, using 2005 as a baseline level". Some targets, more in qualitative terms, were also fixed for some policy measures contained in the Program's Policy Matrix, which represented another section of the M&E framework aimed at monitoring broader policy and institutional improvements related to ENTP. Finally, since the TBS supported eligible sub- 7 Generally described by its MoET title "TBS-II ­ EFA Work Plan 2009-2010". 14 components of an ongoing program, another complication was that the outcomes and results were defined for the country as a whole, making it difficult to attribute results solely to the support from this operation. Improvements in education quality and service delivery were to be captured through annual FSQL district audits undertaken every year since 2004/2005 in the whole country. An FSQL Input Index (FII) was developed to provide a numerical summary score against which yearly progress could be measured for each school, district, province or groups therein could be measures. The FII consists of five sub-components with 29 indicators relating to school quality covering the following categories (and respective weights, which total 100): (i) school organization and management (26); (ii) teaching staff (27); (iii) infrastructure, teaching and learning equipment (25); (iv) implementation of education socialization (7); and (v) education activities and quality (15). The district audits were also providing information on education outcomes such as enrollment and completion rates. The results framework had all outcomes and outputs expressed as national average values, to measure improvements in the quality of basic education overall (as written in the PDO). It was designed also under the assumption that any further improvement in, say, primary enrollment, completion and FSQL indicators, would most likely benefit the poorest areas and disadvantaged groups (other objective of the ENTP program) because of the already high values of some of the primary outcome indicators (the last 10 percent not completing primary would necessarily come from disadvantaged groups), and the very clear pro-poor focus of FSQL. Further disaggregations by disadvantaged groups of some of the indicators were however advised when needed. Districts were to prepare annual reports specifying progress towards ENTP outcomes and outputs. Provinces would then compile these and prepare annual summaries of ENTP implementation. Technical assistance to support local and national capacity building for monitoring and evaluation of ENTPs was provided under Component 2. The project was also designed to promote local monitoring by social organizations, such as parent-teacher associations. M&E Implementation. Thanks to the continuity and relative reliability (with some exceptions8) of the District FSQL census data ­ supported by the PEDC Project ­ as well as the familiarity of most districts and provinces with this dataset, outcome and output indicators were generally regularly updated, becoming routinely used as monitoring tools. However, in many instances, in part because of the project's decentralized institutional set-up (which required keeping track with 64 provinces), there were delays in reporting on the results framework. This was in part mitigated by the consolidation at the national level undertaken by the team in charge of the FSQL Audits in the PEDC Project. Reporting was more problematic for sub-components' specific outputs and, in particular, financial reporting (see below). A limitation of M&E in Vietnam which could not yet be fully solved during implementation was the integration of the District FSQL census data within the broader MoET EMIS to setup a fully functioning, consistent and sustainable M&E framework for all education levels. While this was very much related to unclear signals received from the MoET's management and divisions across existing projects dealing with data collection, this was also the result of the delays in procuring the technical assistance and capacity building activities that would strengthen the MoET's M&E capacity under Component 2 and help set-up an integrated M&E framework. As shown below, recent progress was however made towards finally achieving integration. 8 The adequate measurement of the net enrollment rate was an issue for a long time. 15 M&E Utilization. A comprehensive integrated education management information system applicable to all levels of the education system would be necessary in order to make sound education management decision based on facts. Currently FSQL is the only system through which nationwide data is collected; however, it is only applied for primary education and its sustainability may be at stake after the end of PEDC.9 The MoET's current EMIS needs improvement and is currently supported by a Delegation of the European Commission (EC) project.10 An action plan for the achievement of a comprehensive integrated EMIS, incorporating the most critical elements of the District FSQL audit, is under development improving prospects for good M&E in the future. At the same time the importance of the issue was recognized by the setting up of an 18 member, high level, inter-departmental EMIS committee approved by the Minister. While a good integrated M&E system is a pre-condition for utilization, currently, MoET does not yet necessarily use the education data collected for its decision making planning. This will require further capacity building on how to translate data into management decisions, to be supported under Component 2 and Support to Renovation of Education Management (SREM), among other initiatives. 2.4 Safeguard and Fiduciary Compliance History of fiduciary compliance and audits. Fiduciary compliance was the weakest area of the TBS operation. This should not have come as a surprise given that the funding mechanism relied exclusively on Government systems, which had been identified from the outset to present issues that needed strengthening, and especially in view of the institutionally complex environment in which the operation was to be implemented. It was also the result, in part, of a lack of preparedness of the operation, especially with respect to its fiduciary aspects, at approval, where risks were generally either under-estimated11 or insufficiently addressed up-front, putting a lot of pressure during implementation. As an example, the summary of the March 2006 semi-annual review by the TBS partners mentions: "TBS Partners are currently working on draft Terms of Reference (TOR) for the internal audit......Audits will be conducted [by] an independent auditing firm chosen through competitive bidding......The initial audit will be conducted in 2006 in order to track the use of government funds and to build up the capacity of the Government of Vietnam [Socialist Republic of Vietnam]". Not surprisingly, the first disbursement of funds was delayed pending, inter alia, the submission of procurement audit and financial assessment reports. When eventually submitted, these reports were considered to be of "low quality" and apparently steps were taken to ensure higher quality of the second year audits. Finally, delays in the implementation of Component 2, as well as an initial "hands-off" donors' approach in supervision, also admittedly did not help in fostering early improvements in fiduciary aspects. The second financial management audit was considered of satisfactory quality, but its findings pointed to "significant weaknesses in the financial management of ENTP with weak control and 9 Although the new primary education project ­ SEQAP- may be able to continue financing and implementing the survey at least temporarily. 10 The SREM project. 11 Overall financial management control risk was for instance considered to be only "moderate", including "moderate" for reporting and monitoring. Or, yet, following a DFID assessment, procurement risks associated with project design were only considered "moderate". 16 accounting practices and no certainty that the funds are being used for the intended purposes".12 This is in sharp contrast with a control risk for ENTP considered to be only "modest" one year before and/or shows that the mitigation measures recommended to address the control risk at approval, which were fundamentally already existing supposedly "positive" characteristics of the Government's financial system, improvements to be made through the 2006-2010 Joint Circular yet to be prepared, measures outside the direct control of TBS, or the audit, were insufficient. As a result of the identified weaknesses, the independent auditors were unable to express an opinion on the 2006 financial statements of the ENTP. The second procurement audit was also of satisfactory quality, but "showed some weaknesses in the compliance of the new procurement law and poor response of the government to the outcomes of the previous audit, calling for the communication of the procurement finding to all concerned agencies and for a detailed action plan to address them". Again, this was not entirely surprising given the deficiencies already identified in the Government's procurement system and practices and the lack of adequate mitigation measures, although admittedly more effort could have been put from all sides to disseminate procurement best practices. Both Financial Management and Procurement ratings of the Implementation Status and Results Report (ISR) had been reduced to Moderately Unsatisfactory (MU) by January 2008. The joint semi-annual review of April 2008 highlighted financial management as a "critical concern with limited evidence of progress against the Financial Management Action Plan developed in response to the 2007 September Annual Review findings". The review recommended actions to: (i) prepare for the execution of financial statement and procurement audits; (ii) finalize a complete 2007 Financial Management Technical Review including an in- depth analysis of financial performance of ENTP during 2007 and its main determinants; (iii) finalize Operational Manuals for Financial Management and Procurement; and (iv) adjust the TBS work plan to include clear mobilization of technical assistance for financial management. The review's aide-memoire noted: "Financial management and, to a lesser extent, procurement remain a cause of concern. There is limited progress in achieving timely, accurate and usable Financial Monitoring Reports.....Current progress is hampered by systemic financial management obstacles; however, there is limited evidence of MoET and inter-ministerial activity to support financial management planning and preparation activities at the central and sub- national levels....". Bank ISR ratings for Financial Management and Procurement continued to be rated MU. The 2007 Financial Management Technical Review (FMTR)13 took place just before the 2008 JAR and highlighted some useful findings with respective recommendations. Main findings included improvements in accounting and use of sub-codes, and much better planning and coordination of the preparation of the financial statements and audit of Fiscal Year (FY) 2007, but still very clear weaknesses in reconciliation and financial reporting, together with inadequate FM staffing at the central level. While there was some limited improvement in some areas, the persistent weaknesses in others were again outlining both initial serious systemic deficiencies of the ENTP FM system, such as a lack of clear responsibilities at the central, provincial and district 12 Implementation Status and Results Report (ISR) No. 4 dated January 2, 2008. 13 The results were summarized in Annex 1 of the September 2008 joint annual review. 17 level for reconciliations, financial reports preparation and consolidation, including the use of a double reporting channel (through Treasury and MoET) using different financial report formats and generating inconsistent results, and a persistent lack of action from the Government in tackling some of the identified issues, including a continuous underestimation of the needs for FM staff and lack of action on the audit findings in spite of reiterated insistence on these points by the TBS donors. Several specific recommendations came out of the FMTR. Some of the main and most general ones are reported in Box 1 below. Box 1: Main Recommendations of the 2008 FMTR · Specify and clarify responsibilities in reconciliations and reporting · Implement regular follow up and assistance by the MoET central level to provinces and lower levels that have not submitted either adequate, reliable financial reports or late reports or no reports · Align the format of the financial reports to reduce and if possible eliminate multiple reporting requirements and finalize the format of the FMRs · Decide whether to continue to develop MoET's financial reporting systems and capacity or to move towards reliance on the Treasury system for the financial reporting · Develop and implement training programs on financial reporting · Finalize and issue Joint circular and FM Manual to support all of the above · Ensure FM Manual provides simple, concise and comprehensive guidelines on the FM requirements and support it with relevant training · Develop and implement a longer term plan for FM staff in the MoET for ENTP FM · Disseminate audit report findings throughout MoET and ST as one method of improving awareness of FM weaknesses and ways to improve · Provide regular progress reports on Agreed Action Plans needs to be provided by MoET Source: FMTR 2008 Many of the findings of the FMTR were confirmed by the 2007 financial statement audit report. The auditor's report demonstrated an improved process for facilitating the 2007 Financial Statement Audit and allowed for a qualified opinion, on the basis of financial reports from State Treasury rather than from the implementing agencies. This was a marked improvement upon the no opinion of 2006. The qualified opinion was based mostly on the fact that the scope of the audit was limited (the sample too small) and that auditors were not provided with all of the needed supporting documents in those provinces they visited (representing less than 0.5% of supporting documentation). The auditors also noted significant delays and inaccuracies in reporting disbursements on the part of the State Treasury (incorrect coding), but were able to report that 98% of the ENTP expenditure were used correctly, with the remaining 2% applied to other categories of expenditure which the auditors considered ineligible (but which did not constitute misuse of funds). Similar to the FM Audits, there was progress in MoET's preparation 18 and management of the 2007 procurement audit, and this was acknowledged as welcomed improvement over the management of the past audits. However, the audit found a significant proportion of non-compliant contracts of which 19 percent with medium-to-high risk and repeats the recommendations of previous reports as it finds no evidence of follow up of the MoET on the main audit findings, including sharing the audit findings in full with those provinces audited and provide a summary to all other provinces for their understanding. In order not to lose the momentum of the FMTR and of somewhat more encouraging but still challenging 2007 audits, the joint financial management-procurement working group was also established at this point under the leadership of the World Bank as focal point for 2008/2009. By early 2009, there had been some improvement in fiduciary aspects: the fiduciary manuals had been completed, audit findings of 2006-2007 were shared with provinces and districts, the Joint Circular was officially issued addressing most (although not all) of the financial management weaknesses (including responsibilities in financial reporting and an improved Financial Management Review (FMR) format), provincial training workshops were being carried out on the manuals, Joint Circular and audit findings, and the work plan for the 2008 financial statement and procurement audits had been completed. On this basis, the Bank's ISR rating for Financial Management was upgraded to Moderately Satisfactory in May 2009. As expected, however, the effects of these positive developments could not yet be seen on the 2008 financial and procurement audits, of which the draft reports both showed persistent problems, including instances of non-compliance with the country procurement law and lack of follow-up on procurement "best-practices" (procurement audit) and "mis-statements" in the financial statements (financial statement audit). In particular, while some improvements such as a significant decrease in ineligible disbursements (e.g., site clearance before construction work) have been noted, the draft audit financial statement report contained a qualification where the misstatement of about 4 percent of expenditure was noted during the audit of 19/63 provinces of the program which was considered as material. Nevertheless, further efforts from State Treasury have been noted when the Provincial State Treasuries have taken corrective action to improve the reliance of the disbursement reports used by the Program Management as their financial statements; and mis-statements in the financial statements have been adjusted and recognized as reasonable by the auditors. As a result the auditors gave a qualified opinion of the 2008 financial statements.14 It is to be expected that the improvements of early 2009 have positively reflected on the 2009 financial management and procurement processes, but, in spite of some evident steps in the right direction, the fiduciary record of the program by closing remains nonetheless generally weak. The qualifications in the anticipated auditor's report need to be taken seriously by donors and Government. They indicate that while relying on the Government system for the ENTP program was probably too much of an under-estimated risk at preparation, in particular given the insufficient mitigation measures, the Government was also slow in acting on the financial management recommendations. Similarly, while the risks of following the Government's procurement procedures have probably been under-estimated at preparation given the non compliance of the country law with World Bank's best practices, MoET has been overall too 14 The opinion was only "qualified" because the auditors were not able to perform additional audit procedures to ensure that all material mis-statements (if any) in non-audited provinces were detected and adjusted. 19 slow in taking action, resulting in several actions being only implemented in 2009 and some actions having not yet been implemented. Sustained effort, through Component 2, to improve FM and procurement processes needs to continue (starting with the on-going workshops on the 2008 audit findings), including continuing with the audits.15 Safeguards. None of the Bank's Safeguard Policies applied to the TBS operation, although it did focus on areas related to ethnic minorities under Component 2 (the Study on Ethnic Minorities) and through financing of sub-components in less advantaged areas, where the concentration of ethnic minorities is higher. The main reason for not triggering this safeguard advanced at the time was that the project was expected to enhance the livelihood of indigenous people by improving the access to and quality of primary education without expected negative consequences. The Bank's Environmental Assessment Policy was also not triggered, under the understanding that ENTP sub-components would largely focus on school renovation and rehabilitation and that new classroom construction would occur within the existing school premises. 2.5 Post-Completion Operation/Next Phase Despite continued difficulties in implementation in view of weak implementation capacity combined with a complex institutional reality, prospects for continued emphasis and focus on the ENTP are very good. ENTP as an on-going Government program. First and foremost, the ENTP is an ongoing government program which reflects continuing national priorities and which uses government systems for implementation. These government systems have been strengthened and will continue to improve with the continuous implementation of Component 2. A Joint Circular Governing the Management and Utilization of ENTP Funds from 2011 to 2015 is in its initial stages of development, building on the 2006-2010 Joint Circular, to guide the future implementation of the program.16 In the meantime, the government through a Decision of the Prime Minister has approved an extension of the targeted program through 2010 with the following objectives: (i) universalisation of lower secondary education nationwide by 2010; (ii) increase the proportion of the trained labor force to 40% of the total labor force by 2010; and (iii) improve access to and equity in education in remote areas, ethnic minorities and difficulty-hit areas. Reflecting achievements to date, the focus is shifting a bit towards lower secondary, yet the objectives that were supported by the project remain relevant, and count on the government's support as reflected in the ENTP's approved budget allocation. For 2010, the government has allocated VND 4,600 billion to the ENTP's seven sub-components. The largest increases have been allocated to Sub-components 2 (Renovation of Curriculum and Textbooks), 4 (Training of 15 The audits should ideally include some of the provinces and institutions which had been audited in previous years in order to find out if there have been improvements in procurement and financial management practice. 16 As detailed in the Summary of the Borrower Completion Report, the new circular confirms support to the existing ENTP sub-components while broadening the scope of some of them (such as the sub-component on ethnic minorities to include broader geographical coverage and new schools) and adding some new sub-components (including universalization of pre-school coverage). 20 Teachers, Lecturers and Education Managers), and 5 (Support to the Education in Mountainous Areas, Ethnic Minorities and Disadvantaged Regions). Capacity building component. Second, implementation of technical assistance for institutional strengthening under the DFID and Canada funded Component 2 will continue until December 31, 2010. During the April 2009 Joint Semi-Annual Review of the TBS, the government and donors have agreed upon a very detailed Work Plan 2009-2010 to guide the priorities and implementation of Component 2 over the final year of implementation. The priorities and work program are described in Section 3.2. 21 Table 3: Allocation of ENTP State Budget by Sub-Components ­ 2006-2010 Total budget No. ENTP Sub-Components 07/QD-TTg Allocated budget by years 2006-2010 Central budget 2006 2007 2008 2009 2010 plan TOTAL FUNDS 18,500.00 2,970.00 3,380.00 3,480.00 4,000.00 4,600.00 660 150 170 150 130 55 1 Universal lower secondary education/consolidation of universal primary education 2 Renovation of curricula, textbooks and teaching materials 2,530.00 1,120.50 563.5 252 230 364 910 78 150 168 220 294 3 IT human resources training and IT application to schools 4 Training of teachers, lecturers and education managers 600 80 100 80 110 230 2,900.00 330 500 540 652.3 847.7 5 Education in mountainous areas, ethnic minorities and disadvantaged regions 6 Upgrading infrastructure for schools 6,400.00 711.5 1,196.50 1,290.00 1,457.70 1,709.30 7 Capacity building for vocational training 4,500.00 500 700 1,000.00 1,200.00 1,100.00 Source: MoET-DPF 3. Assessment of Outcomes 3.1 Relevance of Objectives, Design and Implementation The objectives of the TBS operation continue to be relevant to Vietnam's current development priorities. Vietnam has largely achieved universal primary education and is on track to reach its other Millennium Development Goal (MDG) of eliminating gender inequality in primary education by 2015. The operation is fully consistent with the SRV's draft Education Development Strategic Plan for 2010-2020 and its ENTP which both continue to emphasize: (i) pursuing eradication of illiteracy and universalizing primary school enrolment; (ii) universalizing lower secondary enrolment; (iii) continuing support for education development; (iv) innovating curriculum, textbooks and teaching materials; and (v) organizing training and other activities to improve teachers' and administrators' capacity. The TBS operation also continues to be relevant to the SRV's ongoing decentralization, public sector reform and strengthening and institutional capacity building objectives. The objectives of the TBS operation are also consistent with the Bank's current country and sector assistance objectives, including the updated Country Partnership Strategy which puts emphasis on FSQL to improve the governance of the education sector and achieve equity targets, while renewing commitment to quality education for all in Vietnam. The TBS operation's design and implementation continue to be relevant. While the operation addressed, and continue to address, important SRV sector priorities, it tackled, through a somewhat ambitious mechanism and, therefore, with slower than expected results, key issues that are still very pertinent to the education sector: strengthening and capacity building for strategic planning, management, monitoring and implementation within a decentralized framework, with the objective of achieving higher quality and equity in basic education. As mentioned in Section 2.2, implementation of the TBS operation was impacted enormously by the extremely complex institutional environment in which it was operating, with varying levels of 22 institutional capacity and the resulting need for coordination among different institutions and levels of government, and the resulting difficulties in financial administration and control, that were all the more important given the operation's design as an instrument of budget support. The lack of adequate systems for monitoring, evaluation, and control of expenses, as well as institutional capacity to improve those very systems, made supervision, both by the MoET and the donors all the more cumbersome. As a result, the Bank and other donors correctly adjusted and intensified the focus of their supervision efforts to prioritize the development of the systems needed to strengthen the Government's implementation capacity for the ENTP. 3.2 Achievement of Project Development Objectives Outcome indicators. Although there was some fluctuation during the operation's implementation period, the Socialist Republic of Vietnam (SRV) achieved generally good progress towards educational outcomes (Table 4). Access to and quality of primary education improved from 2003/2004 (original baseline in Project Appraisal Document or PAD) and 2004/2005 (updated baseline in first ISR) to 2007/2008 and the recently available 2008/2009 according to the vast majority of indicators. Gross intake in Grade 1 increased from 90.1 percent at the baseline in 2003/2004 to about 107 percent in 2009, net primary enrolment increased from 83 percent to 100 percent according to DFA data (which however had probably not adequately measured this indicator up to 2007 given issues in the assessment of the school age population) and from 94 percent to 96 percent (2008 data) according to EMIS data, completion rates increased from 92.3 percent in 2003/2004 to 98.2 percent in 2009, and drop-out rates decreased from 3.2 percent to 2.3 percent. Repetition rates, however, have increased since 2006 after an initial brisk decline. Learning outcomes17 (admittedly slower to move and therefore likely to capture at best only part of the effects of the interventions at this stage) have also moved in the right direction, with substantial increases in both math and language. Achievement in Grade 5 language increased substantially with 61.0 percent of students reaching the "independent learner" benchmark in 2007/2008 versus only 51.3 percent in 2001; and achievement in Grade 5 math as well with 86.7 percent of students reaching the "independent learner" benchmark in 2007/2008 versus 79.9 percent in 2001. Finally, the share of non-salary spending per student increased significantly in real terms, at least up to 2007, reflecting more emphasis on quality related inputs. On a mixed note, while the continuous decrease of the pupil-teacher ratio over the 2003-2009 period may indicate some slight quality improvement in terms of smaller class sizes and higher interaction between teachers and students, it is more likely to indicate losses in efficiency resulting in a higher salary burden over time per student. This decrease was largely the result of a decline in primary-school aged population which did not translate in fewer teachers because of systemic rigidities. During the 2005-2008 period, SRV also made progress towards increasing equity in access to primary education. As could be expected from the already high initial baselines, further improvements in access and completion were generally pro-poor resulting in a closing gap 17 Produced from two comparable Grade 5 assessment studies undertaken by MoET in collaboration with the World Bank, NIES and the University of Melbourne. 23 between urban and rural areas and Kinh and ethnic minorities in enrollment18, and, more generally, advantaged and disadvantaged areas in completion (Figures 1 and 2). Gaps between urban and rural areas also decreased in terms of learning outcomes19 between 2001 and 2007, but unfortunately increased somewhat between Kinh and ethnic minorities (Figure 3). A likely explanation at this stage for this slightly increasing gap is that the higher access for ethnic minorities to primary education has likely translated into even more disadvantaged ethnic minority groups being enrolled with a negative reflection on learning outcomes. But effects on learning outcomes take longer to be seen and it is likely that the pro-poor improvements in FSQL indicators over the project life will translate into decreasing gaps in learning outcomes between advantaged and disadvantaged groups in the future. Notwithstanding this point, SRV would be well advised to put renewed emphasis on the quality of education for ethnic minorities. New strategies to address ethnic minority issues20 and new operations such as School Education Quality Assurance Project (SEQAP), introducing full-day schooling for ethnic minorities, go into that direction. While these aggregate data shows that SRV has made generally good progress towards its desired outcomes, it needs to be clear that it is not possible to attribute this progress solely to the ENTP or TBS through ENTP, especially in view of the relatively minor financing shares involved and the presence of other donor financed projects. Table 4: TBS for Education NTP Outcome and Results Indicators 18 While the overall NER was over-estimated in 2005/2006 according to DFA, relative values across groups are to be taken as accurate (in other words, the bias was uniformly distributed across groups). EMIS calculations also confirm closing gaps. 19 Calculated in terms of standardized raw scores. 20 As documented in the recent status report on Component 2 supported ethnic minority development programs (MoET, 2009). 24 Outcome Indicators: Baseline Updated Year1 Year2 Year3 Year4 09/03-06/04 Baseline 09/05- 09/06- 09/07-06/08 06/08-06/09 (a) (PAD) 09/04-06/05 06/06 06/07 (ISR) (a) Gross Intake Grade 1 90.1 100 108.0 104.6 105.0 106.9 Net Primary 83.0 100 100 100 99.0 100 Enrolment (b) DFA Net Primary 94.4 95.0 95.2 96.0 96.1 NA Enrolment (c) Dropout (Grade 1) 3.2 3.0 2.9 3.2 2.0 2.3 Repetition (Grade 1) 2.7 2.3 1.9 2.0 3.4 3.6 Completion (d) 92.3 89.0 96.8 94.8 97.1 98.2 Achievement in Grade 79.9 86.7 5 Math (e) Achievement in Grade 51.3 61.0 5 Language (e) Pupil-Teacher Ratio 23.1 22.0 21.5 21.0 20.5 19.4 Yearly Per-Student 66,000 NA 107,647 136,080 NA NA Spending on Non- Salary Items (in VND, 2003) Results Indicators: Average District 62.1 65.0 67.8 68.9 69.6 70.9 FSQL Score No. and % Schools 2,443 NA 2,820 3,102 NA 3,804 that Score >80 in 16% 18.1% 19.9% 24% FSQL Index No. and % Schools NA 11,081 12,359 12,759 13,306 14,124 that Score >60 in 71% 79% 82% 85% 90% FSQL Index (f) No. and % of 228,849 237,738 256,692 246,205 166,236 222,303 Teachers (all sites) to 64% 66% 74% 71% 52% 64% have Received 5 or more Days on In- Service Training FSQL No. and % of Grades 123,228 127,095 230,515 239,584 238,146 235,739 to Have Teaching 75% 81% 83% 87% 89% 88% Aids for Reading FSQL No. and % of Schools 9,216 11,248 12,118 13,710 16,308 18129 (all sites) to have 23% 28% 31% 35% 43% 48% Potable Water FSQL No. and % of 73,948 110,137 127,849 202,689 210,161 217,113 Classrooms (all sites) 34% 49% 58% 91% 94% 96% to have a Good Blackboard FSQL No. and % of 140,226 136,904 143,526 150,711 155,996 163,501 Classrooms (all sites) 65% 61% 65% 68% 70% 73% to have Achieved School Construction FSQL Disaggregated See Table expenditure levels by 1/Figure 6 for NTP sub-projects and by disaggregation province (g) 25 across sub- projects Disaggregated funding sources by province (g) Source: DFA data, various years, unless otherwise indicated; Notes: in Italic, estimates; (a) Except for grade 5 achievement, where baseline is for 2001; (b) NER according to DFA data. Issues with the calculation of the primary school age population led to over-estimation of the indicator in 2004, 2005 and 2006; (c) NER according to EMIS data. This recent re-calculation using EMIS data seems closer to reality, although probably somewhat under-estimated for 2007/08; (d) Primary completion rate of the 14 years old; (e) Independent learner benchmark according to Grade 5 studies (MoET-NIES-Melbourne ­World Bank); (f) Not one of the original indicators, only added for information; (g) Disaggregation across provinces has been reported in the FMRs and financial statements. Figure 1: Changes in NER across Groups Figure 2: Changes in Completion across Groups Source: Source: DFA DFA Figure 3: Changes in Grade 5 Achievement across Groups Source: MoET-NIES Component 1: Achievement of FSQL Standards through TBS to the Education NTP From outcome to result indicators. While the effects of Component 1 are to be measured through both the outcome and results indicators, the main primary objective of this component was to improve FSQL indicators, and through FSQL the outcome indicators. Aggregate data taken from the District FSQL Audit (DFA) also reveal good progress in the FSQL Input Index (FII) and the individual FSQL indicators between 2003/2004 or 2004/2005 and 2008/2009 (Table 4). In particular, the FII increased from 62 to 71 percent, the proportion of schools achieving a FII of more than 80 percent from 16 to 24 percent and the proportion of schools achieving a FII of more than 60 percent from 71 to 90 percent. Teacher training (except in the last two years), availability of teaching materials and quality of the infrastructure all improved substantially over the life of the project, showing an impact of the ENTP sub-components (with 26 the limitation of specific attribution). An increase in 9 percentage points in the FII over the life of the project represents between 50 and 24 percent of the FSQL shortfall depending on the FII upper threshold (80 to 100 percent). Taking the Government definition of an FII>80 as standard for achieving FSQL (Decision 55), the broad target of covering approximately one-third of the shortfall in FSQL deficiencies in primary schools can be said to have been achieved. However, this target was achieved focusing more on the schools further away from the 80 standard than on the ones closer to it, as shown by a stronger increase in the proportion of schools achieving a FII of more than 60 percent (see Figure 4 for the evolution on the FII across schools between 2005 and 2008). Along this line, the data show greater progress in the FII achieved among disadvantaged districts and the poorest district quintile (Figure 5 and Table 5). Again, as this is aggregate data, by district classification, it is not possible to attribute all of the improvements to the ENTP or TBS through ENTP (all the more as disadvantaged districts are also included in PEDC). Figure 4: Distribution of FII across Schools Nationwide from 2005 to 2008 S tr ng 3500 3000 2005 2500 2006 2007 2000 2008 1500 1000 500 0 i m FII 0 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 10 Source: MoET, Analysis of DFA Data in the School Year 2007-2008 Figure 5: FII and Poverty Levels from 2005 to 2008 27 75,0 70,0 65,0 FII (2005) 60,0 FII (2006) FII (2007) 55,0 FII (2008) 50,0 W 1 N h óm giàu R ichest Q uin tile 3 N h ó m tru n g 4 N h óm n gh èo 5 N h óm n gh èo V bìn h / M id d le W 2 N h óm giàu H V Q uin tile n h ì/ N ear Q uin tile H Q u in tile Q u in tile K Po o rest n hì/N ear U F L R 5 R W 3 W K Q K Q Source: MoET, Analysis of DFA Data in the School Year 2007-2008 Table 5: FSQL Input Index Nationwide and by Different Groupings21 Indicator FII Number Number of FII FII FII FII Increase (FSQL Input of Primary (2005) (2006) (2007) (2008) of FII Index) Districts and (2005- (2008) Secondary 2008) Schools Nationwide 677 15,610 65.0 67.8 68.9 69.6 4.6 Averagely 204 5,104 66.4 68.6 69.4 69.9 3.5 Disadvantaged Districts Disadvantaged 227 4,978 57.6 60.7 62.4 63.4 5.8 Districts (*) Advantaged 246 5,528 71.5 74.1 75.1 75.4 3.9 Districts Richest 133 2,649 69.1 72.5 73.1 73.4 4.3 Second Richest 133 3,112 68.4 71.2 71.9 72.6 4.2 Averagely 128 3,620 66.7 69.1 70.6 71.4 4.7 Advantaged Second Poorest 138 3,432 64.5 66.9 68.1 68.2 3.7 Poorest 135 2,797 59.3 61.7 63.3 64.8 5.5 Source: MoET, Analysis of DFA Data in the School Year 2007-2008 (*) Also corresponding to PEDC districts. 21 Source: MoET, Analysis of DFA Data in the School Year 2007-2008. 28 From the equity perspective, it is also noticeable that the relative financial weight of the sub- project on support to ethnic minorities and disadvantaged groups has increased in time and remains sustained, representing in 2008 the third most important sub-component (see Figure 6).22 However, it is infrastructure development which has increased the most and this is good news from the perspective of the quality and equity of primary education only if sufficient priority is maintained on primary and if it is sufficiently well targeted to help address FSQL shortfalls in construction. The 2010 planned allocations provide continuous emphasis on the education of disadvantaged groups, with relatively lower increases for infrastructure. 22 And that does not include a share of sub-component 6 on infrastructure directed to ethnic minorities. 29 Figure 6: Evolution in the Share of ENTP Sub-Components over 2006-2008 Source: Table 1 From outcome and result indicators to outputs. At a more disaggregated level, SRV made good progress in each of the five TBS financed Sub-components, as shown below and in Annex 2. · Sub-component 1: Illiteracy elimination, consolidation of Universal Primary Education (UPE) and achievement of Universal Secondary Education Illiteracy is not monitored by MoET on national scale but it is deemed to be very low in Vietnam. The SRV moved toward universal primary education from 2006 to 2008: Net enrollment increased from 94.4 percent to 96.0 percent (or even 100 percent according to DFA 2009 data) and completion rates increased from 92.3 percent to 97.1 percent during the period (raising further to 98.2 percent in 2009). To help move in this direction, this sub-component financed programs to reduce illiteracy, and training of teachers and education managers and teaching-learning materials to support universalisation in mostly disadvantaged provinces, resulting in a higher proportion of teachers trained and grades with learning materials in these provinces. However, the progress in outcome and result indicators cannot be attributed to funding from the TBS operation alone since there were several other projects aimed at achieving these objectives, and funding from multiple sources including provinces, communities and parents. · Sub-component 2: Renovation of the curriculum and textbooks/teaching methods The primary and secondary education curricula were revised, new textbooks and teaching-learning aids conforming to the revised curricula were printed and distributed on a large scale (both in replacement of the existing ones and to address the needs of new curriculum areas) and in-service training provided. These activities also resulted in a higher proportion of teachers trained and grades with updated teaching aids. Textbooks 30 were distributed free-of-charge to children from poor families.23 This intermediate result can be expected to have a significant impact on learning outcomes of children from poor families. · Sub-component 4: Upgrading infrastructure of Teacher Training Institutions and teacher qualifications By 2008, nearly all basic education teachers had received pre-service and/or in-service training, with the result that 99% of teachers have attained the 9+3 level. There was also progress towards raising the level of required teacher qualifications to 12+2: Between 2004 and 2008, the percentage of primary teachers attaining this level increased from 71.9 percent to 84.5 percent.24 The sub-component mostly financed pre-service training to 12+2 for primary teachers, in-service training for teachers and education managers on the new curriculum and textbooks, and infrastructure upgrading for teacher training colleges with priority given to disadvantaged provinces, contributing to the improved teacher qualifications as well as to the proportion of teachers having received at least 5 days of in-service training. As also confirmed by the recent field trips, training was carried out mostly through a system whereby one teacher received training in an institute and then returned to train the other teachers of his/her school. This cascade model is generally considered to be a very cost-effective model. · Sub-component 5: Support for the education of ethnic minorities and disadvantaged regions This sub-component of ENTP mostly provided support to the renovation and refurbishment of provincial and district boarding schools for ethnic minorities and schools in mountainous and disadvantaged areas. This investment has clearly already impacted positively school attendance of disadvantaged and minority groups ­ as indicated by closing gaps in the NER and even completion rate ­ as well as some of the FSQL indicators which are typically lower in disadvantaged areas. However, as was mentioned and shown above, more focus needs now to be given to the still very weak learning outcomes of ethnic minorities. · Sub-component 6: Infrastructure development This sub-component of ENTP supported infrastructure improvements through the construction, renovation and refurbishment of about 13,367 classrooms, contributing to increasing the proportion of FSQL classrooms and schools. However, it needs to be noted that a significant and increasing fraction of classrooms has been and is being built/refurbished in secondary schools and even higher education institutions, making the trend towards post-primary education particularly visible for this sub-component. 23 The joint evaluation team could verify on its school visits at several provinces that in most of the schools all pupils had textbooks and used them in class lessons. Enquiring on this issue the teachers, principals and BoET officials all confirmed that textbooks were provided free of charge to children from poor families. 24 MoET, DFA Report 2007-08. 31 Component 2: Institutional Strengthening for ENTP Administration and Implementation Component 2 was jointly supervised by TBS EFA partners and co-financed by Canadian International Development Agency (CIDA) and DFID. Implementation of Component 2 suffered delays from the outset, mostly due to MoET's weak institutional capacity to manage a large set of activities within an institutionally complex setting, requiring not only inter- institutional coordination at the central level, but also at the provincial and district levels. The fact that the TBS was implemented within the mainstream organizational structure ­ by itself a laudable goal from the institutional sustainability perspective ­ (i.e., without a separate Project Management Unit) only aggravated the burden on an institutionally weak institution. The delays in implementation of Component 2 also came to impact MoET's compliance with the donors' reporting and fiduciary requirements, most of which had counted on progress with implementation of institutional and capacity building activities and the contracting of technical assistance to allow for full compliance. The implementation period of Component 2 continues through December 31, 2010. A detailed Work Plan 2009-2010 will guide the priorities and implementation of Component 2 over the final year of implementation. Details of this Work Plan are described later in this section. Current achievements - from results to output. Despite delays, several activities financed by Component 2 to date have represented important improvements to the institutional framework and the systems under which the Education NTP is operating. Annex 2 contains a matrix of originally expected results by policy theme and focus area, as well as progress to date in achieving those results (main outputs under each expected result).25 Some of the main notable Component 2 achievements include: · Issuance of the MoET-MoF Inter-Ministerial Joint Circular. The Joint Circular regulates the purpose, contents, targeting, and budget management of the ENTP (which covers all levels of education from kindergarten to higher education). In the absence of its prior development, the Joint Circular was prepared by the MoET and MoF, together with the TBS partners, during implementation and finally adopted in December 2008. Adoption of this Joint Circular, although it entailed a long preparation process, was a significant achievement that provides policy, regulatory and managerial guidance and direction to the ENTP program. · Building Capacity for Education Planning (BCEP) Coordination. The integration of BCEP with Component 2 of the TBS has been one of the key achievements of the technical assistance program. The TBS operation had underestimated the complexities of administering and managing ENTP in a decentralized environment given the inter- ministerial framework for implementation. A plan to strengthen administration was developed in coordination with BCEP and will be rolled out in 2009 planning, public financial management, and monitoring and evaluation. BCEP's objective was that of strengthening education planning capacity of central and local levels with localized education planning software (VANPRO) developed, tested, piloted and used by provincial and district planners. All 63 DoETs are using this new instrument to produce 25 Most of these outputs are closely related to the implementation of Component 2. 32 their five year and annual provincial education plans which are used to disaggregate planning data by MoET. More importantly, the approach to planning has changed from one that was input-oriented to one that is output-oriented. The approach has helped to improve effectiveness and efficiency in education planning as well as enhancing stakeholder participation in the planning process. As BCEP closed in September 2009, TBS under Component 2 will continue its approach by: (i) using the resources of the BCEP (staffing, tools, approach and methodology); (ii) transferring coaching to training institutions; (iii) institutionalizing the planning tools developed; (iv) continuing support to provincial and district planners; and (v) planning for integration of VANPRO with other existing databases, and those under development such as EMIS. · Allocation Formula. To address the lack of transparency in the allocation of the ENTP envelope across provinces, Component 2 supported SRV in the development of a needs-based pro-poor allocation formula for ENTP, which has now been applied ­ with some fine-tuning in time ­ since 2008 and represents another clear achievement. The formula's basic principle is a two stage targeting process, through which, in a first stage, the envelopes for the two sub-components not yet targeted but with most potential to be targeted (sub-components 1 and 4) are allocated to provinces according to the level of their FII to make sure provinces with larger FII shortfalls receive more of these sub- components; and, in a second stage, specific allocation criteria ­ taking into account school-age population, students, teachers and achievement of shortfall in specific FSQL indicators ­ are then used to finalize the allocation of each sub-component across provinces. The joint donor's assessment confirmed that ENTP budget allocations from MoET to DoETs are now broadly targeted to allocate increased resources to the poorer provinces, with the less-poor provinces receiving only relatively small allocations26, and, indeed, these findings are confirmed by a simple correlation analysis between ENTP allocated resources by province and the 2006 provincial FII which shows evidence of a increasing negative correlation between the level of the 2006 FII and ENTP allocations over the 2006-2008 time period. This result indicates that well targeted ENTP resources have in part helped close the FSQL gap illustrated above. · Operations Manuals. Although late in the process, FM and Procurement Manuals applicable to the management and implementation of ENTP were finally prepared and rolled out to the provinces through workshops. A draft M&E manual was also prepared which will be finalized in January 2010 when additional international M&E technical assistance assumes this responsibility. · Annual Audits. Component 2 financed the cost of the annual TBS financial management and procurement audits, and the follow-up, including workshops and training in connection with the outcome of those audits. While Bank and donor financing 26 See Draft Joint Donors' Final Evaluation Report, October 2009. 33 contributed only about 10% of the cost of the ENTP, independent financial management and procurement audits (and the subsequent follow-up which involved substantial capacity building) carried out under the TBS operation have covered the entirety of the Program. · Advances in Institutional Coordination on EMIS. Currently, different projects are supporting MoET in data gathering (PEDC) and system development (SREM) but these projects will end in 2010. By that time MoET will not have a comprehensive system that collects all necessary data required for sound education management. During the last few months good progress has been made in the development of the costed EMIS roadmap which connects the vision, values and objectives of MoET. A costed strategic action plan was attached to the Roadmap. The Roadmap has major strengths such as: (i) it defines key roles and responsibilities; (ii) costs scenario for the implementation of a nationwide school management information system; and (iii) includes a short term action plan from now till 2010 which provides opportunities for revisions and updates of the Roadmap. Nevertheless there are still some important challenges that need to be addressed, including the need to develop unified standards and indicators and to define a clearer legal framework for data and indicator management. Component 2 will continue provide support to MoET to refine education indicators, to develop the policy framework for data for education management, and to agree on a single, verifiable data source for agreed national indicators and a single focal point for the collection and management of data. Future priorities. Institutional strengthening activities under Component 2 will continue after the closing date of the TBS operation according to the agreed Work Plan 2009-2010 for the component. The most recent JAR found that activities under this component were picking up. However, continuous effort and monitoring of MoET is needed to ensure that all activities of what is an ambitious work program can be completed in time, including prompt action on making sure all consultant positions are filled. The following were the priorities identified for completion during the slightly over one year remaining: · Improving ENTP Management and Administration. Continued support will be provided towards: (i) improved budgeting and resource targeting; (ii) the training approach and methodology implemented by the former BCEP team now integrated within Component 2; (iii) computerized input and activity monitoring for ENTP; (iv) ENTP payment control mechanisms through the State Treasury's Integrated Treasury and Budget Management Information System (TABMIS); (v) State Treasury capacity building for ENTP in TABMIS at the national level; (vi) a study to finalize ENTP (and other NTP) payment control and management mechanisms; (vii) training to provincial State Treasuries on ENTP payment control and management mechanisms; and (viii) web- based access to ENTP support materials. · ENTP Capacity Building Strategy. This group of activities would support: (i) ENTP leadership and strategy workshops; (ii) provincial and district level capacity building through activities to train the trainers and deliver workshops and coaching; (iii) 34 piloting ENTP short-courses for in-service training on planning, financial management and monitoring and evaluation; (iv) capacity building for decentralized medium-term planning and projection; (v) improved national and provincial FM procedures by strengthening MoET financial management and internal audit; and (vi) monitoring and evaluation of the capacity building strategy. · Improving ENTP Assessment and Evaluation. Support would be provided for: (i) improved data collection, management and analysis by MoET's Department of Planning and Finance (DPF) Statistics Division; (ii) a school mapping pilot; (iii) national and provincial workshops on format, content and utilization of provincial and district results monitoring matrix; (iv) provincial monitoring and coaching field visits; and (v) support to policy assessment and development, including studies on ethnic minority education and development and implementation of the final 2010 and baseline 2015 ENTP evaluation instrument. · Streamlining ENTP Governance. This would support activities to: (i) strengthen inter-ministerial coordination through, inter alia, semi-annual steering committee meetings; and (ii) develop the 2011-2015 Joint Circular to support a continuation of the ENTP through 2015, after assessing implementation of the 2006-2010 ENTP. · TBS Management and Administration. Component 2 would continue to provide support to: (i) the administration of the Secretariat Unit and its periodic reporting function; (ii) Annual and Semi-Annual Reviews and Audit; (iii) ENTP external audits; and (iv) the TBS Component 2 financial audit. The Action Plan provides detailed costing by activities within the above five broad categories and specifies, by individual task, the specific inputs, outputs and timing. 3.3 Efficiency FSQL as a cost-effective intervention. The support provided under the TBS operation increased the efficiency of the ENTP. The economic analysis prepared during project preparation had concluded that "by accelerating the achievement of FSQL, with priority given to schools in the more disadvantaged localities, we anticipate benefits in terms of internal and external efficiency, more rapid achievement of SRV universal primary education goals and improved equity on a household income and geographic basis". Although it is not possible to quantify by precisely how much the TBS operation "accelerated the achievement of FSQL", it is clear that it did support a prioritization of disadvantaged districts and the poorest communities and better quality of service delivery, by supporting the most critical ENTP sub-components, the introduction and use of FSQL for allocation and monitoring, and the introduction of other important institutional changes (notably in planning practices). In turn, these improvements were conducive to achievement of benefits in internal and external efficiency through improved learning outcomes, 35 reduced dropout rates, increased completion rates and transition rates to lower secondary, improved academic performance in lower secondary as a result of improved learning outcomes in primary, and higher earnings for those who enter the labor force as a consequence of improved academic performance overall, all of these with an especially greater impact in disadvantaged areas. To make this case stronger, a replication of the economic analysis with the newly available Grade 5 results (see Annex 3) confirms that, after controlling for "family and student backgrounds and personal characteristics" (which have by far the highest influence on student achievement in both math and language), the achievement of FSQL is the next most important factor influencing achievement (and probably the second in terms of cost-effectiveness) after school head and teacher improved pedagogical practices (admittedly the most cost-effective intervention) and the adoption of full day schooling (effective but costly). Donor coordination and transaction costs. There are other efficiency-enhancing measures to which the project contributed. While donor coordination in support of the program was intense and structured (therefore requiring important up-front preparation for rather formal Annual and Semi-Annual Joint Reviews), it most likely reduced the individual transactions costs for the government of working independently with individual donors on perhaps less coordinated programs. Despite some initial differences in approach, donor coordination worked well, and served to present a unified approach in support of an established government program, where donor funding represented only about 10 percent of the program costs. Program leverage. The donors' leverage in supporting systemic improvements in the management and administration of the ENTP which made and is making a positive impact on the country systems used for implementing this (and other) government programs was high, especially in comparison with what was this rather small proportional investment on their part. Decentralization and efficiency. The decentralization supported by the ENTP and further strengthened by some institutional improvements introduced by TBS (in provincial planning notably) served to mobilize local resources for the sub-components supported by the program, and at the same time enhance social efficiency by providing greater "voice" and delegating the process of determining priorities to provinces, districts, and schools and communities. The draft Final Evaluation Report concludes that the major role for and significant funding from provincial level administrations and local communities has been a significant factor in the implementation of the ENTP. Efficiencies in output delivery. Among the ENTP Sub-components supported by the TBS operation, Sub-components 2 (Reform of Curriculum and Textbooks), 4 (Teacher Retraining and Upgrading Short Courses), 5 (Support for the Education of Ethnic Minorities and Disadvantaged Regions) and 6 (Improvement of School Infrastructure) were noted27 for either promoting high efficiency in education or being highly efficient in their delivery. The reform of curriculum and textbooks was considered to have high efficiency in that the textbooks are targeted directly to poor children, since textbooks are provided free of charge only to children from poor families whereas less-poor families need to purchase the textbooks. The carrying out of teacher retraining and upgrading courses was considered highly efficient, with courses provided on a 27 Through analysis of secondary data and field-work in several provinces carried out during the donors' external review. 36 cascade basis where either: (i) a teacher from a school or district was sent to training to later return and deliver that training to other teachers in the school or district; or (ii) a teacher in a province was sent out to train other teachers in a poorer district or school in that province. In Sub-component 5, efficiency-enhancing measures included the provision of free textbooks and teaching aides to schools in disadvantaged areas, and the repair, renovation, and furnishing of dormitory schools and semi-boarding schools which allow disadvantaged students in remote areas to attend classes more regularly. Finally, there was generally high efficiency in the building of new classrooms in poorer schools and/or districts, although there may have been some exclusion of some poorer schools and/or districts that did not benefit as of yet. Pupil-teacher ratio. On a less positive note, the project failed somewhat to induce more efficiency in service delivery through an increase in pupil-teacher ratios. These continued to decrease during project implementation due to a combination of declining school age population and rigidities in teacher management (see also Annex 3). Component 2 delivery. Finally, the efficiency of the intervention could have been enhanced had the implementation of Component 2, and related capacity building, taken place earlier. This would have helped address earlier on some of the still unresolved fiduciary issues of ENTP, which decreased the potentially efficiency-enhancing effect of using country systems. 3.4 Justification of Overall Outcome Rating The Overall Outcome of the TBS operation is rated as Moderately Satisfactory.28 As described above, the Project, and the Government program that it supported, continue to be extremely relevant to the country's priorities today, and the implementation of the program and the further strengthening of the various systems that are needed to support the Government in its implementation, are ongoing, and continue to count on the Government's financial support (at least up to 2015) and donor technical and financial assistance through the continuation of Component 2 (until December 2010). During the operation's implementation period, access to and quality of primary education improved, as did equity. There were, however, some shortcomings and delays in implementation of the operation's fiduciary strengthening activities which resulted in delays in addressing FM and procurement weaknesses. These weaknesses are now expected to be addressed through the recent fiduciary efforts and the continuation of institutional strengthening activities under Component 2. Progress towards the accomplishment of the program's (and operation's) objectives was made with consideration to improving efficiency, both in the operation and strengthening of the Government's ENTP program and in the delivery of assistance by the donors. 3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development In its design, the TBS operation assumed that the use of the ENTP would enhance pro-poor allocative efficiency and provide additional resources for the accomplishment of agreed sector 28 Within the World Bank rating scale: unsatisfactory; moderately unsatisfactory; moderately satisfactory; satisfactory; highly satisfactory. 37 priorities, especially targeted to disadvantaged groups, supporting lower poverty and social development. There is evidence, through aggregate data (see Table 5) that the FSQL for disadvantaged districts and the poorest areas of the country witnessed proportionately greater improvements in FSQL than others, and well above the national average. These improvements in disadvantaged areas are difficult to attribute exclusively to the TBS operation or ENTP program, but they clearly played a significant role. Vietnam is on track to achieve gender parity in primary education with male students catching-up on female students, as indicated by the latest outcome results. (b) Institutional Change/Strengthening During preparation it was unquestionable that the donors had overestimated MoET's capacity in terms of regulatory powers, procedures, legislative backing and overall institutional capacity to manage such a complex program. This resulted in a need for donors to focus an increasing amount of attention on institutional strengthening. Viewed from a different perspective, however, this could be seen as being one of the most important aspects of the donors' value- added to the SRV. The TBS supported the selected Sub-components of an ongoing program, while at the same time aiming to strengthen the government's capacity to implement that program. Component 2 was specifically designed with that as an objective. In retrospect, significant progress can be attributed to the TBS operation in institutional strengthening, starting with the issuance of the Inter-Ministerial Joint Circular that regulated the purpose, contents, targeting, budget management and implementation of the ENTP, which represents a significant achievement in giving agreed policy, regulatory and management directions to the program. The achievements under Component 2 (as documented in Section 3.2 above) all demonstrate the significant institutional impact the TBS operation is having (and continues to have with the continuation of this Component for another year) on the ENTP and the MoET's institutional strengthening. Importantly, as documented in the Borrower Completion Report, TBS has also fostered an increased demand for and awareness of the importance of good management practices for ENTP at the central and provincial level which is already translating in high priority for continuous/further planning, financial management and M&E training at the decentralized level in the new 2011-2015 ENTP Joint Circular. (c) Other Unintended Outcomes and Impacts The multi-donor nature of the TBS operation increased cohesion and interventions by the involved donors, and presented a unified voice in working with the MoET. 3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops Not applicable. 4. Assessment of Risk to Development Outcome The Risk to Development Outcome is assessed as Negligible. TBS funding for the eligible Sub- components represented a minor amount of its total funding and it was implemented within the Government's existing framework, utilizing Government systems. The operation's most 38 significant impact in terms of influencing Government policies and systems was provided by the much smaller (in terms of funding) Component 2, which, as mentioned earlier, has already produced, and will continue to produce, important institutional results. The SRV has committed to providing a steadily increasing percentage of the state budget to the education sector (currently estimated at just over 20 percent). There is some concern, that over time, ENTP funding has been allocated increasingly to secondary and even higher education. However, resources from the ENTP are basically allocated and managed by the provinces, with direct involvement of the PCs, and this bodes well for a continued focus on the program's priorities and, together with the continued emphasis on FSQL, targeting towards disadvantaged groups. 5. Assessment of Bank and Borrower Performance 5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry The Project, whose preparation the Bank together with other donors facilitated, was innovative, and was a good vehicle for responding to the Bank's assistance strategy and the government's request for strengthening their program that responded to their priorities. It built on and complemented a strong program of assistance to SRV's education sector. The overall framework for donor provided budget support to finance priority sub-projects within the ENTP, the pooling of resources and use of common fiduciary and M&E systems, along with a program of technical assistance for institutional strengthening, characterized what is basically a SWAp to investment lending. Possibly given the limited experience with SWAps at the time of preparation, possibly given the evolution in Bank operations towards requiring that project preparation be advanced to permit ready implementation after signing, the very systems needed to implement, monitor and supervise the project were not in place at the time the project and IDA's financing were considered by the Bank's Board. At the same time, the project's risks, especially with respect to its fiduciary aspects, were underestimated, and the mitigation measures that were identified were not sufficient to mitigate what were in retrospect important risks. For this reason, Bank performance in ensuring Quality at Entry is rated Moderately Satisfactory. (b) Quality of Supervision (including fiduciary and safeguard policies) The Bank's supervision effort was timely, focused, coordinated with government and donors in a structured process and aimed at supporting the government in improving implementation through the strengthening of its ENTP and the required governmental systems. In part because of the project's design which provided support to the development of required FM, procurement, payment and M&E systems during implementation, in part because of the ascertained 39 weaknesses in these systems, the Bank's supervision was heavily focused on the operation's fiduciary aspects, and the implementation of technical assistance included under Component 2 that was designed to strengthen those systems and the program. Supervision reporting was candid as to the problems the project faced during implementation. The Bank also placed a lot of emphasis on the achievement of FSQL, and worked closely with the MoET to ensure continued focus on this objective in order to make progress towards the project's PDOs. The Bank worked closely with the other TBS partners and the government in the JAR and JSARs, and their follow- up, to ensure that TBS partners were presenting themselves as one voice to the client, even though there were areas where donors' perspectives differed (at least at first, there was some difference of opinion as to how "hands-on" donor supervision should be for what was essentially a program of budget support). When issues arose, the Bank promoted and supported the development of specialized Working Groups as an efficient method to analyze and address problems. There is some concern expressed in the joint external evaluation that perhaps were it not for the needed emphasis on fiduciary aspects during supervision, more attention could have been devoted to the impact of the project on learning outcomes and pedagogical aspects. In all, the Bank's supervision efforts is considered Satisfactory, especially in view of its drive to focus on and respond to priority issues as they emerged, and its drive to leave in place well functioning governmental systems that can be used to implement in the future the ENTP or other educational programs effectively and efficiently. (c) Justification of Rating for Overall Bank Performance The Bank's Overall Performance is rated Moderately Satisfactory. Despite strong supervision efforts which resulted in the TBS operation achieving in good part its Development Objectives, shortcomings in overestimating the government's institutional realities and capacity to put in place rapidly the strong systems needed to implement and manage the ENTP forced supervision efforts to focus on issues that would have been better addressed during preparation. In the end, however, the Bank's supervision efforts seized the opportunity of project supervision to work with the government and the donors in unison to leave in place a stronger education system, which, albeit technical assistance is still ongoing under Component 2, will have the instruments needed to implement, monitor, plan strategically and supervise further progress in the ENTP and other education programs. 5.2 Borrower Performance (a) Government Performance The Government's Performance (i.e., the MoF and the broader government) is considered Moderately Satisfactory. The government has continuously assigned priority to the education sector in general and the ENTP, in particular. From 1997 to 2008, total public expenditure for education has increased by almost 300 percent in real terms, from 3.5 percent of GDP to 5.0 percent of GDP, and from 14 percent to 20 percent of total public expenditures. The Government honored its commitment to increasing the yearly budget for the ENTP by approximately 10 percent per annum from the 2005 level over the implementation period (see Table 3). Since ENTP funding represented approximately 5 percent of the total education budget, of which 95 percent is allocated to teachers' salaries, ENTP funding in effect doubled the funds available for non-salary expenditures in priority programs. However, implementation was 40 also affected by systemic issues that went beyond strictly those of the education sector per se and which may have deserved stronger focus from the Government. For instance, the Government supported the work towards issuance of the Joint-Circular, but the process was slow and would have deserved further prioritization to provide for smoother implementation earlier on in the program implementation. Another case was the constant yearly under-spending of allocated ENTP budget, caused in part by the late release of budgets to provinces and districts. Overall, the government's performance was responsive and ready to absorb technical advice on improvements that addressed the strengthening of systems used not only for the education sector, but also those central systems, such as TABMIS and the MTEF budget framework with which the education sector by definition needs to interact. Had a bit more focus been put on the program's overall framework earlier on, the program could have achieved even more. (b) Implementing Agency or Agencies Performance The Implementing Agency's Performance is also considered Moderately Satisfactory. MoET's capacity to implement this ambitious project was never raised as an explicit risk by the preparation effort, and perhaps overestimated. Implementation by definition was to be through the ministry's existing departments, coordinated by its Planning and Finance Department. There was to be a full-time Program Manager that acted as a focal point for the ENTP as a whole, and SRV had committed to "ensure that the project is supported by additional staffing and related resources within the concerned ministry and departments". At the same time, it was recognized that there were wide disparities in implementation capacity of provinces and districts that would be receiving ENTP funding, and this was initially subsumed under at least two project risks. Against this complex environment, working through its own existing structure and systems that were in the being improved, the MoET faced a considerable challenge but persevered and managed to move ahead with the administration of the ENTP itself, and the institutional strengthening activities aimed at providing its own structure with important tools for the program's strategic planning and management, as well as for interfacing with government systems being developed and strengthened by other ministries. Notwithstanding this achievement, MoET could have had a more pro-active attitude in addressing issues as they arose, implementing Component 2, and following-up on the recommendations and technical advice provided during and after the joint reviews. This is particularly clear in the case of fiduciary processes were MoET performance can be considered moderately unsatisfactory. MoET was slow in submitting the audited financial statements and audit reports, following-up on the audits and donors' recommendations to improve financial management and procurement systems, and taking action to ensure more financial management support for the program, better coordination across entities for financial consolidation and reporting, and faster delivery of the fiduciary manuals and related workshops. Recent efforts underline however renewed commitment to the improvement of fiduciary processes and the continuation of Component 2 with continuous emphasis on fiduciary systems bodes well for further improvement. (c) Justification of Rating for Overall Borrower Performance The Overall Borrower Performance is considered Moderately Satisfactory based on the Government's and Implementing Agency's performance in implementing this complex program 41 through its existing institutional structure, while at the same time strengthening the very governmental systems that were needed to support the program. 6. Lessons Learned There are several lessons learned from the Bank's financing of the TBS for ENTP operation. Many of them are perhaps not worth reiterating since they are well known and apply to all operations that the Bank finances: the importance of consistent Government commitment to the outcome of a project, the need to have at least minimum FM, procurement and M&E systems in place before implementation begins, and the importance of having risks well-identified and made explicit, to name a few. Rather than focus on these, in view of the rather innovative nature of the TBS operation, and the increasing reliance on SWAps as an approach to structure investment projects, it would be relevant to focus on several of the lessons that the implementation of this operation provides, especially with respect to its design. · Need for basic systems before structuring an investment operation as a SWAp. SWAps can provide an excellent mechanism for supporting and, in the process, strengthening existing Government programs, and scaling up the benefits from improved capacity to apply to the entirety of the program being supported. Very few projects count on perfect FM, Procurement and M&E systems at the outset this is often an added objective in support for any project but there are basic minimum requirements that should be in place before implementation begins. This is all the more important in the case of SWAps, where external finance is pooled together with government funds and where important financial and monitoring information, and procedures, will make use of the very systems that are being strengthened for purposes of their supervision, both by the Government and by donors. It is difficult to judge what is this "minimum" that should be in place before implementation, but it is clear that at least, the basic manuals that guide fiduciary and other aspects should be available at the outset, even if the assistance to be provided will support their improvement, strengthen and require the eventual updating of project requirements in those manuals, and also that all agencies involved in implementation are familiar with at least the initial basic requirements. This is all the more important when a program is implemented in a decentralized environment, which is often the case for SWAps. In the case of Vietnam, the SIL + "budget support" approach which eventually translated into a SWAp, while a conceptually sound approach to reap the benefits of both a SIL and a broader budget support (targeted sector financing and TA, with increased sustainability, potential for strengthening country systems and donor harmonization), ended up being very difficult to implement in the complex and low capacity institutional Vietnamese environment, especially in the absence of sufficient upfront mitigation measures, making a somewhat more traditional SIL approach (at least from the perspective of the use of fiduciary systems) probably a safer bet at the time. On the other hand, with better basic systems and upfront mitigation measures in place the TBS approach could be worth replicating. · M&E/attribution. Following from the above lesson, and generally obvious, good M&E systems are needed in order to track a project's or a program's outputs and outcomes. More importantly, however, is the question of how to be able to attribute 42 those outputs and/or outcomes to the external financing in an environment where government and donor funds are pooled, as is generally the case for SWAps. On the one hand, it could be argued that a small amount of external financing is used to leverage the operation's impact on the entirety of a program, and therefore reliance on broad country- wide program-level indicators could be justified. On the other hand, however, it becomes all the more important to define carefully those specific outcomes, or intermediate outcomes, that can be realistically achieved during the operation's implementation period. This is all the more difficult in the case of education, where there are indicators that often work at odds with each other (e.g., expanding coverage in remote, poorer areas often results in a deterioration in quality that affects the entire system), and where the achievement of final, and even sometimes intermediate, outcomes more often than not spans well beyond the implementation period of a somewhat fast-disbursing operation. · Donor coordination. SWAps with multi-donor financing offer a useful vehicle to coordinate assistance in support of a government program. While differences of opinion may well exist among donors, the very process of coordinating, participating and working together with the government presents an environment conducive to addressing those differences to present "one voice" to the client. · Impact on executing agency. At the same time, while donor coordination has obvious advantages in terms of presenting a unified position, not supporting conflicting objectives, etc., and the requirements of donor coordination on the client are purportedly simplified and made consistent among donors, this does not necessarily lessen the administrative burden on the implementing agency since a lot of often more formal up- front preparation is required on their part to prepare and participate in joint assessments of implementation. Even when a SWAp modality is used, working through the existing structure of government and especially the executing agency, additional experienced staff for handling donor-specific requirements and activities is warranted. · SWAp offers good opportunity for coordinating with other programs implemented in country. Through the leverage, visibility and knowledge that providing support for a large, broad, ongoing program affords, SWAps present good, catalytic vehicles for developing synergies with other stand-alone operations that can benefit the program and used to its advantage. The synergies between the TBS and the stand-alone PEDC and BCEP operations worked positively to strengthen the planning and management of the education sector as a whole. · Capacity building. Programs of technical assistance for institutional strengthening are often integral parts of SWAps. It is important that this capacity- building is set up early in the implementation process ­ with related staff- to support early institutional, and, particularly, fiduciary strengthening. The sequencing between the capacity building component and the remaining of the program needs particular attention. 43 Consideration should be given to starting with capacity building before the budget support itself if existing systems are weak. · Monitoring and follow-up. In complex decentralized settings, which are often the SWAps environments, it is important to make sure the central level and other relevant agencies implement regular follow up and assistance to decentralized levels to make sure fiduciary processes are well understood and implemented. · Sector dialogue. SWAPs do also offer a good opportunity to address systemic sector issues, such as, in this case, broad education quality and equity issues, by coordinating with donors and Government around a common sector strategy. It is important to use them to maintain a vivid and continuous policy dialogue on technical and strategic aspects. However, basic minimum requirements for country systems need to be in place to maximize the potential of this policy dialogue. In their absence, as was the case for TBS, too much time and energy may be focused on getting fiduciary systems right at the expense of the sector strategic dialogue. The above lessons are all derived and therefore also directly apply to the TBS operation. In addition there are four other main lessons which are more specific to TBS itself: · Need for coordinating agency. Beyond better monitoring and follow-up from the central to the decentralized level, in the case of TBS, it would also have made sense to set-up a coordinating agency which, in terms of financial management, would have been tasked to prepare the financial statements at the different levels and consolidate on a program-wide level. · Making better use of existing system strengths. Instead of relying on a parallel MoET/Treasury reporting system difficult to reconcile, it may have been wiser to maximize the role of the Treasury in reporting given its overall more developed financial reporting expertise and generally more reliable financial reporting system. · Need for complementary quality-enhancing interventions. This operation has confirmed the validity of focusing on minimum quality levels for schools as an efficient tool to improve education quality (and learning outcomes) in Vietnam. Further analysis, however, also confirmed the effectiveness, in Vietnam, of other quality-enhancing measures not directly taken into account in minimum quality levels, such as teacher and principal feed-back practices and more time to learn (full-day schooling). These measures may need to be taken up by other operations to complement the impact of FSQL on quality.29 As more and more schools achieve FSQL, it may also be important to develop and formalize "enhanced" FSQL indicators with more emphasis on teaching and learning aspects, including pro-active pedagogical practices and time to learn. 29 The new World Bank operation taken to the Board in June 2009 (SEQAP) will address the issue of transition to full-day schooling. 44 · Quality-enhancing interventions for ethnic minorities. Finally, the operation has also shown that achieving minimum FSQL quality levels may not be sufficient to close the gap in learning outcomes between the Kinh and ethnic minority groups in Vietnam, highlighting the importance of introducing or at least considering other learning outcomes' enhancing measures, such as culturally-sensitive pedagogical practices, flexible school schedules, sufficient time to learn, Vietnamese language strengthening courses and teaching of local languages, as complementary measures to maximize impact on ethnic minority groups. Continuous focus on the particular challenges faced by ethnic minority is imperative in Vietnam. 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners (a) Borrower/Implementing Agencies Implementation and results ­ strengths of ENTP/TBS-EFA. MoET recognizes some key strenghts of the ENTP which support the findings of this ICR. These are that this system helps provinces be active in budget allocation to programmes/projects in the geographical area; mobilise additional resources depending on the current situation and actual requirements of each province; and actively hamornize National Targeted Programmes within the same geographical area. TBS-EFA was supportive of and even encouraged this approach. Implementation and results ­ weaknesses of ENTP/TBS-EFA. MoET also points to some weaknesses of the system which can be classified under three main headings: · Cons of decentralisation. Budget allocation in a number of provinces may not follow the notional budget structure which is provided to implement the prioritized objectives of each project in accordance with the general directions of the sector and therefore result in fragmented interventions, leading to much construction work in progress, not meeting the progress set out, etc30; · Limitations of authority and capacity of DoET and MoET. Less importance is attached to the role of the DoET to give advice to provincial/municipal People's Committees on the administration and budget allocation for the ENTP. The management and direction capacity of the central executive agency (MoET) is limited resulting in difficulties to achieve the general objectives set out. Related to the two above weaknesses, the periodic reporting of provinces and other Ministries' agencies to the executive agency has not yet been followed strictly, resulting in difficulties for the executive agency to consolidate and assess the implementation of the National Targeted Programmes of each sector before submitting to the Government and relevant agencies as required. The reality shows that in those provinces/cities where DoETs take the lead in the advice given to provincial/municipal People's Committees, the implementation of the ENTP has generally been done according to the objectives set out and general directions of the sector and reporting requirements have been strictly followed with high quality, for example: Tien Giang, Kien Giang, An Giang and Binh Phuoc. 30 Clearly in the eyes of MoET decentralization is both a blessing and a curse and they have trouble in finding the right balance between central and local priorities. 45 · Weak application of TBS budget codes. Budget project codes have taken time to be clarified during the budget delegation to relevant units in many provinces as required or targeted supplements from provincial budget for district, commune budget and payment order-based allocation. As a result, allocation to different projects in the geographical area and thus, the accounting for the ENTP funds, has not been done comprehensively; has been difficult to be monitored and assessed, and is fragmented. This led to the result that funds are used less effectively and it is difficult to achieve those objectives as planned. · Starting earlier with Component 2. In both design and operational terms, MoET concurs that, with the benefit of experience, Component 2 of TBS-EFA should have been begun before Component 1 or at the least concurrently with it. Building capacity earlier in the program through the style of activities in Component 2 could arguably have reduced some of the problems encountered in implementing Component 1. Others · World Bank rating. Interactions between the WB and the MoET and other SRV ministries and agencies concerned with the loan have been uniformly balanced and productive. Staff of the WB have been consistent supporters, useful advisers and, on occasion, relevant critics in helping TBS to maintain its momentum and sense of direction in strengthening EFA. These elements justify an overall positive performance of the World Bank. The World Bank with other donors has been properly concerned to improve the quality of procurement and audit procedures and of underlying financial management practices. However, the World Bank and other donors may not have fully appreciated the dynamism but also complexity of the Vietnamese situation when, in 2005, the Project Appraisal Document and the Memorandum of Understanding were finalized, resulting in unrealistic expectations about TBS. In retrospect, risk assessment was probably not as thorough or realistic as it needed to be; or perhaps too much reliance was placed on the assessment done for or through the perspective of PEDC which has a different governance/management structure and implementation. · Government's support. There has been high level SRV support to MoET in its implementation activities underpinning E-NTP and TBS-EFA and this is seen in the many examples of legislation and regulations. · MoET's performance. MoET in collaboration with other ministries and provincial and district units has steadily and consistently provided support and encouragement for new and improved planning and financial management practices. Notwithstanding this effort, in general, capacity for monitoring and evaluation in all areas of education and management in Vietnam is not yet as well developed as needed for a Ministry engaged on modernization. Until 2009, a lack of capacity may be seen in the area of financial management and reporting at DoET and BoET levels when training in new procedures and requirements has not always kept pace with the decentralization of public administration functions. For instance, until recently E-NTP financial reports 46 from provincial authorities were submitted late or not at all. However, for the JAR in October 2009, there was a marked improvement with 50 of 63 provinces submitting reports on time compared with 29 received in April 2009. The division of responsibilities for E-NTP financial management and reporting among concerned entities at the sub- national level such as MoET, State Treasury, and MoF is also progressively becoming clearer. · Underspending. Underspending is a systemic whole-of-government issue in Vietnam. During the latter part of 2009, and particularly the last quarter, TBS funded workshops are supporting MoET in collaboration with other ministries, particularly State Treasury to provide provincial level training and capacity building to address the efficiency aspects of procedures at provincial level. Thus far six national workshops have been held in the October/December period involving multiple stakeholders. This is a direct and positive response to audit and other findings about underspending even if outside the exact time frame of the World Bank report. · M&E: MoET has recognised the importance to planning and management of effective M&E for E-NTP and across operations. An M&E manual draft has been developed. It is planned in TBS2 Work Plan that training workshops in provinces will include an M&E session. Using budget savings an additional international M&E adviser has been contracted to help implement Workplan activities from January 2010. · Additional lessons learned. The lessons of this report are shared by MoET. Additional ones are: MoET to annually assign tasks with detailed performance indicators to be completed by each province in each year; as provided by regulation, assign to each DoET the key role in allocating E-NTP expenditure of each province; for the provinces not following regulations in reporting and other important matters, consider termination of their E-NTP allocation; assist and support improved reporting through the development and training in the use of templates, particularly relating to Joint Circular 125 of 2008; undertake a sample study of effective E-NTP management in different regions to create a flexible basis for training and socialization nationwide, and reach the higher efficiency in E-NTP implementation; consider in the new ENTP a resource allocation cycle of five years to assist localities to meet their objectives (provincial suggestions); funding for M&E activities should be provided in budgets as well as expenditure norms for implementation, and if an education sector entity lacked M&E performance capacity, it should be possible to hire external consultants based on a separate budget (provincial suggestions); to allow completion of a specific capital investment to meet planned provincial objectives, ENTP resources might be focused, within an annual cycle, on one sub-component. 47 (b) Co financiers This ICR includes the perspective of the several co-financers in many different ways. First, and most importantly, a joint evaluation of the program (commissioned by the EC) has been undertaken from the 21st of September to the 21st of October 2009 by a team of external evaluators. This evaluation entailed a comprehensive review of all aspects of the program, including fieldwork in six provinces, interviews with multiple stakeholders at both central and local level, and significant document and data gathering and review. The evaluation gave rise to a comprehensive report which has been submitted for comments to the TBS partners and Government and has constituted a critical input for this ICR. Its executive summary is enclosed in Annex 8. Second, this ICR has been shared with all TBS partners and incorporates many of their editorial and substantive comments. Third, some additional comments on some substantive aspects of the report are enclosed below.31 · TBS and efficiency. The argument presented in this report follows the logic of the initial economic analysis of 'anticipated benefits in efficiency' through an acceleration of FSQL achievement. While one might 'anticipate' a correlation between acceleration of FSQL and efficiency, the fact alone that FSQL improved does not definitively ensure efficiency gains due to TBS. To get a proper read on efficiency, one would need to determine that the marginal increase in resources (GVN and Donors) resulted in a disproportionately larger increase in outcomes (FSQL, other outcomes). That is, each additional dollar spent resulted in more in terms of outcomes than the original dollars spent prior to this intervention. This is virtually impossible to pin down. · TBS and capacity building. "Consideration should be given to starting with capacity building before the budget support itself if existing systems are weak." It is suggested even further, that regardless of existing system strengths/weaknesses, the capacity building component should commence prior to the budget support as this will allow for enhanced engagement with the systems and a better, more accurate understanding of their strengths/weaknesses. · TBS and budget support modality. Donors have had access to Government of Vietnam documents which they never would have had with a project. The quality of these documents generally has improved over time. There is the expectation that the lessons learned from implementation of the programme will have a long term impact. If TBS had been a traditional project it is questionable whether the same lessons would have been learnt. 31 Without differentiating among donors. 48 · TBS and educational outcomes. There should have been a greater balance between reporting and taking action on pursuing broader educational outcomes and research and studies and a move away from the current concentration on FM and audit reporting on the administration of the education TBS. · TBS and visibility. In any future programmes, greater attention should be given to visibility and specific responsibility should be allocated as to who or which donor should deal with this. (c) Other Partners and Stakeholders [not applicable] 49 Annex 1. Project Costs and Financing (a) Project Cost by Component (in USD Million equivalent) Actual/Latest Appraisal Estimate Percentage of Components Estimate (USD (USD millions) Appraisal millions) Component I: achievement of 123.00 123.00 100.00 FSQL standards Component II: institutional 5.00 2.60 52.00 strengthening for NTP Total Baseline Cost 128.00 125.60 98.12 Physical Contingencies 0.00 0.00 0.00 Price Contingencies 0.00 0.00 0.00 Total Project Costs Front-end fee PPF 0.00 0.00 .00 Front-end fee IBRD 0.00 0.00 .00 Total Financing Required 128.00 125.60 98.12 (b) Financing Appraisal Actual/Latest Type of Estimate Estimate Percentage of Source of Funds Co financing (USD (USD Appraisal millions) millions) BELGIUM: MoFA-Directorate- 6.00 6.00 100.00 General for Int. Coop.(DGIC) Borrower 380.00 380.00 100.00 CANADA: Canadian International 11.00 9.70 88.00 Development Agency (CIDA) UK: British Department for 38.00 36.90 97.00 International Development (DFID) EC: European Commission 20.00 20.00 100.00 NEW ZEALAND, Govt. of (Except 3.00 3.00 100.00 for Min. of Foreign Affairs) International Development 50.00 50.00 100.00 Association (IDA) 50 Annex 2: Outputs by Components/Sub-Components Component 1 51 ENTP TBS- Achievements of 2006-2008/09 financed Sub- component Main Outputs (2006-2008/09) FSQL result indicator which can be related (*) to sub-component Indicator 2005 2009 Sub-Component 1: -Programs to reduce illiteracy -No. and % of 237,738 222,303 Eradication of and/or prevent illiteracy Teachers (all sites) to 66% 64% Illiteracy, recurrence implemented in the having received 5 or Universalisation ofpoorest provinces more Days on In- Primary Education -Surveys conducted and data Service Training FSQL (non and Achievement of collected on measuring and cumulative) Universal assessing universalisation Secondary -Training of teachers and -No. and % of Grades 127,095 235,739 Education education managers to address to Have Teaching 81% 88% the challenges of Aids for Reading universalisation FSQL -Additional teaching and learning materials for primary education, and teacher salary supplements, provided to all schools with less than universal primary education -Additional teaching and learning materials for secondary education, and teacher salary supplements, provided to schools with less than universal secondary education Sub-Component 2: -Renovation of the curriculum -No. and % of 237,738 222,303 Renovation of for the primary and lower Teachers (all sites) to 66% 64% Curriculum and secondary education cycle having received 5 or Textbooks/Teaching completed or consolidated more Days on In- Service Training Methods during the reference period FSQL (non - Printing and distribution of cumulative) textbooks, teaching aids and teacher guides on a large scale -No. and % of Grades 127,095 235,739 (including the replacement of all to Have Teaching 81% 88% books and minimum teaching- Aids for Reading learning aids for general FSQL education) -In-service teacher training on new curriculum (about 140,000 teachers) 52 Sub-Component 4: -Pre-service training to 12+ 2 -No. and % of 237,738 222,303 Upgrading Teacher provided to about 2,000 teachers Teachers (all sites) to 66% 64% Training -In-service training mostly on having received 5 or Institutions and new curriculum and textbooks more Days on In- Teacher Training provided to about 200,000 Service Training FSQL (non teachers and 25,000 education cumulative) managers -Infrastructure upgrading (including construction, renovation, equipment and other facilities) provided to teacher training institutes of mostly disadvantaged provinces Sub-Component 5: -Construction, renovation of, and -No. and % of Grades 127,095 235,739 Support for provision of equipment, supplies to Have Teaching 81% 88% Education of Ethnic and learning materials for, Aids for Reading Minorities and classrooms, laboratories, FSQL Disadvantaged libraries, canteens, dormitories, -No. and % of Schools 11,248 18,129 Groups and multi-functional rooms of (all sites) to have 48% provincial and district boarding Potable Water FSQL 28% schools for ethnic minorities -Construction of additional -No. and % of dormitories for semi-boarding Classrooms (all sites) 110,137 217,113 schools for ethnic minorities to have a Good 49% 96% -Construction and renovation an Blackboard FSQL provision of learning materials for schools in mountainous and -No. and % of disadvantaged areas Classrooms (all sites) 136,904 163,501 to have Achieved 61% 73% School Construction FSQL Sub-Component 6: -Construction, renovation of, an -No. and % of Schools 11,248 18,129 Infrastructure provision of equipment and (all sites) to have 28% 48% Development furniture for, 13,367 classrooms Potable Water FSQL ­ including pre-school, primary, secondary and higher education -No. and % of 110,137 217,113 Classrooms (all sites) 49% 96% to have a Good Blackboard FSQL -No. and % of Classrooms (all sites) 136,904 163,501 to have Achieved 61% 73% School Construction FSQL 53 (*) but only partly attributed Component 2 Policy Matrix (original) Issues Starting Year Issues Starting Year Issues Starting Year 2006 2007 2008 A. Education Management Enhancing decentralized · Assess central, provincial · Review of Provincial 5- planning and decision and district level financial Year Planning approach making and performance reporting and differential trends in mechanisms, including progress towards the role of the inspectorate reducing FSQL gaps · Assessment of central, · Approaches to capacity provincial and district assessment and level NTP management enhancement in the capacity sector Monitoring education · Mainstreaming indicators · Assessment of student participation, quality and of Fundamental School achievement in Grade 5 student outcomes quality Levels into EMIS Promoting community · Identification and involvement and school application of options to accountability strengthen bottom-up community-level accountability B. Education Finance Strengthening resource · Education budget amounts · Education budget · Education budget allocation to 18.1% of national amounts to 18.6% of amounts to 19.1% of budget national budget national budget · Proposals for formula · Review of trends in non- · Proposals for funding funding and national salary expenditure share formula to ensure that a allocation of the ENTP in education sufficient level of budget reaches schools to deliver an adequate and effective level of service Enhancing pro-poor · Assessment of · Impact of private costs · Plans for expansion of budgeting and spending interventions to enhance to education and local- full-day schooling for educational outcomes of level monitoring by disadvantaged children ethnic minority students social organizations of · Mid-term review of primary school fee ENTP 2006-2010 exemptions for poor implementation children Strengthening financial · Progress in the set up of management and auditing an integrated Treasury and Budget Management Information System Inter-ministerial · Adoption of Education coordination between Medium Term MoET, MPI and MoF Expenditure Framework 54 Policy Matrix (main outputs) Policy Themes Focus Areas Indicators Progress To Date A. EDUCATION MANAGEMENT A1 Enhancing A1.1 Financial Strengths and weaknesses of · Financial Education Planning & Management decentralized planning and existing system and Advisers conducted analysis by July/August 2007 planning and reporting systems recommendations for · Analysis with recommendations completed August decision-making enhancing mechanisms 2007 identified Activity plan specifying · Activity plan developed by Financial Education development process for the Planning & Management Advisers August 2007 strengthening of system and mechanisms adopted by Government All provinces and districts · Improved planning procedures developed and adopt and apply steps aligned with sector-wide planning methodology specified above for the (BCEP) strengthening of the · JC issued to guide strategic planning and education planning, management financial and results-based · JC including revised FMR and an IR aligned with reporting ENTP and sector M&E framework adopted by SRV, workshops conducted to promote and explain financing and planning arrangements · Planning Manual support tool finalized by 2008 · Training and roll out of new planning procedures underway · Support for the extension of VANPRO to district level planning and process of training and coaching underway A1.2 ENTP Strengths and weaknesses of · Assessment instruments developed August 2007 management existing system and · Assessment field trips began August 2007 capacity recommendations for enhancing management capacity identified Activity Plan specifying · National and Provincial action plans expected to development process for the be incorporated in Capacity Building Strategy strengthening of (still to be finalized) management capacity · ENTP Capacity Building Strategy includes adopted by Government detailed schedule of activity at national, provincial and district level · Local schedules confirming dates and locations of training, coaching and capacity building developed early 2009 · Capacity Building Strategy to be implemented starting 2009 A2 Monitoring A2.1 Monitoring DFA and FSQL data used to · Available annually in District FSQL Audit Reports education Education monitor and track primary (under the PEDC Project) participation, Participation and school/district progress quality and Quality student outcomes Set of ENTP indicators · M&E/EMIS advisers contracted August 2007 providing tangible outputs to · Work to develop ENTP reporting formats which be used for monitoring use FSQL indicators under way results of ENTP · Draft M&E manual completed EMIS system starts to · Preliminary inter-project and EMIS Technical 55 provide data on the progress Working Group meetings held throughout 2007 of primary schools and · Governance framework developed 2008 districts in reaching FSQL · M&E/EMIS advisers presently working with SREM and PEDC to develop strategy · Strategy and roadmap for integration of FSQL into EMIS reviewed at JAR09 FSQL tool for Lower · Lower Secondary School indicators drafted by Secondary adopted Second Lower Secondary Education Project in 2007 · TA facilitating ongoing collaboration on FSQL and EMIS also for Lower Secondary A2.2 Monitoring Assessment of Student · Data collected and consolidated for analysis Student Outcomes Achievement in Grade 5 · Report completed in 2008/2009 · Revision to improve comparability with 2001 results completed Policy recommendations · Under way: see for instance implications for new developed from Assessment 2008-2020 Education Sector Strategy and the of Student Achievement in MoET ethnic minority development programs Grade 5 A3 Promoting A3.1 Community Good practice identified; · Presently working with BCEP to identify best community involvement in strategy developed practice and develop a strategy involvement and school planning · Field survey instrument developed school decision-making, · Related field visits to be conducted accountability monitoring and accountability Community participation · No action in 2008 on community participation capacity building programs · BCEP and SREM extensions do not include scope to carry out community capacity building B. EDUCATION FINANCE B1 B1.1 National Education budget in relation · Financial statements show that education budget Strengthening budget allocation to national budget (target: amounts to 18.1%, 18.6% and 20.0% of national resource 19.1% in 2008) budgets in 2006, 2007, and 2008, respectively allocation Review of trends in non- · Trends reviewed salary expenditure share in · Non-salary expenditure increasing in per-pupil education terms B1.2 Resource Key FSQL indicators for · Financial Education Planning & Management and allocation using ENTP resource allocation of M&E/EMIS and Educational Planning and FSQL (primary) primary school level agreed Management advisers contracted July/August at central and provincial 2007 levels · Revision of ENTP planning, monitoring and reporting templates underway · Meetings to develop a set of principles for 2008 ENTP budget allocation held in August 2007 All provinces applying · New allocation formula completed: since 2008 agreed FSQL indicators for MoET routinely using FSQL as part of ENTP ENTP resource allocation at allocation process and providing provinces with primary school level allocation information B1.3 Resource Local secondary school · Preliminary meeting held in July 2007 between allocation (lower indicators developed and TBS for NTP and Lower Secondary Education secondary) piloted Projects · Lower secondary school indicators drafted by LSE Project · Linked planning, monitoring, reporting and budget allocation mechanisms not in LSE Project brief Official communication by · Still needs to be implemented SRV for the introduction of lower secondary school indicators; all provinces 56 applying lower secondary school ENTP indicators as key resource allocation mechanism B2 Enhancing B2.1 Educational Study conduction; inter- · First draft of study on Ethnic Minorities presented pro-poor Achievement of ministerial meeting to JSAR09 budgeting and Disadvantaged conducted to present, · Study currently under revision spending Children discuss and develop a · Support to new development programs for ethnic strategy for auctioning minorities in the period 2009-2015 recommendations; mid-term · Development and implementation of the final review of ENTP 2006-2010 2010 and baseline 2015 ENTP evaluation implementation instrument to be undertaken in 2010 B2.2 Household Study conduction; inter- · Studies begun and recommendations expected end Costs of ministerial meeting 2009 Education conducted to present, discuss and develop a strategy for auctioning recommendations B2.3 Full Day Strategy, action plan and · Ethnic minority community related studies will Schooling timeline developed inform the development of the strategy for full day schooling for disadvantaged children · Support from SEQAP to be coordinated B3 B3.1 Treasury Draft TABMIS · Public Financial Management Project working Strengthening Budget disseminated to relevant with MoF, State Treasury and line agencies to financial Management stakeholders develop TABMIS management Information · Currently analyzing financial management systems and System mechanisms and linkages between ministries in auditing (TABMIS) order to develop accountancy codes and program software before piloting · State Treasury Manual developed Official communication of · To be facilitated by the Public Financial instruction for introduction Management Project (PFMP) of TABMIS; Official instruction on utilization of standardized financial reporting format All provinces utilizing · PFM Project plan to begin piloting in DoFs and TABMIS; All provinces BoFs in all provinces in 2008/2009 adopt and complete unified reporting format B4 Inter- B4.1 Medium Revised MTEF agreed; plan · PFMP responsible for coordinating the ministerial Term Expenditure and timeline for the development and piloting of MTEF coordination Framework expansion of MTEF across · MoET participated in regional forum on MTEF between MoET, (MTEF) all 64 provinces developed MPI and MoF Four provinces and · PFMP leading the development and respective districts adopt implementation of MTEF related activities at and utilize MTEF central, provincial and community levels. MoET under the guidance of PFMP is involved in the design of education sector MTEF at central level. DoET, under the guidance of PFMP is involved in the design of the education sector MTEF at provincial levels 57 Annex 3: Economic and Financial Analysis Economic Analysis The PAD for the project did not undertake a traditional economic analysis of the project, with a calculated economic rate of return, and as such cannot be replicated. Instead it focused on assessing the benefits of investing in operational (non-salary) items, and more particularly in achieving FSQL, with the assumption that bringing schools, in particular in disadvantaged areas, up to this level would generate: (a) efficiency benefits through improved quality; and (b) equity benefits through reduction in the disparities in inputs and outcomes. The expectation was that improvements in FSQL, through an improved input mix, would be conducive to achievement of benefits in internal and external efficiency through improved learning outcomes, reduced dropout rates, increased completion rates and transition rates to lower secondary, improved academic performance in lower secondary as a result of improved learning outcomes in primary, and higher earnings for those who enter the labor force as a consequence of improved academic performance overall, all of these with an especially greater impact in disadvantaged areas. While the link with drop-out, completion and transition rates (as proxies for internal efficiency) were not explicitly modeled in the economic analysis, the link with learning outcomes (used more as a proxy of effectiveness and external efficiency through its link with earnings) was, using a 2001 large scale study on reading comprehension and mathematics achievement. The results provided the empirical basis for the expectation that reaching FSQL would improve student achievement and related educational and labor market outcomes, by showing that student achievement was significantly correlated with teacher capacity and school resources (which are key FSQL inputs). Using a similarly large scale 2008/2009 analysis on reading and mathematics achievement32, we were able not only to confirm but also improve on these results by showing a direct significant correlation between the FSQL Input Index and learning outcomes in both math and reading33, and a significant impact in terms of magnitude.34 In terms of magnitude, the impact of the FII comes just after school head and teacher improved pedagogical practices and the adoption of full day schooling in the category of schools' characteristics when we compare the standardized coefficients35 (family and student socio-economic background and characteristics, which are obviously controlled for, remaining as expected the strongest predictors). Although we do not have specific cost data on the different interventions (most of them being very difficult to cost), it is probably fair to say that improving pedagogical practices (in this specific case bringing teachers to provide regular feed-back to students and principals to monitor teaching through classroom observations) is the most cost-effective measure, followed by investing in FSQL and then transiting to full-day schooling (which carries great rewards but for substantial in-depth investment in terms of resources, teachers and infrastructure). If teacher and principal training included in FSQL was well targeted towards improving their capacity to provide feed-back and 32 Result of a joint study between MoET, NIES, University of Melbourne, and the World Bank. 33 While the 2001 study was in fact not directly regressing achievement on the FII but rather used teacher and school resources related variables, not always strictly included in FSQL. 34 An improvement in about 10 points of the FSQL Input Index is associated with an increase of 7.2 score points in math and 4.8 points in reading. 35 By standardizing the coefficients we are able to rank variables in terms of their relative strength of influence on the outcome within the same sample. 58 undertake effective classroom observation, the cost-effectiveness of investing in FSQL would be further increased. Table 1: Estimated effects from the three level multi-linear model for reading and mathematics (2007/08) Reading Mathematics Factor Name Standardized Metric SE Standardized Metric SE (a) (b) (c) (d) (e) (f) Province Level Intercept 263.70 18.82 144.39 23.48 Leadership School head 0.06 0.49 0.16 0.10 0.68 0.21 observation of teaching School level School socio- School location 0.03 4.90 1.98 economic background Family possession 0.07 4.20 0.98 0.12 5.83 1.64 Learning conditions Minutes to school -0.04 -0.86 0.29 Teacher feedback 0.06 24.30 4.20 0.12 38.19 5.47 Teacher support Level of excellent 0.03 5.37 1.95 0.05 6.41 2.66 teacher award Class learning tools 0.03 2.90 1.04 0.04 3.20 1.69 School Resources Fundamental school 0.04 0.48 0.15 0.08 0.72 0.25 index Student level Ethnicity 0.09 20.85 4.49 Socio-economic Family possession 0.10 4.03 0.35 0.12 3.84 0.91 status Parent education 0.11 2.38 0.18 0.13 2.45 0.33 Student Gender 0.08 19.79 1.57 characteristics Health status -0.07 -7.82 0.75 -0.09 -8.77 1.67 Number of meals per day 0.04 10.39 1.97 0.05 10.02 3.25 Sum of repeated Learning condition class -0.07 -9.66 0.74 -0.09 -9.61 1.40 Sum of learning tools 0.00 6.81 1.27 Full day schooling 0.05 11.14 2.59 0.09 14.94 4.33 Teacher support Teacher feedback 0.03 7.34 1.72 0.05 8.17 2.83 Source: NIES- University of Melbourne-World Bank-DFID(2009). Financial Analysis A rigorous financial analysis was not done either in the Project Appraisal Document, but past trends in education spending were presented as well as some longer term expenditure projections, making the main point that the program was timely to "bridge" the needs for additional non-salary expenditures with no negative sustainability implications in the long-run. The "bridging" argument was based on the fact that in spite of a doubling in real terms between 1998 and 2004, "operational" (non-salary) expenditure in primary education remained very low, but that in the future a combination of growing budgetary resources available for education and declining enrolments in primary education would allow the allocation of greater resources to 59 non-personnel inputs in the primary sector. The program would then "bridge" resources between these two time horizons. The sustainability argument was based on the above mentioned future trends and the fact, that with the exception of teacher training which needs to be refreshed, most ENTP expenditures would carry no or little annual recurrent implications, which would imply that, even while continuing growing in absolute terms (as expected), the relative weight of the ENTP recurrent expenditure would actually decrease in terms of the overall education recurrent budget. The updated expenditure review shown in Table 2 shows that both predictions were correct. Indeed, while still low in 2004, "operational" expenditure per student had grown significantly by 2008 ­ in part because of the ENTP and in part reflecting the overall increase in primary education expenditure per student. ENTP, and its focus on FSQL, acted therefore as an operational resource boost while regular education resources per student were gradually increasing. Additionally, while continuing to increase in absolute terms, the relative weight of the ENTP recurrent expenditure indeed has gradually been falling from 2006 to 2008 because of the ENTP expenditure nature and the increasing overall education recurrent budget, suggesting no negative sustainability implications and that the State Budget is gradually covering an increasing share of the maintenance of FSQL. On the other hand, while the trends in overall and per student education expenditure are positive (the expenditure review indicates very clearly that the priority given to education expenditure has been growing in Vietnam in the 1997-2008 time period according to all indicators, and that primary education expenditure has increased in real terms (by 165%) and even more notably in per-student terms (by 300%) taking advantage of the rapidly decreasing primary student population), the data also show growing inefficiencies in the system in terms of rapidly decreasing pupil-teacher ratios (see also Figure 1), accompanied by an increasing share of recurrent-salary costs (in spite of an absolute increase in non recurrent or operational expenditure per student), which the country will need to tackle urgently to improve the cost-effectiveness of its education system. TBS-EFA was not successful in addressing this issue (for which it also had few available tools). In this context, the continuation of ENTP remains important to continue supporting operational expenditures, although on a decreasing basis, at least up to when more efficient teacher deployment and a consequent decrease in the total salary burden is not achieved. Figure 1: Education Expenditure per Student (US$) and Pupil-Teacher ratio (primary) 140 120 100 80 60 40 20 0 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 Pupil-Teacher Ratio Expenditure per Student Power (Pupil-Teacher Ratio) Power (Expenditure per Student) Source: Table 2 60 Table 2: Education Expenditure Review In Billions VND 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 GDP (1) 361,500 441,646 481,295 535,762 613,443 715,307 839,211 974,266 1,144,015 1,478,695 Total Expenditure (2) 70,749 73,419 84,817 103,151 119,403 135,490 162,150 187,353 226,935 297,232 351,421 407,095 Total Recurrent Expenditure 48,825 49,998 51,551 68,045 75,084 80,566 102,521 121,238 154,978 (3) Total Salary Expenditure (4) 21,580 24,220 23,275 29,523 35,931 37,050 Total Capital Expenditure (5) 21,924 23,421 33,266 35,106 44,319 54,924 59,629 66,115 71,957 Total Education Expenditure 9,979 12,750 13,232 16,344 19,505 22,596 34,847 54,798 66,770 81,419 (6) Total Education Recurrent 7,767 9,199 9,803 11,874 14,148 16,548 22,881 25,343 32,351 40,745 51,880 59,295 Expend (7) Total Education Salary 4,308 5,544 5,883 7,707 9,231 10,317 Expend. (8) Total Education Capital 2,212 3,551 3,429 4,470 5,357 6,047 9,504 13023 13,898 Expend. (9) (6)/(1) % Ratio of Educ 3.53 3.70 4.05 4.22 4.87 5.8 5.5 Exp/GDP (6)/(2) % Ratio of Educ 14.10 17.37 15.60 15.84 16.34 16.68 18.60 19.0 20 Exp/Tot Exp (7)/(3) % Ratio of Educ 15.91 18.40 19.02 17.45 18.84 20.54 22.32 20.90 20.87 Rec/Tot Rec (9)/(5) % Ratio of Educ 10.09 15.16 10.31 12.73 12.09 11.01 14.37 Cap/Tot Cap (7)/((6) % Recurrent Educ 77.83 72.15 74.09 72.65 72.54 73.23 72.73 77.7 73.1 Share (9)/(6) % Capital Educ Share 22.17 27.85 25.91 27.35 27.46 26.76 27.27 19.5 17.0 Total Primary Education 4497 5258 6380 7143 10803 19471 22015 Expend. (10) Number of Pupils (Primary) 102500 9751000 8841000 8350000 6894351 6753648 (11) 00 Number of Teachers 336792 347833 362627 347000 344763 (Primary) (12) (10)/(6) % Ratio of Pri 35.27 32.17 32.71 31.61 31.00 29.60 27.00 Educ/Tot Educ (10)/(11) Primary Exp per 439 539 808 3260 Student (in 000 VND) (11)/(12) Pupil-Teacher Ratio 30.43 28.03 23.03 19.87 19.59 Total Primary Education Exp. 4,608 5,258 6,380 6,896 9,381 13,417 12,234 (in 2000 VND) (13) (13)/(11) Primary Exp per 450 539 780 1811 Student (in 2000 000 VND) Primary Non salary 66 108 136 ("operational") Exp per Student (in 2003 000 VND) 61 ENTP (State Budget 2,970 3,380 3,480 Funding) (14) (14)/(7) % of Total Education 5.7 5.2 4.2 Expenditure ENTP (State Budget 2,258 2,184 2,190 Funding) Recurrent (15) (15)/(7) % of Total Recurrent 5.5 4.2 3.6 Education Expenditure Source: Vietnam Public Expenditure Review (2005) up to 2002; MoF data and education sector projections after 2002; IMF World Economic Outlook Database; MoET-DPF; Italic: estimates 62 Annex 4. Bank Lending and Implementation Support/Supervision Processes (a) Task Team members Responsibility/ Names Title Unit Specialty Lending Luis Benveniste Sr. Education Specialist EASHD Task Team Leader Binh Than Vu Senior Education Specialist EASHD Carol Ball Senior Operations Officer EASHD Mai Thi Thanh Senior Education Specialist EASHD Stefanie Stallmeister Education Specialist EASHD Ahsan Ali Senior Procurement Specialist EAPCO Irina Luca Senior Procurement Specialist EAPCO Joel Turkewitz Senior Procurement Specialist OPCPR Jennifer Thomson Senior FM Specialist EAPCO Hung Viet Le FM Specialist EAPCO Thong Quang Tran FM Specialist EAPCO Hoi-Chan Nguyen Senior Counsel LEGEA Xiomara Morel Senior Finance Officer LOAG1 David Webber Lead Finance Officer LOAG1 Chiyo Kanda Peer Reviewer OPCIL Chingboon Lee Peer Reviewer LCSHD Michel Welmond Peer Reviewer MNSHD Linh Van Nguyen Team Assistant EASHD Chandra Chakravarthi Program Assistant EASHD Van Minh Nguyen Senior Public Finance Specialist EASPR Behdad M. H. Nowroozi Sr Financial Management Specialist EAPFM Financial Management Hung Kim Phung Senior Finance Officer CTRFC Christopher Shaw Lead Education Specialist EASHD Grayson Clark Economist/Consultant James Cobbe Economist/Consultant Ekta Kothari Economist/Consultant Peter Moock Economist/Consultant Non World Bank staff Agency Alfred Fickers Royal Embassy of Belgium Carlos Lietar Royal Embassy of Belgium Ngo Thu Huong Royal Embassy of Belgium Peter Van Acker Royal Embassy of Belgium Canadian International Development Brian Allemekinders Agency (CIDA) Camille Baudot CIDA Nora Fyles CIDA Marvin Lamoreux CIDA Le Anh Thao CIDA Blanka Pelz CIDA Department for International Development, Nguyen Huu Hieu UK (DFID) Steve Passingham DFID 63 Vo Thanh Son DFID Corinne Boulet European Commission Jos Jonkers European Commission Anne Claire Leon European Commission Nguyen Thi Lam Giang European Commission Anouk Van Neck European Commission New Zealand Agency for International Michael Ward Development Supervision/ICR Emanuela Di Gropello Senior Human Development Economist EASHD Task Team Leader Sylvester Kofi Awanyo Senior Procurement Specialist EAPPR Carol Chen Ball Senior Operations Officer EASHE Hung Viet Le Sr Financial Management Specialist EAPCO Quynh Xuan Thi Phan Financial Management Specialist EAPFM Jennifer K. Thomson Sr Financial Management Specialist OPCFM An Thi My Tran Education Specialist EASHE Binh Thanh Vu Sr Education Specialist EASHE Jeffrey Waite Lead Education Specialist MNSHE Mai Thi Thanh Sr. Education Specialist EASHD Cung Van Pham Sr. Financial Management Specialist EAPCO Rosario Aristorenas Program Assistant EASHD Nguyet Minh Nguyen Team Assistant EACVF Cesar Palma Banzon Extended Term Temporary EASHD Suzana Campos Augusto Consultant EASHD Non World Bank Staff Agency Bridget Crumpton DFID Brian Allemekinders CIDA Le Anh Thao CIDA Anouk Van Neck European Commission Carlos Lietar Belgium Carlton Aslett Belgium Ngo Thu Huong Belgium John Durston New Zealand Agency for International Development Myra Harrison New Zealand Agency for International Development Maggie Lowe New Zealand Agency for International Development 64 (b) Staff Time and Cost Staff Time and Cost (Bank Budget Only) Stage of Project Cycle USD Thousands (including No. of staff weeks travel and consultant costs) Lending FY04 30 113.04 FY05 65 244.59 FY06 0.57 FY07 0.00 FY08 0.00 Total: 95 357.63 Supervision/ICR FY04 0.00 FY05 0.00 FY06 35 88.98 FY07 26 66.42 FY08 17 64.26 FY09 27 103.08 Total: 105 322.74 65 Annex 5: Beneficiary Survey Results [Not Applicable] 66 Annex 6. Stakeholder Workshop Report and Results [Not Applicable] 67 Annex 7. Summary of Borrower's ICR36 1. Analysis of TBS-EFA Objectives, Design and Implementation Overall the TBS-EFA is widely considered to be successful in its contribution to achieving the E-NTP and EFA objectives set by the SRV. Achievements are supported generally by progressively positive statistics in the primary education sub-sector and there is also evidence of a flow-on effect in increased enrolment in both lower secondary and upper secondary education. The design of multi-donor assistance for targeted budget support was the first of its kind for education in Vietnam. TBS can therefore be viewed as an initiative that is successful despite an honest recognition of deficiencies. These include some minor limitations in its design, some underestimation of complexities involved, some unevenness in implementation, and some inadequacies in management. These successes and shortcomings have been reported and, to an extent, analysed elsewhere (for example: JAR and JSAR missions, Aide Memoire, audit reports) and MoET will not detail them here. Lessons have been learned and knowledge and capability enhanced that will strengthen both ongoing E- NTP objectives and any future TBS programs. TBS-EFA as a whole and specific assistance from the World Bank are comparatively small even though vitally important part of the total budget for education in Vietnam. In the implementation of the E-NTP, it is not possible to identify specifically in terms of budget inputs, program activities, or results, the separate contributions from individual donors or from TBS-EFA. This was not intended and was not a design element. However some particular activities supporting E-NTP objectives, such as training in the provinces and the development of procedures and manuals may be easily identified with TBS-EFA. It is valid to observe, as noted by the October 2009 Evaluation Mission funded by the EC, that in decentralized implementation of most E-NTP activities, there has been little or no identity given to TBS-EFA and donor support. TBS-EFA was designed to contribute towards achieving EFA objectives by providing targeted budgetary support through the E-NTP disbursement channel. In response to Government request, priority was given to the improvement of educational outcomes in basic education. More specifically, the program supported progress by the SRV towards the improvement of educational outcomes through a package of investments with two design components: to enhance the quality of education in classrooms and to strengthen the administration of the education system. During TBS-EFA implementation which covers all provinces, the share of schools achieving Fundamental School Quality Levels (FSQL) has increased. As schools below FSQL often serve poorer communities, the beneficiary group contains a higher proportion of poor children and ethnic minorities than elsewhere in the nation. The program was designed and implemented nationwide with a focus on five of the seven existing education NTP sub-components: · Sub-component I: Illiteracy elimination, consolidation of Universal Primary Education (UPE) and achievement of Universal Lower Secondary Education. · Sub-component 2: Renovation of the curriculum and textbooks/teaching methods. · Sub-component 4: Upgrading infrastructure of Teacher Training Institutions and teacher qualifications. 36 Summary of the Borrower Completion Report prepared by the MoET-Department of Planning and Finance. 68 · Sub-component 5: Support for the education of ethnic minorities and disadvantaged regions. · Sub-component 6: Infrastructure development. The design of TBS-EFA consists of two major components as follows: · Component 1 was designed to support the implementation of FSQL standards and improved educational outcomes by support for E-NTP objectives and specifically the five sub-components listed above. · Component 2 was designed to strengthen institutional capacity to administer and implement the E-NTP. A common observation based on TBS-EFA experience to date and recognizing some implementation challenges relates to the complexity of the program linked to available management capacity to guide its implementation. The current E-NTP with TBS support is being implemented in a period following extensive decentralization to sub-national levels of many public administration functions. Some procedures are new and changing and all of the staff to transact them are not yet fully trained and embedded. Provincial and district units responsible for TBS-EFA implementation and reporting do not have direct links to central entities. While a central ministry such as MoET has certain oversight of policy and quality issues, it has no authority over management responsibility in the provinces. A key example is the allocation, disbursement, and management of the education budget which is primarily a matter for the People's Committee in the provinces. In summary the current operating situation is more complex than donors, SRV agencies, and the Project Appraisal Document could have fully envisaged in 2005. Reliable evidence shows that educational outcomes in primary education have generally improved in Vietnam during the lifespan of TBS-EFA. The latest figures confirm positive trends: the net enrolment rates for primary education are trending higher as are Grade 5 learning outcomes in Vietnamese. There appears to be a slight increase in repetition in Year 2 to Year 3. FSQL is progressing but national targets may not be reached on time. These assessments reported to stakeholders at the JAR in October 2009 relate to E-NTP and total system factors. However, they highlight the emphasis that TBS is placing on the need for improved M&E in educational and management operations in implementing Component 2. MoET appreciates that continuing efforts to strengthen management capacity at central and other levels is a prerequisite to achieving current and improved E-NTP objectives. This is central to Component 2 of TBS-EFA and one of a number of lessons learned to which operational adjustments have been made. Training and human resource development initially in the key areas of financial management, audit and procurement has been a focus of workshop activities by the former BCEP team that was integrated with TBS during 2009. A different training methodology is being followed that aims to improve educational administrators in provincial and district units. It is also intended to utilize and enhance provincial institutions and so carry forward management-training activities after the completion of TBS-EFA. This focused training should assist in dealing with issues such as delays leading to under-spending of budgets within a given financial year, and slow or incomplete reporting of E-NTP finances. In both design and operational terms, it seems with the benefit of experience that Component 2 of TBS-EFA should have been begun before Component 1 or at the least concurrently with it. Building capacity earlier in the program through the style of activities in Component 2 could arguably have reduced some of the problems encountered in implementing Component 1. 69 During 2009 implementation of key elements in the approved TBS-EFA Workplan and Budget has been somewhat restricted by difficulties and delays experienced in the recruitment of national and international consultants and in management of contracts to engage these experts. The need for action has been noted in the Aide Memoire produced after JSAR and JAR missions. MoET has sought to recruit international experts through the two-stage process of calling for expressions of interest and then selecting companies to bid competitively through the preparation of proposals to supply the consultants. SRV regulations, aimed at ensuring probity, require formal procedures and sign-off at high levels. Given the multiple commitments of senior government officers, such formalities had compounded the process. It is now expected that the hiring of all national consultants will be finalized in the final quarter of 2009 and international consultants will be contracted early for service early in 2010. This issue could be reviewed in the design of any future TBS type programs. Among models for consideration are early specific management support to a central ministry or establishment of a small Project Management Unit (PMU), to allow prompt support and execution of staffing and other critical TBS matters. One significant matter affecting E-NTP ­ and not one for which either the problem or its solution lies exclusively with TBS-EFA or any single project ­ is that of a modern and efficient system data collection, management and utilization. A continuing need is for an integrated approach to an EMIS that will serve the needs of E-NTP and of central and sub- national education entities. Positive steps were taken by MoET through its work with specialized projects and concerned donors that led to the promulgation in August 2009 of an EMIS Roadmap. This Roadmap sets out a pathway for the strengthening of existing systems and the development of a consistent EMIS that can serve, among other needs, enhanced policy development and financial management. Since August 2009, a Ministerial EMIS Committee has been established and the initial Roadmap revised. Several projects including TBS are collaborating to support this high priority task. As a possible lesson learned, some observers would contend attention to EMIS should have been an earlier priority because of its importance not only within MoET and its linked units like DoET and BoET, but also in relation to concerned ministries such as State Treasury and MoF. 2. Assessment of World Bank's Performance Interactions between the World Bank and the MoET and other SRV ministries and agencies concerned with the loan have been uniformly balanced and productive and could be rated as positive overall. The World Bank's PAD for TBS-EFA is a primary document and one which has been used by other donor partners. This PAD is a foundation underpinning the MOU signed in 2005. Staff of the WB have been consistent supporters, useful advisers and, on occasion, relevant critics in helping TBS to maintain its momentum and sense of direction in strengthening EFA. TBS-I and TBS-II are an integral and significant part of the E-NTP supporting Vietnam's progress towards achieving EFA and the MDG. Contributions by the WB and other development partners to TBS funds become indistinguishable in implementation of the E- NTP. Analysis of the contribution made by the World Bank and other partners to the five E- NTP Sub-components directly assisted by TBS is measured by the more comprehensive record of national progress towards EFA in other sections of this report. Although progress in the short term may sometimes be seen as uneven in certain specifics, WB assistance has been positive and has helped the SRV through MoET to focus efforts on EFA and to achieve results that are enviable among comparable nations. 70 Difficulties experienced by MoET in meeting all requirements envisaged in the MOU relate to a range of factors including, as some key examples: at national level, a complex institutional framework involving inter-ministerial issues not always subject to the policy direction or management of a single ministry; concurrent with the implementation of TBS and E-NTP a relatively successful decentralization to sub-national levels of governance, financial allocations and sector management in the setting of locally determined priorities that highlights issues in central/provincial relationships; an underestimated lack of capacity to monitor and report educational and financial results linked to E-NTP among other programs. The flexible and responsive approach of WB and associated donors has assisted MoET and TBS to identify these and other weaknesses and to plan and work towards capacity building to meet identified challenges. Vietnamese society is extremely dynamic and its educational indicators are changing quickly. Both are positive developments. Some of the difficulties that MoET has faced in managing TBS come, on the one hand, from this rapid rate of change in many areas, matched on the other hand by continuing levels of systemic rigidity as complex organizations like ministries seek to adapt. It would be fair to observe that these two factors may not have been fully appreciated by the WB and other donors when, in 2005, the PAD and the MOU were finalized. Initially these incomplete understandings about MoET and underestimating implementation in a decentralized context may have resulted, at the operational level, in unrealistic expectations about TBS, and its capacities to manage or influence activities at the provincial, district and other levels. The twice-yearly TBS-EFA review missions known as Joint Annual Review and Joint Semi- Annual Review involved the four concerned SRV ministries and the seven donor supporters. Participants receive reports on E-NTP and TBS activities and results and the World Bank continues to be a leader in these JAR/JSAR in recognizing both changed conditions for implementation as well as TBS achievements and shortcomings. At the conclusion of each review mission, an Aide Memoire is drafted by donors and signed off by MoET. It is arguable that this action together with the issuance of the Joint Circular No. 125 of 2008 (Governing the Management and Utilization of E-NTP Funds till 2010) is, at the least, in parallel with and possibly qualifies parts of the MOU of 2005. Any perceived change in focus or activities that might vary from the MOU is probably in accord with subsequently agreed and/or legislated developments affecting TBS that the SRV and donors have assented to. The World Bank has also shown flexibility in this regard. At the same time, the World Bank with other donors has been properly concerned to improve the quality of procurement and audit procedures and of underlying financial management practices. This has resulted in a rigorous focus on needed capacity building for financial management especially at provincial level, and the development of new procedures and manuals, progressively leading to strengthened practice and improvements in reporting. 3. Assessment of MoET's Performance Many of the points that can be made in this section have been referred to or implied in other parts of this report; and to restate them all should be unnecessary. MoET has always provided the highest level of support to E-NTP and TBS-EFA within its capability. Having regard for the complexity of this form of development assistance, its newness to education in Vietnam as a support modality involving donors, other ministries and projects, and the different management and reporting requirements, overall the TBS undertaking has been both successful and it has provided a number of learning experiences. The lessons learned are not only for MoET to note, but are likely to have also been of benefit to other concerned ministries and to associated sub-national entities in the provinces and districts. 71 Progressively the management and administration of E-NTP and TBS have achieved results in line with objectives for EFA when compared with previous periods, either annually or in five-year plans; and the general educational outcomes have been referred to elsewhere in this report. Specifically E-NTP 2008 was implemented in a timely fashion which provided a basis for ongoing progress in implementation of objectives, and in meeting targets for the total period 2006-2010. The E-NTP capital source has been well utilized in the planned ways and this will continue to make contributions to stabilize and develop EFA goals. There has been high level SRV support to MoET in its implementation activities underpinning E-NTP and TBS-EFA and this is seen in the following examples of legislation and regulations. E-NTP is being implemented on the basis of Decision No. 42/2002/Q -TTg dated 19/3/2002 and Decision 07/2008/Q -TTg dated 10/01/2008 of the Prime Minister and the Joint Circular No. 125/2008/TTLT-BTC-BGD& T dated 22/12/2008 of Ministry of Finance (MoF) and Ministry of Education and Training. Decision No. 42/2002/Q -TTg (of the Prime Minister dated 19/3/2002) shows the following good points, broader than education but in part a development associated with E-NTP (and implicitly TBS-EFA): (i) Provinces are to be active in budget allocation in order to execute projects in provincial areas; (ii) On the basis of actual socio-economic conditions in a region, each locality is to build a mechanism to mobilize more resources to implement each program/project; and (iii) In planning and implementing a number of programs or projects, localities can become active in integrating targeted programs within local areas. MoET in collaboration with other ministries and provincial and district units has steadily and consistently provided support and encouragement for new and improved planning and financial management practices. Some details of the complex planning and budget process are given to illustrate how MoET has changed and expanded through E-NTP and TBS-EFA activity in the period of decentralization. Based on requirements for implementing tasks in E- NTP, MoET develops an annual spending forecast for E-NTP which it sends to the Ministry of Planning and Investment and MoF. After the national budget is determined, MoF issues a budget allocation decision, details of which are as follows: (i) to MoET, to be applied to units directly under the control of MoET; (ii) to other ministries, to be applied to those which control training institutions with functions within E-NTP; to each province, a total spending forecast for all NTP including E-NTP. The allocation and integration of E-NTP is an area of responsibility of Provincial People's Committees. In operational fact, the roles of DoETs in different provinces may vary in terms of responsibility for the planning, building, expenditure allocation, and managing of E-NTP implementation activities; and each adds to management complexity in MoET's relationships. There are three commonly used models for the differences in responsibilities exercised by a DoET: (i) The Provincial People's Committee assigns the provincial Department of Planning and Investment leadership in the E-NTP budget allocation and the DoET may be assigned to directly manage the spending of Project Consolidation of universal primary education, achievement of universal lower secondary and support to the universal upper secondary education; (ii) the DoET is assigned to cooperate with provincial DPIs to build the E-NTP budget allocation forecast for each project, and then to submit it to Provincial People's Committee for approval; and (iii) the DoET is assigned by the Provincial People's Committee to lead in the E-NTP budget allocation. There are weaknesses, challenges, and opportunities for further improvement associated with E-NTP and the support TBS has given. Many of these have been documented in reports to donors and stakeholders. The mechanism for E-NTP management in the provinces is not entirely clear, there is no standard pattern or general template to guide operations, and so make more effective interactions with central ministries. Although Joint Circular No.125 of 2008 specifies "to assign to MoET leadership in cooperation...", many provinces have not 72 been assigned this responsibility. In some cases a DoET may not be active while in others the DoET does not seek to undertake the task. There is also need to improve the activity of internal inspection of units responsible for implementing NTPs in the provinces. Financial reporting and implementation appraisal has not received enough attention, and therefore report quality is not high, and does not strictly follow regulations. The situation is improving and the use of manuals plus planned training activities will strengthen personnel and procedures. Not all provinces are following new regulations and procedures in the same way. This is reflected in the variation in reporting to the E-NTP administrator. Consequently, this unevenness creates difficulties in synthesizing and assessing E-NTP implementation at the sub-sector, the sector, and national levels; and the deficiency is seen at the central point of reporting in MoET but does not originate there. It is highlighted most in financial and audit reports. There is a need to learn from experience and to include standard but flexible templates to be used in building up report information in a generally consistent format. Omissions and any mistakes can be more easily detected. In part, this relates to the need for an integrated EMIS that includes personnel and finance data. During 2009, TBS-EFA has been working on template design to address this need. Because of the separation between the planning and setting of policy objectives for a national program like E-NTP and TBS-EFA and the implementation with the budget disbursement and management in the provinces, program administration at the centre may appear to be less effective than it is. Another weakness is linked to the allocation of budget funds at a gross level nationally with prioritization and funding determined at the provincial level. Sometimes the involvement of several levels of government tends to delay E-NTP funding allocations and implementation, causing difficulties in attaining objectives in set time frames. Within a province, the selection of E-NTP funded projects or activities may be scattered and unfocussed leading sometimes to half completed construction works and other tasks that fail to meet a planned deadline. The capacity of MoET or another central ministry to manage and influence these issues is limited and in the design of another TBS this could be taken into account. During recent years, E-NTP implementation with TBS-EFA support has achieved positive results, completed some remarkable developments in facilities and conditions, and supported pedagogical and curriculum improvements, all of which have enhanced the quality of education in Vietnam. During the implementation of TBS-EFA, MoET has identified the following areas for improvement as lessons learned: (i) attempt to have MPI and MoF annually assign the gross budget forecast and detailed expenditure according to each project in E-NTP for each locality; (ii) MoET to annually assign tasks with detailed performance indicators to be completed by each province in each year; (iii) as provided by regulation, assign to each DoET the key role in allocating E-NTP expenditure of each province; (iv) ensure compliance with current regulations by each province, particularly in performing periodic reporting; (v) for the provinces not following regulations in reporting and other important matters, consider termination of their E-NTP allocation; (vi) assist and support improved reporting through the development and training in the use of templates, particularly relating to Joint Circular 125 of 2008; (vii) provide needs based training in M&E in both general and specific areas of educational management; and (viii) undertake a sample study of effective E-NTP management in different regions to create a flexible basis for training and socialization nationwide, and reach the higher efficiency in E-NTP implementation. 4. Sustainability of TBS-EFA Objectives 73 Primary evidence for sustainability until 2010 may be found in three major areas that relate to E-NTP and also TBS activities: Firstly, the overall improvement in key primary education indicators (not exclusively linked to TBS ­EFA) including rates for participation, drop-out, repetition, etc, curriculum revisions linked to the widespread provision of textbooks particularly to students of poorer families, and improvements to classrooms, other school infrastructure and teaching training ­ followed by increasing enrolments in secondary education ­ overall, a growing demand for education; secondly, the progressive increase in the proportion of state funds allocated to the education sector ­ generally calculated around 20% ­ overall, a dedicated policy commitment to education from the national budget; and thirdly, the endorsement by JSAR held in April 2009 of a detailed and costed Workplan and Budget for TBS-EFA until closure at 31 December 2010 and its acceptance by MoET. A copy of the Workplan and Budget is attached as Annex No. 1 to the original Borrower's Report. Further evidence indicative of sustainability may be seen in the following documents and actions fostered by MoET through TBS. The inter-ministerial Joint Circular No 125 of 2008 which was developed over a period of several years through the efforts of the concerned ministries and donors including a high level involvement by the World Bank. It is regarded a significant joint achievement in giving agreed policy, regulatory, and management direction and/or refinement to TBS and E-NTP operations. It is evident that the effects of this Joint Circular and of other support for improved financial management cannot be seen in the 2008 audit and financial reports. Preparations for an update and revision of this Joint Circular applicable to the next E-NTP period, 2011- 2015, have already begun with initial consideration at a series of national TBS workshops held in October 2009. It can be noted that just as the E-NTP covers all levels of education from pre-school to higher education, the Joint Circular also covers all levels of education. Arguably, support for TBS has broadened to include a more integrated perspective from primary through to lower secondary and even upper secondary education. This flexibility recognizes the pipeline enrolment effect flowing upwards from focused efforts to achieve universal primary education and the need to strengthen lower and upper secondary education. To add resources to needed capacity building, particularly in the decentralized context, the former BCEP with its successful training approach and methodology was integrated with TBS from October 2009. A comprehensive yet flexible training schedule for the period from 1 October 2009 to 31 December 2010 was presented to JAR in October 2009 and endorsed. Already two series of national workshops dealing with key issues such as planning, joint circular revision, audit requirements have been held during October and November. A series on improved procurement procedures is to be implemented in December 2009. These contributions to sustaining new and reformed practices and procedures in education management will be further strengthened with the technical assistance and associated training linked to inputs by national and international consultants. 5. Ongoing E-NTP Implementation and Forward Planning As part of MoET's commitment to support sustainability for achievements to date, there is an ongoing process of reviewing the current E-NTP and of discussing and formulating the next E-NTP for the period 2011-2015. Initial planning for the next E-NTP has also been the subject of two national workshops during 2009. In reviewing some of the difficulties and obstacles encountered during implementation, respondents have initially reported that because of ever increasing demand for education, E- NTP 2006-2010 has not always met provincial requirements, for instance, in such areas as: widely available kindergarten education for 5 year old children; the quality of foreign 74 language teaching; and provision of education of talented and gifted students. Other observations include: the financial management capacity of investors in education who are beneficiaries of NTP is still low; M&E for E-NTP overall has not yet received sufficient attention, and specifically the M&E capacity of education managers is still limited and requires uppgrading; planning for E-NTP budget utilisation in the provinces faces the difficulty of a lack of capacity for accurate budget allocation for the medium term; some provinces report a loosening of direct management control as a result of decentralisation to BoETs, communes and schools; actual allocations of funds are sometimes much less than proposed budgets (as much as 30% less). Provincial participants at 2009 national workshops and other consultations were asked to assess lessons learned from the policy and practice of implementing E-NTP 2006-2010 and to suggest initial guidelines for improvement. Their responses included the following observations: too many units lead in managing E-NTP causing difficulties in the education sector, especially with M&E functions and in bridging E-NTP to match provincial objectives; there should be a more uniform management responsibility for all provinces; many confirmed DoETs are best situated to exercise management over E-NTP including planning, resource allocation, M&E, and report coordination. Other suggested changes were that mechanisms should allow funds transfer between E-NTP sub-components particularly where some were under-spending and others needed additional funds. Some suggested this was a possible resource allocation mis-match that came from unrealistic planning as well as from difficulty in responding to rapid changes in the external environment, with results that affected E-NTP objectives; and that perhaps a resource allocation cycle of five years would assist localities to meet their objectives. Funding for M&E activities should be provided in budgets as well as expenditure norms for implementation; and if an education sector entity lacked M&E performance capacity, it should be possible to hire external consultants based on a separate budget. To allow completion of a specific capital investment to meet planned provincial objectives, it was suggested that E-NTP resources might be focused, within an annual cycle, on one sub- component. Provincial participants in the E-NTP review and planning process gave the following unranked responses as a random assessment of ongoing needs and suggested new activities: current E-NTP sub-components to be maintained but at different levels according to the situation in different provinces; the upgrading for teachers sub-component to include upgrading for managers with training and qualifications to a standard level; a sub-component of universal secondary education is needed to include as content drop-out prevention; the existing sub-component for education of ethnic minorities and disadvantaged regions needs more regions to be added including islands and areas adjacent to islands; and the sub- component for building boarding schools (pursuant to Circular 44 of MOF and Joint Circular 125) need to be changed to allow new construction as well as extensions and renovations; and continue to maintain the sub-component for strengthening vocational training capacity. In addition, to meet development requirements of the education sector in the next period, participants suggested the following additional sub-components may also be needed: support for universal kindergarten education for 5 year olds; establishment of provincial gifted schools to foster talent; integration of education for disabled children; training and upgrading teachers in foreign languages; strengthening vocational training capacity in professional secondary education. 75 To accomplish even some of these suggested needs and objectives, respondents identified requirements and conditions that would advance education under E-NTP 2011-2015. The following suggestions built on existing procedures and an evaluation of their usefulness in implementing the current E-NTP: a Joint Circular to guide NTP implementation and administration; improved management capacity to include planning, financial management and M&E for staff in charge in the provinces; planning based on a output orientation and the VANPRO model introduced by the former BCEP; overcome slow disbursement rate by strengthening financial management capacity of both managers and beneficiaries and link to performance based M&E; raise importance of M&E by making procedures and performance requirements clear at the provincial level and specifically at DoET and BoET levels to help localities in fully implementing E-NTP; scale up utilizing common or interactive software to integrate projects and programs within education sector. In the last quarter of 2009 MoET made additional commitments to sustainability by progressive planning procedures to build a new E-NTP for the period 2011-2015. This action is being taken to contribute to implementation of Resolution No. 35/2009/QH12 of National Assembly Meeting Session XII dated 19/6/2009. This concerns a review and formulation of policies and orientations to strengthen education and training mechanisms for the period 2011-2015 and the overall education and training strategy 2010-2020. 6. Financial Statements for 2008 Manuals for financial management and for procurement have been prepared and as they are used, these manuals will be revised and adapted to meet new and changed regulations. Training has been undertaken in these key areas, including some not directly sponsored by TBS. For example, further training activities are scheduled particularly in relation to focused inputs linked to the TBS Workplan that relate to the work of international consultants and specialist national consultants. Understandably, the effect of this training will not become evident until the audit and financial reports for 2009 are completed. It is expected that noticeable improvements in financial management, procurement, and auditing will be apparent in 2009 and 2010 with associated positive reports. A copy of the E-NTP 2008 Financial Report in Excel format is attached as Annex No 2 to the original Borrower's Report. This report was also presented to the JAR mission in October 2009. 7. Audit Status of TBS-EFA In general, capacity for M&E in all areas of education and of management in Vietnam is not yet as well developed as needed for a Ministry engaged on modernization. This deficiency is acknowledged by inclusion in the TBS-EFA Workplan of significant activities to strengthen M&E capacity. Until 2009, a lack of capacity may be seen in the area of financial management and reporting at DoET and BoET levels when training in new procedures and requirements has not always kept pace with the decentralization of public administration functions. For instance, until recently E-NTP financial reports from provincial authorities were submitted late or not at all. However, for the JAR in October 2009 there was a marked improvement with 50 of 63 provinces submitting reports on time compared with 29 received in April 2009. 76 The division of responsibilities for E-NTP financial management and reporting among concerned entities at the sub-national level such as MoET, State Treasury, and MoF is progressively becoming clearer. Improvement in operations follow increased understanding from the issuance of guidelines such as Joint Circular 125 of December 2008, with its specification of duties, accounts, and financial management procedures, and, during 2009, from TBS training workshops in new financial management practices, audit requirements, and procurement procedures. It would seem that some of the financial management risks and the assessment of M&E capacity that accompanied rapid decentralization might have been underestimated when the TBS-EFA was planned. But these factors are intrinsic to the impressive rate of change and the development surge that characterizes progress in Vietnam and its education system in the past decade. MoET is increasingly confident that evidence of strengthened capacity and improved performance will be seen in financial reports and audits from 2009 onwards. It is understood that although these sub-national units are not managed by MoET, the focus of actual capacity in financial management for E-NTP, which is attributed to MoET, is revealed in the audit of provincial financial and procurement reports. In previous years, MoET acknowledges that there have been some particular problems in achieving timely and complete E-NTP audit reports. This had resulted in the auditors issuing qualified reports, for example, in 2007 but based on visits to 13 provinces/cities, a sample stated by the auditors as "just too small." For the 2007 audit report, it can be noted that 98% of funds was reportedly used correctly. However, the auditors recorded, among other shortcomings, that there were delays and inaccuracies in reporting and in disbursements particularly by the State Treasury. In 2009, three E-NTP audit reports for 2008 prepared by Deloitte Vietnam were received by MoET and were also presented to concerned ministries and donors at the JAR held in October 2009. The auditors' initial report of "adverse" was based on visits to 16 provinces/cities, later extended by 3 more after discovering "mis- statements." Without pursuing the detail, MoET notes that "ineligible disbursements" had declined in 2008 from 2% in 2007 to "approximately 1.12% of the total sample and approximately 0.12% of expenditures" in the sample 19 provinces/cities. In addition to critical findings, the auditors made a number of positive statements about improved financial management and MoET considers improvements will continue. Since receiving the initial adverse report from the auditors, a series of four meetings has been held involving variously the DPF of MoET, State Treasury and the appointed auditors, Deloitte Vietnam. The principal issue of concern causing the currently adverse audit report seems to focus on the reporting mode rather than on financial management or accounting practice. State Treasury has agreed to send a letter to State Treasury offices in all provinces requesting that the format of each financial report be revised and resubmitted. There is an implication that the audit and report may have been based on less than complete information. The auditors, Deloitte Vietnam concur with the action being taken and have said that if necessary they would visit the provinces again to check information. The situation seems to typify a system and procedures that are in transition with some elements of inadequacy including issues relating to communication about the preferred documentation and format. Subsequent developments include the issuance by Deloitte Vietnam of a draft final audit report that has been revised to become a "qualified opinion" for E-NTP financial reports for 2008. MoET has to review the auditor's written reports in consultation, if necessary, with linked ministries before giving a response to Deloitte. So the final audit report will at least be "qualified" or, after discussions, may be upgraded further. Another recent positive 77 development concerns capacity building for decentralized functions not only in MoET and its related provincial entities. In early December 2009, State Treasury is collaborating with TBS in organizing a series of training workshops to give guidance and further strengthen implementation of management and payment procedures in E-NTP. This form of training cooperation on a topic that has caused comment from the auditors will build capacity for sustainable decentralization of financial management and reporting. 78 Annex 8. Joint Partners External Evaluation37 Executive Summary 1.1 Introduction Vietnam's poverty reduction and economic growth over the last 15 years have been described as: "one of the most spectacular success stories in economic development". Due to carefully implemented economic reforms and consistent economic management, the annual real growth rate of Gross Domestic Product has averaged around 7.5% and the rate of poverty has fallen from around 70% to below 20%, with widespread improvements in the lives of Vietnamese citizens and with predictions that the poverty rate could fall as low as 10% within the next two or three years. Vietnam continues to make impressive gains in access to education at all levels, with net enrolments rates reaching 94.6% for primary, 90.1% for lower secondary, and 63% for upper secondary (data for school year ended in 2004), and with primary school completion rates now exceeding 90%. However, significant challenges remain, including that barriers to education are concentrated in specific poorer regions of Vietnam, where schooling conditions are significantly harder due to lack of adequate human resources, prevalence of vernacular languages, high poverty incidence, lack of adequate financial resources, low resource mobilization ability at the local level and the existence of smaller remote communities in mountainous areas. In Vietnam, the primary and secondary education system is characterized by a dual, vertically and geographically divided structure. The overall direction and control are in the hands of the People's Committees, while the daily management, administration and delivery of instruction are in the hands of the Ministry of Education and Training. The Ministry of Finance has overall responsibility for recurrent expenditure, while responsibility for capital expenditure lies with the Ministry of Planning and Investment. The country has achieved Universalisation of Primary Education during 2003/ 2004. It is planned that Universalisation of Lower Secondary Education will be achieved during 2010. 1.2 The SPSP TBS-EFA Programme The SPSP programme of TBS EFA is a multi-donor programme co-financed by the European Commission and by Belgium, Canada, Spain, New Zealand, the United Kingdom, and International Development Association (IDA), together known as the TBS partners. The programme's development objective is to assist the Government of Vietnam in the implementation of its National Education for All Plan through sector budget support for selected sub-components of the E-NTP designed to enhance the quality of basic education, and through 37 Final Evaluation of Sector Policy Support Program in Education (Targeted Budget Support for Education For All) (2009). 79 strengthening of the administration of the E-NTP. During the life of the programme, the share of schools achieving Fundamental School Quality Levels, a minimum standard of school quality, will increase. In addition, the programme will also promote an overarching policy framework in education finance, planning and governance that will act as an enabling environment for improved service delivery. There are two programme components. Component 1 strives to accelerate implementation of FSQL standards by channelling sector budget support (SBS) to five priority spending areas in the E-NTP. TBS EFA priority spending areas in the E-NTP are: a) Illiteracy elimination, consolidation of Universal Primary Education and achievement of Universal Lower Secondary Education; b) Renovation of the curriculum and textbooks/teaching methods; c) Upgrading infrastructure of Teacher Training Institutions and teacher qualifications; d) Support for the education of ethnic minorities and disadvantaged regions; and e) Infrastructure development. The allocation of resources is based on an assessment of the financing gap needed to improve educational outcomes and satisfy minimum standards of access and quality in disadvantaged schools and districts, as defined by the FSQL regulation. The FSQL standards establish minimum service delivery criteria in terms of physical infrastructure, instructional materials, teaching staff qualifications, in-service professional development, school organisation and management and school-community linkages. It includes a framework to enhance teacher capacity, school leadership and community participation while promoting inclusive education for traditionally disenfranchised communities. Relevance: The E-NTP is relevant and consistent with the latest version of the Vietnam Education Development Strategic Plan for 2008-2020 and primary elements of its National Targeted Programme. The programme was designed under the previous ESDP 2001- 2010. The programme was designed38 to: (i) support the Government's strategic approach to poverty reduction based on evidence-based needs analysis, (ii) aim to channel ODA more efficiently and reduce transaction costs through the utilization of a sector budget support aid delivery mechanism, and (iii) be identified as a transition from traditional projects to full budgetary support. Effectiveness: During the TBS programme implementation 2006-2008, the educational outcomes indicators have generally improved. However, by definition, it is not possible to attribute all or any of the above effects directly or solely to the E-NTP. Efficiency may be assessed in relation to each of the E-NTP budget sub-headings as follows: 38 World Bank PAD: 2005. 80 Illiteracy elimination, consolidation of Universal Primary Education and achievement of Universal Lower Secondary Education: There have been significant levels of resources applied to teacher retraining in programmes to support illiterates, textbooks and Vietnamese language teaching materials aimed at illiteracy elimination but despite relative progress by disadvantaged districts during the three years of the programme the remaining FSQL gap between them and the other groups of districts is still significant. Reform of curriculum and textbooks: Very high efficiency, targeted directly to poor children, since textbooks are provided free only to children from poor families, whereas less-poor families have to purchase textbooks. Teacher retraining and upgrading short courses: High efficiency, with courses provided on a "cascade" model, whereby either (i) a key teacher is sent for training from a school or district, and then he/she returns to impart the training to fellow teachers in the school or district, or (ii) a key teacher is sent out from the provincial level to impart training to teachers in a poorer district or school. Improvement of school infrastructure: Generally high efficiency, with the building of new classrooms in poorer schools and/or poorer districts, although it seems that at least some other poor schools and/or poor districts did not benefit. Improvement of pedagogical colleges and teacher training: Expenditures on these colleges are more questionable since these colleges continue to produce large numbers of new teachers but, due to the underlying demographic trends, each year, fewer new teachers are now required, and none at all in some provinces or districts, for the foreseeable future. E-NTP budget allocations from MoET to DoETs seem to have been broadly targeted to allocate increased resources to the poorer provinces, with the less-poor provinces receiving only relatively small allocations. There has been significant under-spending of the E-NTP budget each year, under all budget sub-headings. The quality of monitoring has not been very satisfactory and has been criticized by both senior management of MoET and by the donors. Previous evaluation reports have commented that MoET is significantly lacking in technical expertise relating to M & E and EMIS. Impact: As a matter of principle, it is too early to be able to draw any conclusions at all at a national level with regard to impact, since impact effects may typically not arise until after several years have passed. However, the basic education sector has continued to develop successfully over the lifetime of the programme, so that, at least, it can be said that there is a positive correlation. In particular, the Government of Vietnam (GOV) is to be complimented on the fact that Vietnam has come very close to achieving EFA, which is a significant achievement. Sustainability: A major factor is the steadily rising demand for education, which is likely to continue. Thus, access to primary schooling is likely to continue to increase. However, there continue to be significant reservations with regard to the quality of the education provided. Although the TBS funding as such will not continue, the GOV has committed itself to providing a 81 steadily increasing percentage of the state budget for the education sector (currently estimated at just over 20%). 1.3 Fundamental School Quality Levels (FSQL) The FSQL for Vietnam is defined by PEDC as follows: "Fundamental School Quality Level (FSQL) is the first step to achieve criteria of a national standard school of Vietnam. FSQL identifies a series of minimum necessary inputs for provision of high-quality education in primary school as well as expected outcomes of achieved schools. Schools achieving FSQL will have good initial step in the process of gaining criteria of a national standard school." The FII (FSQL input index) consists of five components with altogether 29 indicators related to the evaluation of school quality. Between the baseline year 2005 and 2008, the FSQL input indicator has progressed on nationwide from 65 to 69.6 score. Differentiating the districts in three big groups indicates that the disadvantaged districts progressed with plus 5.8 points more in comparison to the averagely disadvantaged districts (plus 3.5) and the advantaged districts (plus 3.9). The resource allocation of all educational programs including the general education budget benefited the poor districts more. 1.4 Quality of primary education Quality of primary education. Measuring and monitoring the quality of education is a much more difficult task than to measure the access to education. The TBS-EFA uses the student: teacher ratio as a proxy indicator for quality of education and so far available the student achievement in mathematics and Vietnamese as an outcome indicator for quality of education. With regard to TBS EFA support in relation to the 5 TBS EFA priority spending areas in the E- NTP: (a) Illiteracy elimination, consolidation of Universal Primary Education and achievement of Universal and Lower Secondary Education: The elimination of illiteracy on national scale is not monitored by MoET, also the E-NTP Implementation Report (JAR 2009) does not mention it. The general policy of MoET is to get children into the formal education system. (b) Renovation of the curriculum and textbooks/teaching methods: The renovation of the curriculum for the primary and lower secondary education cycle was completed or consolidated during the reference period (2006 ­ 08). (c) Upgrading infrastructure of Teacher Training Institutions and teacher qualifications: Nearly all teachers in basic education schools had received pre-service and/or in- service training. Applying this criteria nearly all teachers are rated as qualified (99% of the teachers have attained the 9+3 level). 82 (d) Support for the education of ethnic minorities and disadvantaged regions: Under E- NTP 2006-09, the second largest budget item (VND 2,900 billions) after school infrastructure has been provided for this target group. (e) Infrastructure development: Between 2006-08 under the school infrastructure sub- component of E-NTP, new construction and renovation of 13,367 classrooms for all subsectors but only 3,416 classrooms were built for primary education. For the improvement of primary education school facilities, this is a rather marginal impact. 1.5 Responses to Additional Points raised in the TOR Problems and constraints. The donors seem to have found the implementation of this programme quite frustrating, time-consuming and perhaps even exasperating, especially in view of the difficulties in timely completion of the required conditions for the successive tranche releases of funds. These problems were real and cannot be ignored. However, they should not detract from the relative successes of the programme in largely leading to relatively high efficiency in the provision of textbooks, classrooms and teacher training inputs to the poorer schools in Vietnam. A major issue arising was the question of MoET progressively giving greater priority to upper secondary schools. The LogFrame, as attached to the EC's Financing Agreement, was of frankly poor quality throughout and it is frankly disappointing that a more appropriate set of LogFrame Indicators could not have been produced. Almost all the Indicators quoted were not directly applicable to the E-NTP as such.39 In more detail: Outcome Indicators: It should have been realized that it would never have been possible to attribute "gross intake in grade 1", drop-out, repetition, etc., solely to the effects of this programme (quite apart from the problems arising from the lack of a satisfactory EMIS). Similarly, it should have been realized that for "student achievement in grade 5..." no up- to-date data exist during the lifetime of this programme.40 Results Indicators: Data relating to FSQL have been considered above. For "educational quality assessments", it should have been realized that no up-to-date data exist for the lifetime of this programme. 41 Education Management. The quality and timeliness of annual progress and financial reports has been a persistent problem and this aspect of the programme has not worked well. A review of GOV plans for education has been undertaken by MoET. 39 The World Bank is in disagreement with this statement. While there was room for developing the results framework further, outcome indicators are by definition difficult to attribute to any single program, which does not mean that they should not be monitored. 40 The World Bank is in disagreement with this statement. There was a clear plan from the beginning to re-assess grade 5 learning outcomes in 2007 to capture possible preliminary effects of the program. 41 The World Bank is in disagreement with this statement. There was a clear plan from the beginning to re-assess grade 5 learning outcomes in 2007 to capture possible preliminary effects of the program. 83 Education Finance. Performance was good at the national level, in that the target indicator given ("Education budget amounts to 18.6% of the national budget") has been far exceeded by GOV. It was not so good at the local level, where families are still required to make substantial financial payments to schools. Contribution to developments in the basic education sector in general. This direct effect cannot be identified precisely. However, the basic education sector has continued to develop successfully during the lifetime of this programme. Lessons learned. With any future programme, more care should be taken in advance to identify the practical problems that will arise in practice, e.g., with regard to human resource constraints and M. & E. limitations. However, harmonization by the donors did have some positive effects, including, for example, this present combined final review on behalf of all the donors jointly, as opposed to an onerous series of separate final reviews by each donor. Changes relating to GOV policies. Apparently during this relatively short period, there have not been any major changes in government policies, strategies or actions by GOV but in practice MoET has continued the previously well established trend in expenditure decisions to gradually give greater priority to upper secondary schools. Contribution to the results identified at the outcome and impact levels: (i) educational outcomes generally good but cannot be attributed directly or solely to E-NTP, (ii) impact too early to be able to assess, and (iii) sustainability prospects good, especially in view of the high demand for education and the GOV commitment to a high level for the State education budget. Overall relevance of the TBS EFA arrangements in view of the evolving country and sector specific contexts. Generally good, with the design and objectives of this programme having a high degree of relevance, and in view of the crucial importance of a successful education sector for the economic and social development of Vietnam, but with the major concern relating to the relatively low quality of the education provided in schools at all levels. Efficiency of TBS EFA operations. Considering both the process and the relation between effects (direct outputs, induced outputs and outcomes) and inputs there generally was high efficiency, with the school classrooms, textbooks, and teacher training provided generally being targeted, as far as it was possible to do so, to the poorer provinces and the poorer districts. Links with any induced outputs (e.g., gender equalisation?, environmental impact?) are more difficult to identify at this early stage. Component II. Component II will continue up until 2010. The objectives of the Work Plan 2009-2010 as revised and updated are found to be in alignment with the identified needs of the MoET. Four major streams of activity are being implemented: (i) support for improved E-NTP management and administration; (ii) a comprehensive capacity building for E-NTP at provincial and sub-provincial levels; (iii) provision of improved tools for E-NTP reporting, assessment, analysis and evaluation; and (iv) streamlining governance and activities to improve inter- 84 ministerial co-ordination. A further area covers the strengthening of the management effectiveness and sustainability of the capacity building role of the Support Unit (SU). The current Workplan is considered to be a good example covering new approaches. The challenge for the MoET assisted by its Secretariat Unit (SU) is seen to be one of sustaining recent momentum and the delivery of a comprehensive and ambitious training set of programmes. The approved Action Plan relies on the appointment on international and national technical assistance. The key factor for the success will be the early fielding of these experts. The FEM observed generally that the necessary facilities and linkages are required to be put in place if the SU is to be effective. In view of the imminent appointment of international expertise, it will be necessary to make provision for Vietnamese back-up assistance to be available either by assignment of personnel or on a sub-contracted basis to ensure optimum communications and information transfer from these experts. On the achievement of targets, there has been good progress reported with the integration between both the BCEP team and training methodology in TBS and government institutes with recruitment of PST and trainers in time, and the establishment of a "capacity building administration facility", reporting directly to the Department of Planning and Finance executive and responsible for delivery of all training. The integration of BCEP into TBS 2 has been one of the key achievements to date of TBS 2. Component II has provided support for the extension of VANPRO to District level planning; the training and coaching activities from the capacity building strategy have commenced; and there has been engagement and collaboration with DPF Statistics on the DFA and with the MoET Department of Ethnic Minorities on research on Ethnic Minority education. Public Finance Management. Budgeting, financial accounting, expenditure control systems and annual financial statements steadily improved.42 The issuance of the Joint Circular (JC) and the FM Manual should further contribute to these improvements, although they failed to address all weaknesses. The Audit Reports included both positive and negative aspects. Overall, the TBS was about as successful as it could be within the constraints of the sector and the in-country situation within which it was operating. It is especially important that the design of the programme was fully supportive of the objectives of the GOV's Education Strategy, and in particular the National EFA Action Plan. 1.6 NTP mechanisms The assumption made in the PAD that the NTP mechanism is a useful vehicle to enhance pro-poor allocative efficiency and provide additional resources for the accomplishment of agreed upon sectoral priorities was partially unrealistic. It did not allow for the practical implementation problems that would arise. The evolving focus of future expansion of the E-NTP away from EFA and primary and lower secondary targets was not clear during the negotiation process leading up to the signing of the 42 The World Bank notes that there is still room for significant improvement. 85 Memorandum of Understanding. It has become identified as a major issue in relation to its suitability as a vehicle for continuing support by donors focusing on EFA and primary issues. Poor reporting by government agencies has been identified as a major obstacle to improved partnership between government and the donor partners. In retrospect, as previous evaluations have commented, it seems clear that donors were initially over-ambitious and that insufficient attention was paid to the significant problems that would arise subsequently, especially with regard to audits, M. & E., reporting back, and sustainability. Previous evaluation reports indicate that donors were surprised that there should have been such under-spending but this could have been anticipated since it seems clear that this is endemic throughout the public finances in Vietnam.43 The capacity building component under Component II was not as extensive as originally envisaged but was increasingly prioritized from year 2008/9 with work on education planning etc. It had been anticipated it should have had sector wide impact on management. This never happened and its omission has impacted negatively. Nevertheless it becomes opportune to utilize the funds earmarked for Component II to address milestones such as the introduction of the Joint Circular to be studied along with the lessons learned from earlier capacity building/institutional strengthening initiatives and for them to be incorporated into the MOET's design for this component. GOV officials emphasise that while the programme inputs under Component I were relatively small in relation to the size of the total state budget for education and training the effects of the (much smaller) Component II could well be much more significant for MOET. 43 Source: "Ministries asked to release school funds", Vietnam News, 2 September 2009 86 Annex 9: List of Supporting Documents Aide Memoires, Back-to-Office Reports, Project Status reports and Implementation Status Reports, 2004-2009 Ann Bartholomew, "Scoping Study to Design an Evaluation Framework for Targeted Budget Support in Vietnam", June 2007. Final Evaluation of Sector Policy Support Programme in Education (Targeted Budget Support for Education For All), Vietnam, November 2009. Project Appraisal Document, 31735-VN, 2005 Project Appraisal Document, 23846-VN, 2003. Targeted Budget Support ­ Education for All Project, "Annual Review ­ Planning, Budget Allocation and Monitoring of ENTP and Use of FSQL", September 2007. Targeted Budget Support ­ Education for All Project, "The Utilization of Fundamental School Quality Level for Planning and Budget Allocation for Educational National Targeted Programs: The Case of Tien Giang and Tuyen Quang", September 19, 2007. Targeted Budget Support ­ Education for All Project, "TBS-EFA Implementation Program Joint Annual Review" 29 September to 8 October 2008. Targeted Budget Support ­ Education for All Project, "TBS-EFA Implementation Program Joint Annual Review" 21 September to 8 October 2009. Targeted Budget Support ­ Education for All Project, "Planning, Standards and Quality Thematic Working Group", updates 2008. Targeted Budget Support ­ Education For All Project, "Financial Management and Procurement Thematic Working Group", updates, 2008-2009. Vietnam Ministry of Education and Training, "Building Capacity in Education Planning Project (BECP), TF053232, Summary Report" Hanoi, Vietnam, September 2009. Vietnam Ministry of Education and Training, Primary Education for Disadvantaged Children, "2006, 2007, and 2008 District FSQL Audit, FSQL Statistical Data & Analysis", various years. Vietnam Ministry of Education and Training and Ministry of Finance, "Governing the Management and Utilization of ENTP Funds till 2010" Draft Joint Circular, November 4, 2008. Vietnam Ministry of Education and Training, Audited Financial Statements, Financial Management Audits and Procurement Audits, 2006-2008 87 Vietnam Ministry of Education and Training, "Borrower Completion Report", Department of Planning and Finance, November 2009. Vietnam Ministry of Education and Training, University of Melbourne, World Bank, NIES, "Reading and Mathematics Assessment Study", 2001 Vietnam Ministry of Education and Training, World Bank, NIES, "Reading and Mathematics Assessment Study", 2008 University of Melbourne, World Bank, DFID, "Secondary Analyses of Data on Education Outcomes", November 2009 88 IBRD 33511R 102°E 104°E To 106°E 108°E 110°E To To Babao Kunming VIETNA M Kaiyuan To Tiandong CHINA Ha Giang Cao Bang 4 PROVINCE CAPITALS 5 To Lao Cai Re Nanning Phong Tho 1 d 9 NATIONAL CAPITAL 3 Bac Can 8 22°N 22°N To RIVERS Tuyen Quang 10 Lang Son Hepu Bla 7 Thai 2 c k Yen Bai Nguyen MAIN ROADS 13 Dien Son La Viet Tri 12 Vinh Yen 14 RAILROADS Bien Phu 11 16 Bac Giang 15 6 Bac Ninh To HANOI 17 Hai Duong Ha Long PROVINCE BOUNDARIES Muang Xai Hoa Binh Ha Dong 19 20 Hai Phong INTERNATIONAL BOUNDARIES 22 18 Hung Yen 21 Ha Nam 23 24 Thai Binh Nam Dinh Ninh Binh 26 Ma 25 20°N 27 20°N To PROVINCES: Luang 1 Lai Chau 32 Thua Thien Hue LAO Prabang Thanh Hoa 2 3 Dien Bien Lao Cai 33 34 Da Nang Quang Nam PEOPLE'S 28 Gulf 4 Ha Giang 35 Quang Ngai D EM . REP. DEM. of Hainan I. (China) 5 Cao Bang 36 Kon Tum A Vinh Tonk in 6 Son La 37 Gia Lai 7 Yen Bai 38 Binh Dinh n Ha Tinh n 8 Tu Yen Quang 39 Phu Yen 29 a a a 9 Bac Can 40 Dac Lac 18°N To 18°N Khammouan m m m 10 Lang Son 41 Dac Nong 11 Phu Tho 42 Khanh Hoa 12 Vinh Phuc 43 Binh Phuoc Dong Hoi C 30 13 Thai Nguyen 44 Lam Dong o 14 Bac Giang 45 Ninh Thuan r d d d 15 Quang Ninh 46 Tay Ninh Dong Ha To il 16 Ha Noi 47 Binh Duong Savannakhet 31 17 Bac Ninh 48 Dong Nai THA ILA ND THAILAND Hue lll e e e 18 Ha Tay 49 Binh Thuan r 19 Hung Yen 50 T.P. Ho Chi Minh 32 33 Da Nang a a a 20 Hai Duong 51 Ba Ria-Vung Tau 16°N 16°N 21 Hai Phong 52 Long An 22 Hoa Binh 53 Tien Giang Tam Ky 34 23 Ha Nam 54 Dong Thap 24 Thai Binh 55 Ben Tre Quang Ngai 25 Ninh Binh 56 An Giang 35 26 Nam Dinh 57 Vinh Long Ngoc Linh (3143 m) 27 Thanh Hoa 58 Tra Vinh 36 Kon Tum 28 Nghe An 59 Kien Giang 29 Ha Tinh 60 Can Tho 38 30 Quang Binh 61 Hau Giang 14°N Central 14°N Pleiku 31 Quang Tri 62 Soc Trang 37 Quy Nhon 63 Bac Lieu 64 Ca Mau Highlands 39 Tuy Hoa CAMBODIA 40 Buon Ma Thuot 42 41 Nha Trang Gia Nghia 12°N To Da Lat 12°N Kampong Cham 43 To Dong Xoai 44 45 Kampong Chhnang Phan Rang- 46 Thap Cham Mekong Tay Ninh 48 Thu Dau47 Mot 49 Bien Hoa Gu l f 52 50 Ho Chi Minh City Phan Thiet 54 of 56 Cao Lanh Tan An 51 Th a i l a n d Long Xuyen 53 My Tho Vung Tau Phu Vinh Long Ben Tre VIETNAM Quoc 60 57 10°N Rach Gia Can Tho 55 10°N lta 59 61 Tra Vinh Vi Thanh 58 De This map was produced by 62 Soc Trang the Map Design Unit of The g World Bank. The boundaries, 63 n k o0 Bac Lieu colors, denominations and Me Ca Mau any other information shown 50 100 150 200 Kilometers on this map do not imply, on 64 the part of The World Bank Group, any judgment on the legal status of any territory, 0 50 100 150 Miles or any endorsement or acceptance of such boundaries. 104°E 106°E 108°E JANUARY 2007