Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BD: Northern Areas Reduction-of-Poverty (P114841) Report Number: ICRR0021767 1. Project Data Project ID Project Name P114841 BD: Northern Areas Reduction-of-Poverty Country Practice Area(Lead) Bangladesh Urban, Resilience and Land L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IDA-50150 31-Dec-2015 24,573,313.66 Bank Approval Date Closing Date (Actual) 27-Oct-2011 31-Dec-2018 IBRD/IDA (USD) Grants (USD) Original Commitment 29,290,000.00 0.00 Revised Commitment 29,290,000.00 0.00 Actual 24,608,635.09 0.00 Prepared by Reviewed by ICR Review Coordinator Group Katharina Ferl Vibecke Dixon Christopher David Nelson IEGSD (Unit 4) 2. Project Objectives and Components DEVOBJ_TBL a. Objectives According to the Project Appraisal Document (PAD) (p. 7) the objective of the project was “to facilitate access to employment opportunities in the garment sector for poor and vulnerable women from lagging areas of Bangladesh by providing information, technical and life skills training, transitional housing, and other support to adjust to urban life and formal sector employment.” Page 1 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BD: Northern Areas Reduction-of-Poverty (P114841) The Financing Agreement of December 4, 2011 (p. 5) stated that the objective of the project was “to facilitate access to employment opportunities in the Recipient’s garment sector for poor and vulnerable women from lagging areas”. This ICR Review will use the objective as stated in the Financing Agreement. b. Were the project objectives/key associated outcome targets revised during implementation? No c. Will a split evaluation be undertaken? No d. Components The project included four components: Component 1: Raising awareness and selecting candidates in the monga-prone districts of northern Bangladesh (appraisal estimate US$0.92 million, revised to US$1.04 million, actual US$0.75 million): This component was to finance Information, Education, and Communication (IEC) campaign, targeting of intended populations and screening of applicants, orientation program for candidates and selection of beneficiaries, and transportation of beneficiaries from the source areas to the receiving areas. Component 2: Establishing training centers with dormitories in the EPZs located in Dhaka, Ishwardi, and Karnaphuli (appraisal estimate US$23.64 million, revised to US$21.35 million, actual US$18.4 million): This component was to finance the construction of training and transitory accommodation facilities and the provision of furniture, equipment, and management services for the facilities. Component 3: Providing initial training and on-going support to beneficiaries (appraisal estimate US$0.73 million, revised to US$1.58 million, actual US$2.2 million): This component was to finance the delivery of a technical training curriculum for garment industry work, the delivery of a life skills training curriculum to adapt to urban life and working in factories, Placement services for on-the-job training and employment in EPZ factories, and self-help and counseling for trainees and graduates. Component 4: Supporting coordination, M&E, and program for expansion (appraisal estimate US$4.0 million, revised to US$5.32 million, actual US$3.2 million): This component was to finance project management and coordination, M&E of the activities, and preparation of a program for expansion. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project Cost: The project was estimated to cost US$36.29 million. Actual cost was US$31.60 million. Financing: The project was financed by an IDA credit in the amount of US$29.29 million of which US$24.60 million was disbursed. Borrower Contribution: The Borrower was to contribute US$7 million, which materialized. Page 2 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BD: Northern Areas Reduction-of-Poverty (P114841) Dates: The project was restructured twice (both level 2 restructurings):  On February 17, 2014 the project was restructured to extend the project’s closing date from December 31, 2015 to October 31, 2017 to accommodate the delays in the construction of the Dormitories and Training Centers (DTCs) and the start of the training program.  On June 5, 2016 the project was restructured to make the following changes: i) extend the closing date from October 31, 2017 to December 31, 2018 to accommodate again the delays in the construction of the DTCs and the start of the training program; ii) change in components and cost (the allocation for component 1 was raised from US$0.92 million to US$1.04 million to fund the extension of the contract for the IEC campaign; costs savings from constructions under component 2 were allocated towards major cost increments in food, security, cleaning, management personnel and other operational costs; under component 3 the cost for providing training was increased from US$1.80 million to US$3.12 million; under component 4 the cost for supporting, coordination, M&E and program for expansion for the extended closing date was increased to US$5.32 million; iii) reallocation between disbursement categories; iv) change in legal covenants; and v) change in implementation schedule. 3. Relevance of Objectives Rationale In 2011, by the time of project appraisal Bangladesh had experienced strong economic growth and significant gains in poverty reduction. According to the 2010 Household Income and Expenditure Survey (HIES), the national poverty rate had declined from 40 percent in 2005 to 31.5 percent in 2010 and estimated at 17.6 percent at the national level, 21.1 percent in rural areas, and 7.7 percent in urban areas. In the Northern areas such as the Rangpur division, the poverty rate was at 42.3 percent, significantly higher than the national and rural averages. The Ready-Made Garment (RMG) sector is important for the Bangladeshi economy. In 2010, women represented 80 to 85 percent of the workers and particularly benefit from the sector by gaining financial independence, protection against early marriage and greater voice and agency in the home. However, women participated less than half the rate that men do in the labor force. According to the ICR (p. 5) the low participation of women in the work force is related to lack of information about job opportunities, adequate training, and lack of networks to be able to move to urban centers. In 2012, the government implemented The Northern Areas Reduction of Poverty Initiatives (NARI, 2012- 2018) to promote inclusive growth and encourage women’s economic empowerment. The objective of the project was well aligned to NARI’s objective. The objective of the project was also in line with the Bank’s Country Assistance Strategy (FY11-14) which supported women’s empowerment, especially, pillar four recognized women’s limited access to employment and included the establishment of employment services for women from the poorest areas of the country as a milestone in the indicators measuring women’s participation in short-term employment programs, and it was in line with the most recent Country Partnership Framework (FY16-20) which focuses on skills development, women’s economic empowerment and inclusive growth and promotes female labor force participation. However, while there was clear alignment between the project’s development objectives and the country- and Bank strategies, the relevance of the objectives was pitched at a level that did not adequately reflect a Page 3 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BD: Northern Areas Reduction-of-Poverty (P114841) potential solution to a development problem. While acknowledging the difficulty of the operational environment, a shortcoming here was that the objective was at a very low level of the logical chain. “To facilitate access to employment opportunities” alone was not outcome focused and did not help in understanding what development results were expected as a consequence of the project. Some of the project’s indicators identified expected outcomes at a higher level (such as long-term employment and increased income for women), but this was not captured in the PDO formulation. Rating Relevance TBL Rating Substantial 4. Achievement of Objectives (Efficacy) EFFICACY_TBL OBJECTIVE 1 Objective To facilitate access to employment opportunities in the Recipient’s garment sector for poor and vulnerable women from lagging areas (all outputs and outcomes lacked a baseline): Rationale The project’s theory of change linked inputs such as providing mechanisms to target and screen applicants, conducting an Information Education and Communication (IEC) campaign, providing technical and life skills training, constructing training centers and dormitories, and developing a framework for job placement with outputs including community awareness and recruitment of program candidates, training program and placement in a RMG job, and increased stakeholder engagement. The theory of change assumed that these inputs and outputs were to facilitate access to employment opportunities in the garment sector for poor and vulnerable women and improved knowledge of life and technical skills of NARI graduates. The medium-term outcomes were defined as improvements in career development and income of NARI graduates. However, as stated in the section above, the project’s inputs were not clearly linked to the project’s potential medium-term outcomes. Outputs: (all outputs and outcomes lacked a baseline)  Three dormitories and training centers (DTCs) located in Dhaka, Ishwardi, and Karnaphuli were established, achieving the target of three DTCs. These DTCs could train and house a maximum of 600 women per training session.  The program trained 9,099 women, almost achieving the target of 10,800 women. The technical training curriculum included modules on machine operation and safety. The life skills curriculum included modules on communication in English and Bengali, labor law and rights, women’s empowerment, professional and personal life management, personal and reproductive health and use of technology. Between July 2016 and November 2018, a total of 23 cohorts were trained. The training ranged from 245 to 532 hours. Page 4 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BD: Northern Areas Reduction-of-Poverty (P114841)  Awareness was raised and 12,241 applicable training candidates were selected from the lagging region of northern Bangladesh, surpassing the target of 10,800 people. According to the Bank team (August 22, 2019), the extend to how much awareness was risen was measured by the number of people applying for the training and included looking at whether the applicants fit the training criteria.  All beneficiaries completed the two and half month training offered by the program, surpassing the target of 95 percent.  20 percent of graduates stayed in the accommodation after the training was completed, not achieving the target of 95 percent. According to the ICR (p. 11) most graduates returned home immediately after completing the training. The Bank team stated (August 22, 2019), the majority of trainees preferred to live closer to the factories where they were employed, and many formed groups through networking with existing factory employees and preferred to live together in shared facilities/housing.  90 percent of beneficiaries were satisfied with the services offered by the facilities and ranked their satisfaction as three or higher on a five-point scale, achieving the target of 90 percent. Outcomes:  43 percent of graduates were formally employed for at least 10 months of the year following the completion of their training, not achieving the target of 90 percent.  90 percent of graduates ranked their satisfaction level with the project as three or higher on a five- point scale, surpassing the target of 80 percent.  All project graduates were offered formal employment within one month of graduation, achieving the target of 100 percent. 67 percent accepted an employment offer (there was no target for this outcome). The ICR (p. 9) stated that the graduates who accepted a job offer entered as operators (53 percent), senior operators (45 percent) or quality inspectors (2 percent). All levels were above the typical entry level for unskilled workers.  Over 50 percent of trainees experienced an increase in income, achieving the target of 50 percent. While the income for almost 80 percent of the trainees was below BDT 8,000 before the training, the income of 65 percent of women was above BDT 8,000–10,000 after the training.  95 percent of the graduates opted to pay for the meal service provided by the project facilities, achieving the target of 95 percent. Only one of the outcome indicators was not achieved (“percentage of graduates formally employed for at least 10 months”), while the targets of all other indicators were either achieved or exceeded. In addition, many graduates pursued higher education which indicated increased empowerment of women. The objective was therefore substantially achieved. Rating Substantial OVERALL EFF TBL OBJ_TBL Page 5 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BD: Northern Areas Reduction-of-Poverty (P114841) OVERALL EFFICACY Rationale The achievement of the objective was rated Substantial. Overall Efficacy Rating Substantial 5. Efficiency Economic Efficiency: The PAD conducted a benefit-cost analysis and identified the quantifiable benefits as (i) an increase in the probability of being employed compared to staying in the monga-prone districts (monga is a cyclical phenomenon of poverty and hunger); and (ii) a higher income in comparison to both women staying in the monga-prone districts and migrants in the garment industry with no initial training and support. The analysis in the PAD estimated lifetime wage differentials of NARI graduates with respect to the comparison group. Using a discount rate of 7 percent, the analysis estimated a Net Present Value (NPV) of US$865 and a rate of return of 23 percent. The ICR (p. 12) used data from the qualitative assessment (conducted between December 2018 and March 2019) and baseline and estimated that a graduate earned US$105 per month compared to women who did not participate in the training program who earned US$37 per month. Using a discount rate of 7 percent, the ICR estimated that over a working lifespan of 15 years, a trainee would earn US$10,827 compared to a non-trainee who would earn US$4,475 which is more than the double. The NPV of lifetime earnings differential was calculated at US$2,260 and a rate of return at 55 percent. Operational Efficiency: The project experienced several implementation delays related to construction and the training program resulting in the need to extend the project twice to a total of 36 months. The original project duration was planned to be four years with training beginning in the second year. However, project preparation was lengthy and took three years and due to implementation delays, the project ended up being six years with training beginning in the fourth year. 84 percent of the funds were disbursed at project closing. Efficiency Rating Modest a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Page 6 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BD: Northern Areas Reduction-of-Poverty (P114841) Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal 0  Not Applicable 0 ICR Estimate 0  Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome The relevance of the objective was rated Substantial given that it was well aligned with both national and Bank priorities, but the objective was pitched at a level that did not adequately reflect a potential solution to a development problem. Efficacy is rated Substantial and Efficiency is rated Modest. Taking everything together, the overall outcome rating is Moderately Satisfactory. a. Outcome Rating Moderately Satisfactory 7. Risk to Development Outcome The project has built the capacity of all the stakeholders including the PCU and the PIU at BEPZA and MOLE. According to the Bank team (August 22, 2019), the main risk for project sustainability is financial, as neither MOLE or BEPZA can completely cover the cost of continuing and expanding the training program. BEPZA’s Board has proposed a model whereby the training centers will remain operational as per the project design and funded by private sector companies already active in the RMG sector. Under this model, part of the dormitories will be used as housing for current female workers, with their lodging expenses paid by their employers. The funds from this scheme will be used to cover expenses for trainees and maintenance of the dormitories. The model will be discussed at the next Governor’s Board meeting which is presided over by the Prime Minister who has already expressed her support for the proposed model. 8. Assessment of Bank Performance a. Quality-at-Entry The ICR (p. 19) stated that the Bank team included multi-disciplinary members and collaborated with a variety of entities in the selected project areas during project preparation. Also, the Bank team conducted field visits to assess the issues, which project beneficiaries were facing. According to the ICR (p. 14) the Bank identified relevant risks. Several risks were rated as High including social tension created by current factory workers and local populations who were not to benefit from the project directly, lack of community support for the project in the source region reflecting entrenched traditional views that take a very limited, Page 7 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BD: Northern Areas Reduction-of-Poverty (P114841) and conservative attitude towards economic opportunities for women, and criticism of the project at the country level for providing public subsidies to the private sector. Also, the Bank identified the risk of criticism by international audiences unfamiliar with the socio‐economic context of Bangladesh and women’s economic possibility in factories. The Bank team (August 22, 2019) stated that the mitigation efforts were adequate. An Information and Education Communication (IEC) campaign was carried out in the NARI recruitment areas to raise awareness among the population about the training program and help potential trainees make an informed decision as well as to develop trust in the program by the local population. The IEC campaign included garment workers from the Northern areas as role models. Capacity building activities were conducted within the Ministry of Labor and Employment (MoLE) and the Bangladesh Export Processing Zone Authority (BEPZA) and included training, dialogues and communication/support from the Bank. Vulnerability of the beneficiaries (during selection, transportation, training) was mitigated through careful selection of beneficiaries, well developed transportation plans and counseling at the Dormitories and Training Centers (DTCs). According to the ICR (p. 14) The Bank team prepared a socio-economic assessment, a vulnerability map and social risk management plan. Also, the project identified adequate institutional arrangements and selection criteria. Quality-at-Entry Rating Moderately Satisfactory b. Quality of supervision According to the ICR (p. 19) the Bank conducted nine supervision missions and engaged with the implementing partners on a weekly basis. Also, the Bank team addressed implementation challenges on a continuous basis with the project’s steering committee and other counterparts in the government. The Bank’s Aide Memoirs and Implementation Supervision Reports were candid and raised relevant issues to the Bank’s management. The ICR (p. 19) stated that the Bank organized visits to the DTCs with international buyers and Bank management and conducted four learning exchanges. Furthermore, the Bank identified the need to develop sustainability plans with both implementing partners but failed to develop an appropriate exit strategy. However, the Bank did not modify the Results Framework during the project restructurings since the Bank was concerned that a more extensive restructuring would result in the cancellation of the credit due to all the implementation challenges. Also, the Bank did not make any changes to following-up with trainee graduates, which had a negative impact on the project’s M&E activities. Quality of Supervision Rating Moderately Satisfactory Overall Bank Performance Rating Moderately Satisfactory Page 8 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BD: Northern Areas Reduction-of-Poverty (P114841) 9. M&E Design, Implementation, & Utilization a. M&E Design The Results Framework included four PDO indicators and eight Intermediate Outcome Indicators, which encompassed all outcomes of the PDO statement. The selected indicators were measurable and relevant. However, the objective of the project was specified at a very low level of the logical chain and closer to input level than output level. Some of the indicators were at a higher level than the actual PDO, which showed that the expected outcomes (“long-term employment” and “increased income”) were at a higher level which was not captured in the PDO (“facilitate access to employment”). MOLE was responsible for the project’s M&E activities. Data for the selected indicators were to be collected from BEZA and MOLE monitoring reports on recruitment, training and placements. Also, the project was to conduct an Impact Evaluation, which was to collect data from trainees in regards to wages and employment. According to the Bank team (August 22, 2019), the M&E arrangements were adequately imbedded in the local institutions. b. M&E Implementation According to the ICR (p. 17) M&E activities were not conducted as planned due to delays in setting up the MIS, and the challenge of following up with NARI trainees after graduation. Also, the ICR (p. 15) stated that the Impact Evaluation was cancelled since the number of applicants in the first training sessions was low, resulting in the training partners allowing applicants from the control group to join the training. The MIS was supposed to be developed by a consultant firm. However, according to the ICR (p. 17) the tendering process only started once the training activities had started, resulting in the project having to rely on data collected by the NGOs conducting the IEC activities, training and job placement. Furthermore, the Intermediate Outcome Indicators to measure medium-term outcomes were not aligned with the original and revised project timelines. Also, the ICR (p. 17) stated that the indicators were not adapted during the two restructurings since the Bank team was concerned that a more extensive restructuring would result in the cancellation of the credit due to all the implementation challenges. According to the Bank team (August 22, 2019) the data was found to be reliable and of good quality and M&E functions were designed to be used in similar programs in the future. Therefore, they are likely to be sustained after project closing. The M&E activities set up by BEPZA to capture data from the recruitment areas and the DTCs worked well. A digital MIS was developed by MOLE, although with considerable delays. MOLE is in the process of integrating their existing system with the MIS developed under NARI for future use. BEPZA will adapt the M&E framework depending on the operational modality they adopt for running the DTCs in the future. c. M&E Utilization According to the Bank team (August 22, 2019) the project’s M&E data guided the decision to shorten the training program. M&E data indicated that participants wanted the training period to be shorter and employers were willing to employ NARI trainees before the three-month training period was over. Page 9 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BD: Northern Areas Reduction-of-Poverty (P114841) M&E Quality Rating Modest 10. Other Issues a. Safeguards The project was classified as category B and triggered the Bank’s safeguard policy OP/BP 4.01 (Environmental Assessment). The project prepared an Environmental Management Framework (EMF). According to the ICR (p. 17) the project experienced some non-compliance issues such as initial delays in the recruitment of an environmental specialist and implementation of physical project activities. By the start of the civil works, the project met all the compliance requirements and undertook several activities that qualified NARI as a green project. BEPZA and a supervision consultant were responsible for monitoring the construction site on a regular basis to ensure the implementation of the Environment Management Plans (EMP) during construction. Also, the contractors assigned an Environmental Health and Safety manager to each site. b. Fiduciary Compliance Financial Management: According to the ICR (p. 18) the project completed all works and received all goods and services within the project closing date. Also, all accounting adjustments and documentation for all expenditures were completed. The staff of the BEPA and MOLE received fiduciary training throughout implementation and executed all financial management activities in accordance with the Bank’s guidelines and policies. The Foreign-Aided Projects Audit Directorate (FAPAD) audited the project on a regular basis and submitted the reports on time. The ICR (p. 19) stated that the project proactively addressed audit observations raised by FADAP (with two pending observations being processed when the ICR was written). Procurement: According to the ICR (p. 19) the project’s procurement plan complied with the government’s and the Bank’s policies and procedures. However, the project faced several procurement challenges including procurement processes were overly lengthy, bid evaluation reports prepared by the BEPZA were not sufficiently comprehensive and lacked detailed information which was required for an adequate evaluation. Also, contracts for supplies, food and security had a short duration of one year resulting in higher prices and the need to be modified several times. Furthermore, according to the ICR (p. 15) contracts for NGO’s responsible for IEC, training and placement services, furniture and equipment suppliers were staggered instead of done in bulk. All these challenges resulted in significant implementation delays. The ICR (p. 19) stated that BEPZA did not have a procurement consultant during the last two years of project implementation, resulting in BEZPA having to use consultants from other projects or its own staff or the Bank having to provide clearance on every procurement package using post-review. Page 10 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BD: Northern Areas Reduction-of-Poverty (P114841) c. Unintended impacts (Positive or Negative) NA d. Other --- 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Relevance of objective was Moderately rated Substantial. Efficacy was Outcome Moderately Satisfactory Unsatisfactory rated Substantial. Efficiency was rated Modest. Moderately Bank Performance Moderately Satisfactory Satisfactory Quality of M&E Modest Modest Quality of ICR --- Substantial 12. Lessons The ICR (p. 19-20) provided lessons learned which were adapted by IEG:  When project activities are designed to depend upon one another (i.e. one component cannot start until another is finished), adequate sequencing of project activities becomes critical. In this project, this was the case regarding the construction of DTCs before the start of the training program. In this project, the delay in the construction of the DTCs led to a delayed start of the training program, which had a negative impact on project outcomes.  A flexible Results Framework with adequate, realistic and measurable outcome indicators may contribute to the documentation of the achievement of the PDO. In this project, a proposed Impact Evaluation was planned to be the basis for data collection. However, once it was dropped, the project’s M&E experienced critical shortcomings. Also, the Results Framework included indicators to measure changes in wages, which is a long- term outcome, and the challenge of tracking trainees after graduation was not taken into account.  When a project includes two implementing agencies, a clear distinction of roles and responsibilities is important to ensure sufficient authority to make critical decisions in a timely manner to avoid implementation delays. In this project, the roles and Page 11 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BD: Northern Areas Reduction-of-Poverty (P114841) responsibilities were not sufficiently defined and none of the agencies had sufficient authority to make critical decisions in a timely manner which resulted in implementation delays. 13. Assessment Recommended? No 14. Comments on Quality of ICR The ICR provided a good overview of project preparation and implementation and conducted an adequate economic analysis. The ICR is concise and sufficiently candid. The ICR provided useful lessons learned. However, the ICR did not include any of the risks identified during project preparation nor discussed how these risks were mitigated. Also, the ICR did not discuss any political, financial, or institutional risks that project outcomes might face and did not state if M&E data was used to inform decision making. a. Quality of ICR Rating Substantial Page 12 of 12