IEG Report Number: ICRR14745 ICR Review Independent Evaluation Group 1. Project Data: Date Posted: 06/26/2015 Country: Nigeria Project ID: P119872 Appraisal Actual Project Name: Nigeria Statistics Project Costs (US$M): 10.00 9.77 Development Program (nsdp) L/C Number: Loan/Credit (US$M): 10.00 9.77 Sector Board: Economic Policy Cofinancing (US$M): Cofinanciers: Board Approval Date : 06/16/2011 Closing Date: 02/28/2014 02/28/2014 Sector(s): General public administration sector (77%); Information technology (20%); Sub-national government administration (3%) Theme(s): Economic statistics; modeling and forecasting (100%) Prepared by: Reviewed by: ICR Review Group: Coordinator: Ryotaro Hayashi Judyth L. Twigg Lourdes N. Pagaran IEGPS2 2. Project Objectives and Components: a. Objectives: . The project development objective, as stated in the Project Appraisal Document (PAD, p 19) and the Grant Agreement (p. 4), was “to initiate the implementation of the National Strategy for the Development of Statistics (NSDS) by producing reliable statistics in Participating States." Among 36 states in Nigeria, there were six participating states: Anambra, Bauchi, Edo, Kaduna, Niger and Ondo. b.Were the project objectives/key associated outcome targets revised during implementation? No c. Components: The project had three components. 1. Organizational, Institutional and Human Resource Development (Appraisal: US$ 2.4 million, Actual: US$ 2.88 million), included three sub-components: Organizational and Institutional Development at National Bureau of Statistics (NBS): Technical assistance and piloting were to be conducted to provide a national framework for data management and development. The development of guideline for overall coordination and statistical training programs were also included under the responsibility of NBS. Human Resource Development and Management at NBS: Technical assistance was to be provided for NBS to: i) develop and implement a strategy for human resource management, ii) establish job profiles by function, standard career path for each profile, and salary scales, and iii) outline a system of internal promotion and merit-based remuneration. This sub-component included introduction of staff performance incentives as well as a staff management information system. Human Resource Development and Management at National Population Commission (NPoPC): Training was to be provided for: i) data entry packages, such as Census and Survey Processing System (CSPro) and Statistical Package for the Social Sciences (SPSS), ii) analytical techniques related to demography, iii) research methodology, population dynamics, gender issues, reproductive health and population coordination, and iv) writing codes for scanner technology and Oracle spatial database management systems. This sub-component also included overseas training for complex survey data analysis, and procurement of equipment for supervising field works for NPoPC. 2. Statistical Framework and Information Technology (Appraisal: US$ 6.6 million, Actual: US$ 6.02 million), included four sub-components: Roll-out of Statistical Template at State Statistical Agencies (SSAs): Technical assistance was to be provided to develop a statistical framework and purchase equipment for the six participating states with the aim of rolling out the statistical template in other states, which were supported by the National Strategy for the Development of Statistics (NSDS). Information Technology (IT) at NBS: IT hardware and software infrastructure, such as personal computers and geographical information system (GIS) software, was to be purchased. Pilot Phase of Electronic Births and Deaths Registration Centers at NPoPC: A universal birth and death information system was to be piloted in existing registration centers, and investment in physical infrastructure, such as furniture and IT equipment, was also to be supported by the project. This sub-component also facilitated collaboration with UNICEF. Data Development, Management and Dissemination: Data development was to be supported to provide quality data for regular dissemination. NBS and NPoPC were also to adopt direct electronic data capture to improve reliability and efficient management of data. Consistent concepts, methods and indicators were to be developed at NBS. NPoPC also was to upgrade information and communication infrastructure at birth and death registration centers. 3. Project Management, Monitoring and Evaluation (M&E), and Statistical Communication (Appraisal: US$ 1.0 million, Actual: US$ 0.87 million), included three sub-components: Project Management and M&E at NBS: Operating costs, logistical support, and purchase of equipment at NBS were to be financed to track inputs and outputs. The multi-year survey program was also to be designed and followed up by preparing annual reports on partner support. Regular exchange of experiences and expertise were also to be promoted. Statistical Communication at NBS: Statistical awareness-raising activities were to be strengthened to enhance the use of statistics in policy making and media. Horizontal and vertical coordination among each institution were to be supported to break silos. The project was also to assist in technical support and donor coordination for effective and efficient statistical communication. Project Management and M&E at NPoPC: Operating costs, logistical support, and purchase of equipment at NPoPC were to be financed to track inputs and outputs. d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: Project cost: 97.7% of the project cost was utilized. The total project cost at appraisal was US$ 10.0 million, compared to the actual project cost of US$ 9.77 million at closure. The remaining amount was cancelled. Financing: The Statistics for Results Catalytic Fund (SRF-CF), a trust fund supported by the Netherlands and the United Kingdom and administered by the Bank, covered the entire financing. There was no contribution from the government. Dates: The project closed as planned on February 28, 2014. 3. Relevance of Objectives & Design: a. Relevance of Objectives: Relevance of Objectives is rated Substantial. The objectives were in line with government strategy and Bank strategy. The project supported the government’s strategy (NSDS), which was approved by the Federal Executive Council in March 2010 covering a period of five years (2010-2014). The NSDS was expected to promote accountability and transparency, which are important government agenda items included in overall government strategy documents such as the National Economic Empowerment and Development Strategy as well as Vision 20:20. The Bank's Country Partnership Strategy (CPS) 2010-2013 outlined the importance of statistical capacity development, including the support for implementation of the NSDS, to contribute to “Governance for Results," which was one of three main pillars of the CPS at project entry. The CPS in 2014-2018 also aims to address quality and accessibility of statistics to improve governance and public sector management, and it was built on operationalizing NSDS. The government recognized the need to further strengthen the implementation of NSDS, and also requested continued Bank support. However, it is a shortcoming that the project development objective’s definition of “reliable statistics” as well as “initiate” the implementation of NSDS creates ambiguity in interpretation. b. Relevance of Design: Relevance of Design is rated Modest. The three project components were linked with the project development objective, but causal chains toward producing reliable statistics specifically in the six participating states were weak. The project activities included human resource development of statistical agency staff, organizational and institutional development, and infrastructure and technology provision, which collectively would contribute to building capacity for producing reliable statistics and initiating the implementation of NSDS. Taking into account the limited budget under the trust fund, the project activities also intended to focus only on six states. According to the project team, there were six geopolitical zones in Nigeria, and one state per geopolitical zone was selected for this project. Each state played a coordinating role with the surrounding states in each geopolitical zone, and they were expected to have a catalytic role for other states within their geopolitical zones. While it made sense to use the federal statistics agencies to implement the project, activities and indicators used for this project did not always address the objective of producing reliable statistics specifically in the six participating states. A part of this inconsistency came from the fact that the objective did not explicitly include improved capacity of the federal statistical agencies (NBS and NPoPC), even though most of the project's activities were to be conducted by NBS or NPoPC. 4. Achievement of Objectives (Efficacy): The project development objective was to initiate the implementation of NSDS by producing reliable statistics in the six participating states. The definition of "reliable statistics" was ambiguous, as discussed in the ICR (p. 2). In the absence of clear definition of “reliable statistics," as discussed with the project team, efficacy is discussed here in terms of an internationally agreed quality assurance framework, which is consistent with the PAD (p. 19), According to the PAD, the quality assurance framework consisted of six elements: timeliness, accessibility, relevance, interpretability, coherence, and accuracy. Outputs:  The number of staff with competencies in core areas such as official statistics, demography, data analysis, sampling design, and methodology was increased between baseline and endline, but target values were not provided.  A series of staff training plans was developed and implemented at NBS and NPoPC. Over 1,100 staff in NBS, NPoPC and related ministries completed training to improve skills such as software use (SPSS), report writing, survey methodology, finance, and management. This training led to significant improvement of the methodological and data processing skills of a large cadre of statistical officers.  Six participating state statistical agencies and ministries, departments, and agencies (MDAs) had furnished office spaces with desks, chairs and computers by 2013, which improved working conditions for the staff.  A code of conduct based on core values was developed and adopted at the NBS and NPoPC, as planned.  Eight mobile centers were operational in all six participating states.  Information technology equipment and devices, such as computers, software licenses, and GIS, were purchased.  Preparation of Statistical Master Plans was supported in three participating states (Bauchi, Edo and Kaduna). Timeliness (meeting frequency and timeliness guidelines ):  A release calendar was established and functional at NBS, and it was disclosed on the agency's website. However, the publications included in the NBS release calendar were federal-level statistics, rather than state-level statistics, including the six participating states. NPoPC did not have a release calendar on its website by project closure, but it disclosed a list of the latest federal-level publications on its website.  The proportion of publications (including press releases) publically available according to the release calendar by NBS was 53% (ICR, p. ii), which nearly achieved the original target of 5 out of 9 (55.6%).  Five basic economic statistics were not generated in time to be included in yearbooks.  Statistics on the six participating states can be found on the “State Information” page at the NBS website, but most of the available statistics are around 2010 (except for Edo state, which has a 2014 statistical yearbook). These statistics were not updated to the latest year, but the project team explained that some backlogs were cleared because of the project's interventions. Accessibility (increase in the rate of user satisfaction , access to websites , and quotes of statistics in the media ):  A baseline user satisfaction survey was conducted for the first time in 2011. A second user satisfaction survey was completed at the federal level and in the six participating states in early 2014, but the results were not reported to the Bank for comparison even after one year of follow-up surveys.  Dialogue among data producers, such as NBS, NPoPC, the State Bureau of Statistics (SBSs), and State Statistical Agencies (SSA), as well as between data producers and users, was strengthened.  The six participating states' demographic, economic and social statistics were disseminated through maps, which exceeded the original target of three states. With these efforts, the incidence of NBS and NPoPC statistics quoted in the media was increased. The ICR states that the number of editorials in the largest national newspapers covering poverty data, GDP, unemployment, and inflation increased. However, these were federal-level statistics, and increased accessibility to data from the six participating states is unclear. Relevance (availability of important new statistics ):  Edo state has posted its 2014 statistical yearbook on the “State Information” page at the NBS website, but Bauchi and Kaduna states have not posted detailed statistics. Anambra, Niger and Ondo have posted some statistics, such as public finance statistics and a digest of agricultural statistics.  Five thematic areas based on internationally accepted common concepts (gross domestic product, poverty, consumer price index, health, and water/sanitation) were adopted and used by states and the NBS, which was less than the target of seven. Some of these data included state-level statistics and were included in statistical yearbooks, and a transition was made to international standards for statistics on GDP, water/sanitation, consumer price index, and poverty.  The NPoPC website does not have pages containing statistics for each state. Though major NPoPC publications, such as the Demographic and Health Survey 2013, included statistics by state and the results were disseminated in each zone, attribution of production of reliable statistics to this project is not clear.  While second user satisfaction survey results have not been analyzed, the user satisfaction survey introduced for the first time could bring a critical mechanism to strengthening the feedback loop for the improvement of data quality. Interpretability (describe concepts, data sources and statistical methods ):  Six states' micro data sets were integrated into the NBS and NPoPC National Data Archive Center, as planned.  50,000 electronic birth/death data points were captured (no baseline/target) as a pilot phase of electronic births and deaths registration.  According to the NPoPC, all concepts and methodology in census, migration, and vital statistics data collection were based on international standards.  There were one million registration at pilot centers in the six states (no baseline/target), but the measurement process was ambiguous.  The sampling frame was improved by the use of satellite images and GIS maps to identify enumeration areas. Coherence  A template for the state statistical yearbook was adopted.  A compendium of statistical terms was updated.  State statistical agencies in five states (Anambra, Edo, Kaduna, Niger and Ondo) were upgraded to State Bureaus of Statistics, to have consistent structure with NBS. These five states also passed Statistical Edicts as a basis for a statistical framework at the state level. Because of security problems in Bauchi state, it was difficult to implement similar institutional reform there. Overall, there is some evidence of improved timeliness of federal level statistics, but evidence on state-level timeliness is slim. The project also intended to address accessibility by conducting user satisfaction surveys in six participating states, but follow-up survey results are not available. With respect to relevance, two states (Bauchi and Kaduna) did not have detailed state-level statistics at all on the NBS website. Two states had only statistical yearbooks (Edo and Anambra), and the other two (Ondo and Niger) posted a variety of statistics with some remaining backlogs. Interpretability of statistics was improved through trained staff and infrastructure. Efforts were made to have institutional coherence as well as developing a template and compendium of statistical terms. There was no evidence provided on accuracy. While there is evidence that federal-level statistics as well as staff capacity and statistical infrastructure were improved, the evidence that the project produced reliable statistics in the six participating states was limited. The efficacy of the project is therefore rated as Modest. 5. Efficiency: The PAD (p. 35) and ICR (p. 14) explained that the project did not lend itself to economic and financial analysis to measure efficiency. According to the ICR (p14), statistical capacity development could generate small amounts of financial revenue through the sale of statistical publications or data, but this would be far from enough to recover the project cost by itself. Yet, the potential economic gains could be large. Reliable data could lead to evidence-based policy as well as improved accountability and transparency. The child registry could also expand the tax base in the long run. In addition, national statistics could serve as public goods for many different stakeholders, including the private sector for investment. However, these benefits are evidence of sector efficiency that the project would have contributed, but they are hard to quantify and the project itself did not fully achieve its objective. The project was completed within the originally estimated time frame and resource envelope. While the project experienced delays for NPoPC-related activities, such as a transfer of key personnel that triggered suspension of disbursement, there was no extension of the project closing date due to the efforts of the Bank team and NPoPC. Most planned activities were completed within the project's anticipated costs. However, the project did not fully achieve its objective which is a sign of inefficient use of resources. On balance, the efficiency of the project is therefore rated Modest. a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation : Rate Available? Point Value Coverage/Scope* Appraisal No ICR estimate No * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: The relevance of the project's objectives is rated Substantial, and the relevance of design is rated Modest. Achievement of the project development objective is rated Modest. The design of the project focused primarily on capacity development at the federal agencies, while the project development objective was specific to the six participating states. As a result, improved results were observed for the NBS and NPoPC, but there was uneven progress at the state level. Efficiency is rated Modest. These ratings suggest significant shortcomings in the operation’s preparation and implementation, resulting in an Outcome rating of Moderately Unsatisfactory. a. Outcome Rating: Moderately Unsatisfactory 7. Rationale for Risk to Development Outcome Rating: The project built a legal foundation for statistics in five out of the six participating states by approving Statistical Edicts. The government requested additional support to the Bank through an IDA credit for strengthening core social and economic statistics as well as capacity development. Other donors also showed increasing attention to the need for statistical capacity development in Nigeria. With this institutional strengthening and donor support, the improved capacity of human resources and infrastructure is likely to contribute to progress toward the production of reliable statistics. Despite these favorable foundations, the lack of regular monitoring of NSDS and difficulty retaining trained staff remain as a challenge. The recent resignation of the NPoPC chairperson is likely to affect sustainability in the short run. There is high risk to development outcome in Bauchi state because of security concerns. In the medium to long run, there are some risks to sustainable financing by the government. The project was completely financed by the trust fund of the Bank, and federal and state governments need to allocate their own resources not only for their staff salaries, but also for operation and maintenance costs of IT equipment and devices, independent of political interests. It is a positive sign that capital expenditures have increased (NBS: 290 million Nigerian Niaira in 2011 to 1 billion Nigerian Naira in 2013; NPoPC: 290 million Nigerian Naira in 2011 to 1.5 billion Nigerian Naira in 2013), but recurrent expenditures did not increase at the same pace. For instance, recurrent expenditures in NPoPC decreased from 2012 to 2013. While follow-up Bank support for statistical development is under preparation, negligible government financial support for NSDS poses a threat to development outcome. a. Risk to Development Outcome Rating : Significant 8. Assessment of Bank Performance: a. Quality at entry: The Bank team learned lessons from past experience. The project was designed to strengthen statistical communication and institutional reform, which went beyond improvements in data development and management. The time period between concept review and project approval was 1.5 years, but the delays were out of the Bank team's control, and the project still seized the moment to initiate the implementation of the NSDS. The Bank team also made a decision to employ a full-time, in-country statistician to facilitate the project, taking into account the lack of existing expertise and complex institutional arrangements of the statistical agencies in Nigeria. The Bank team also involved other partners and donors, such as through the National Partnership Group. The project's risks and mitigation measures were appropriately spelled out. For instance, the appointment of a procurement consultant was included in the Grant Agreement as a condition of withdrawal of funds to address the weak implementation capacity of NPoPC. However, the project design did not weight activities sufficiently toward federal-level capacity development. The project team confirmed that a previous Bank-funded project had supported the federal level, and that this project intended to support key states. The complex federal system in Nigeria necessitated involvement of federal level statistical agencies, NBS and NPoPC, but there were significant shortcomings, particularly in design of the project's objective and in M&E. There was a lack of clarity on indicators and oversight for monitoring and evaluation (see Section 10a). The project's objective of producing reliable statistics in six participating states was too ambitious for a limited trust fund budget and time frame (see Section 3a). Selecting one state from each geopolitical zone was also ambitious. For instance, there was a security concern in the North East zone (including Bauchi state) before the project's approval. Quality-at-Entry Rating: Moderately Unsatisfactory b. Quality of supervision: The project was well supported by the Bank team. The full-time statistician stationed in Nigeria was instrumental for implementation of project activities. Regular visits to the field facilitated implementation and coordination with multiple stakeholders. There were four implementation support missions during the 2.5 years of project implementation, and the missions were appreciated by the implementing agencies. The Bank Task Team Leader remained the same throughout the project to provide consistent support. However, the project was not restructured to correct weaknesses in design, and a formal mid-term review was not conducted even though the review was anticipated at entry. As a result, design shortcomings largely persisted as challenges during project implementation. The Bank team suspended disbursements for five months in 2013 arising from a change in leadership at NPoPC which brought a serious challenge during implementation. According to the project team, the Bank discussed this issue with oversight institutions such as the Technical Committee. Consultation with the Ministry of Finance and NPoPC did not resolve the issue, and accordingly, the Bank suspended disbursements in accordance with the Grant Agreement. The leadership of the NPoPC was later reinstated by the government. Despite these challenges, this delay was compensated for by the renewed efforts of the NPoPC team with strong support from the procurement team of the Bank. A financial management specialist was also located in Nigeria and provided support when needed. Quality of Supervision Rating : Moderately Satisfactory Overall Bank Performance Rating : Moderately Unsatisfactory 9. Assessment of Borrower Performance: a. Government Performance: The government demonstrated its commitment to improving statistical capacity development by developing the NSDS. The NSDS was prepared through consultation with many different stakeholders. Yet, this strategy was too ambitious, and the government budgetary contribution was not realized at all even though the government planned to contribute 65% of the total cost of NSDS. Also, the monitoring and evaluation system was not in place to monitor the progress of NSDS. For this project, a Technical Committee was established in 2012 at the federal Ministry of Finance to coordinate different states and monitor progress, but the committee did not have its own financial resources to carry out this oversight. All state governments, except for Bauchi state where there were security concerns, passed Statistical Edicts, and state statistical agencies were restructured into Statistical Bureaus of Statistics (SBS), which have consistent roles and functions jointly with the NBS. Yet, government budgetary appropriations were not sufficient, and lack of operating costs for the participating states were a challenge (ICR, p. 43). Government Performance Rating Moderately Unsatisfactory b. Implementing Agency Performance: At the federal level, NBS and NPoPC were the implementing agencies that provided extensive support for the six participating states. NBS was in charge of social and economic statistics, while NPoPC covered demographic statistics. Each agency set up a Project Implementation Task Team to facilitate implementation. NBS and NPoPC organized trainings at the federal level, and each participating state sent their staff to be trained. NBS and NPoPC also procured equipment and infrastructure for all six participating states. In addition to NBS and NPoPC, there were SSAs or SBSs in each state to utilize improved staff capacity and infrastructure to collect data. NBS: The performance of NBS was satisfactory. NBS previously had experience with another Bank-funded project, and therefore had familiarity with the Bank procedures. Initially, some financial management and procurement issues were identified, but these were resolved in a later stage. The commitment of NBS was also strong. NBS had responsibility for managing NPoPC disbursements, which might not have been the optimal arrangement, but overall implementation arrangements were sound, and NBS also took a leading role to coordinate with SSAs or SBSs. NPoPC: Despite mitigation measures to mobilize a procurement consultant before disbursement, activities related to NPoPC were delayed because this was the first NPoPC project supported by the Bank. NPoPC also encountered issues in leadership. A project manager who had received a set of trainings to manage the project was transferred, and subsequently little progress was made on NPoPC activities. However, with the strong commitment of achieving the project's objective and efforts by the renewed team, NPoPC completed nearly all planned activities under the project. SSA/SBS: Each state had an independent statistical institution (SSA, SBS) from NBS and NPoPC. The SSAs or SBSs identified their needs in terms of training, infrastructure, and communications, and NBS and NPoPC compiled those needs and supported a set of activities for each SSA or SBS. According to the project team, Bauchi state faced implementation challenges because of security concerns, and Edo state also had management leadership issues. The other four states, particularly Niger and Ondo, performed well. While the field visits sometimes found that not all equipment was utilized at the state level, ICR (p. 13) stated that all six participating states had high demonstration effects for neighboring SSAs or SBSs. Implementing Agency Performance Rating : Moderately Satisfactory Overall Borrower Performance Rating : Moderately Unsatisfactory 10. M&E Design, Implementation, & Utilization: a. M&E Design: The M&E design was intended to follow the Quality Assurance Framework for official statistics, particularly in terms of accuracy, relevance, timeliness, accessibility, coherence, and interpretability. The project's results framework had time-bound output measures for each component clarifying the frequency of data collection, data sources, and responsibility of the implementing agencies. However, the M&E design left ambiguities. For instance, M&E design did not include measures to monitor actual increase of user demand by project closure. In some cases, baseline values were missing, and target values were not well specified. The sampling framework was also ambiguous. For example, the PAD does not provide clear target values and descriptions of how to measure the indicator on the number of staff with competencies in core areas (proportion of professional statisticians against total workforce in NBS, NPoPC and MDAs). The increase in number of people registering at the pilot registration center and electronic birth/death data captured did not have baseline or target values. The description of indicators highlighted activities of the NBS, but indicators to monitor the progress at NPoPC were weak. b. M&E Implementation: The indicators were monitored beginning with the second ISR, and a baseline user satisfaction survey was conducted. However, a follow-up user satisfaction survey has not yet been analyzed even though data were collected. The indicators were not changed substantively, but the project team modified wordings to fit into project context. Each implementing agency, NBS and NPoPC, monitored indicators under their own responsibility, but there was little evidence that the overall M&E system worked at the federal and state government level. c. M&E Utilization: Statistical publications were disclosed on the websites of NBS and NPoPC for users. NBS also started to disseminate a release calendar for users, though this covers only federal-level statistics. M&E Quality Rating: Modest 11. Other Issues a. Safeguards: No safeguard policy was triggered, and the project was rated as Category “C” (PAD, pp. 5-6). The project did not include any civil works that might cause environmental damage (ICR, p. 10). b. Fiduciary Compliance: Financial Management : The FY14 external audit result was unqualified. While ICR mentioned that “financial reporting is viewed as largely timely and adequate," the ICR does not explicitly articulate this FY14 unqualified audit result nor external audit results of the other years of the project. The project experienced challenges, particularly at the initial stages. For instance, both NBS and NPoPC took time to submit sufficient documents for disbursement to the Federal Project Financial Management Division (FPFMD), which was responsible for ensuring compliance with the financial management requirements of the Bank and the government. The appointment of an internal audit was also slow, and interim financial reports were not submitted in a timely manner, particularly in the initial stages. It also took time to complete an approved financial management procedures manual for each agency. These issues were largely resolved by the time of the Bank's third implementation supervision mission. Procurement: Weak procurement capacity, particularly at NPoPC, was identified at entry, but this issue was addressed by mobilizing a procurement specialist for each implementing agency. There were no major procurement issues reported for NBS or NPoPC (ICR, p. 10). c. Unintended Impacts (positive or negative): None reported. d. Other: 12. Ratings: ICR IEG Review Reason for Disagreement/Comments Outcome: Satisfactory Moderately While Relevance of Objectives is rated Unsatisfactory Substantial, Efficiency and Efficacy are rated Modest. There is insufficient evidence that the six participating states produced reliable statistics. In addition, while the objective aimed to produce reliable statistics in six participating states, many project activities were designed for federal-level institutions such as NBS and NPoPC, resulting in a Modest rating for Relevance of Design. Risk to Development Moderate Significant Lack of regular monitoring and financial Outcome: back-up by the government provides risk to development outcome. Because of the security reason, there is high risk in Bauchi state. Stability of leadership and trained staff also remains as a concern. Bank Performance: Satisfactory Moderately The quality at entry is rated Moderately Unsatisfactory Unsatisfactory because the project's design did not prioritize activities at the state-level statistical agencies, in accordance with the objective. The quality of supervision is rated Moderately Satisfactory. The Bank team did not restructure the project to correct weaknesses at design, though extensive support was provided by the Bank team to address implementation issues. Based on OPCS/IEG Harmonized Evaluation Criteria, when one element of Bank Performance is in the satisfactory range and the other is in the unsatisfactory range, the Outcome rating determines the overall Borrower Performance rating. Borrower Performance : Moderately Moderately Government performance is rated Satisfactory Unsatisfactory Moderately Unsatisfactory because of the lack of financial support and oversight of the M&E system. Implementing Agency performance is rated Moderately Satisfactory. Based on the OPCS/IEG Harmonized Evaluation Criteria, when one element of Bank Performance is in the satisfactory range and the other is in the unsatisfactory range, the Outcome rating determines the overall Borrower Performance rating. Quality of ICR: Satisfactory NOTES: - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 13. Lessons: The ICR (pp. 20-21) identifies 21 lessons, mostly related to project design, arrangement, coordination, implementation, and sustainability. Among these lessons, IEG adapts here three salient lessons and also provides one lesson of its own. For a multi-agency project, clarity of the functions and budget of an oversight committee is important to provide guidance and resources for managing the project . This project identified principal members of a Technical Committee, including senior officials at the Ministry of Finance, NBS and NPoPC, but there is little evidence that this committee provided effective guidance for project implementation. The operating cost to organize this committee, including field visits, was unclear, and this might have partly undermined the activities of this committee. Monitoring reports, which were supposed to be the basis of decisions at the Technical Committee, also did not provide a solid foundation for drawing specific guidance. Sustainability of a project requires its strategy to be supported by government funding . In this case, government support for the NSDS did not materialize as planned. Donor support needs to be considered as a supplementary investment, rather than as a substitute for government budget, and failure to do so can lead to serious consequences in the long-term. In-country Bank experts can strengthen project implementation when a project requires highly technical knowledge for supervision , coordination among multiple agencies , and/or strong support for an implementing agency that is not familiar with Bank procedures . This project's ICR emphasizes that the well-qualified statistician based in the Bank country office facilitated project implementation. The in-country financial management specialist and procurement specialist were also helpful when the implementing agencies, particularly NPoPC, which had no experience with Bank-funded projects, experienced challenges with disbursement and procurement. Clear definitions and scope of project development objectives critically affect evaluability . This project's objective was straightforward, but the definition of key terms, such as “reliable statistics,” was unclear. The ICR also provides information that project developed capacity for the six participating states to produce reliable statistics, but capacity development and actual production of reliable statistics are different, and the statement of objectives requires careful consideration of existing capacity, budget, and duration of the project. 14. Assessment Recommended? Yes No 15. Comments on Quality of ICR: The ICR includes candid views and reasonably comprehensive information on the activities and outcomes at NBS and NPoPC. The lessons learned cover many different aspects of project design and implementation. However, the quality of evidence and analysis is occasionally slim, especially in the discussion of Relevance of Objectives and Design, Efficacy as well as implementation challenges. For instance, information about the performance and outcomes of state-level statistical agencies is slim, which makes it hard to assess achievement of the project's objective. Basic information on the selection process for participating states is also missing. In the fiduciary section, actual audit results are not articulated. a.Quality of ICR Rating : Satisfactory