83051 ANNUAL REPORT YEAR II September 1, 2012 - August 31, 2013 Afghanistan Reconstruction Trust Fund (ARTF) Supervisory Agent International Relief & Development, Inc. Contents Year II Results 5 What is CMORE? 7 Education Quality Improvement Program – Ministry of Education Third 9 Party Monitoring Highlights Ministry of Education - Monitoring Training Center Highlights 11 Irrigation Restoration and Develop Program – Ministry of Energy and 13 Water Third Party Monitoring Highlights Irrigation Restoration and Develop Program – Ministry of Energy and 15 Water Community Monitoring Program Highlights National Emergency Rural Access Program - Ministry of Public Works / 17 Ministry of Rural Rehabilitation and Development Third Party Monitoring Highlights National Solidarity Program – Ministry of Rural Rehabilitation and 19 Development Third Party Monitoring Highlights Impact of Afghanistan Reconstruction Trust Fund Monitoring and 21 Feedback on Government of Afghanistan Ministries Guide to Acronyms ARAP Afghanistan Rural Access Project MoPW Ministry of Public Works ARTF Afghanistan Reconstruction Trust Fund MRRD Ministry of Rural Rehabilitation & Development CDC Community Development Council MTC Monitoring Training Center CM Community Monitoring NERAP National Emergency Rural Access Program EQUIP Education Quality Improvement Program NSP National Solidarity Program GoA Government of Afghanistan O&M Operations and Maintenance IRD International Relief & Development, Inc. QC Quality Control ISD Infrastructure Services Department SA Supervisory Agent IRDP Irrigation Restoration and Develop. Program TPM Third Party Monitoring MoEW Ministry of Energy and Water WB World Bank MoE Ministry of Education Supervisory Agent Office International Relief & Development, Inc. Afghanistan Reconstruction Trust Fund (ARTF) Monitoring Program Kabul Office Near British Embassy, Sherpur, Kabul, Afghanistan Phone Numbers: +93 (0)706.719.955 / +93 (0)790.690.739 Supervisory Agent Contacts: Dr. James K. Weeks Director of Monitoring Services jweeks@ird-artf.org David P. Hinkle Deputy Director of Monitoring Services dhinkle@ird-artf.org Daniel T. Bichanich Chief of Party dbichanich@ird-artf.org Umesh K. Tiwari Deputy Chief of Party utiwari@ird-artf.org Year II Results T he beginning of Year II saw a con nua on of Year I ac vi es including the Third Party Monitoring (TPM) of the Educa on Quality Improvement Program (EQUIP), the Na onal Emergency Rural Access Program (NERAP), and EQUIP MoED MTC 628 the Na onal Solidarity Program (NSP), but also 873 saw significant addi ons to the Afghanistan IRDP Reconstruc on Trust Fund (ARTF) Monitoring 317 Program. The Irriga on Restora on and 3,238 Inspection Reports Development Program (IRDP) was added to the NERAP TPM program and the Supervisory Agent (SA) 840 IRDP branched out to two new ini a ves: a Ministry Training Center (MTC) at the Ministry of Educa on CM (MoE) and a Community Monitoring (CM) program NERAP 146 to supplement IRDP under the Ministry of Energy 434 and Water (MoEW). Under the Year II work plan, the SA planned to submit a total of 2,800 inspec on reports with the breakdown as follows: Chief among the many successes of Year II was that of the Community Monitoring (CM) pilot program. Implemented in Year II and providing  EQUIP: 800 observa ons throughout the second half of the year, the CM program under  MoEd MTC: 400 IRDP was quickly recognized as an invaluable tool for MoEW. The near real-  IRDP: 250 me data from project sites provided the Ministry with the ability to  IRDP CM: 200 effec vely monitor construc on progress remotely with fewer travel costs  NERAP: 350 and security risks. The CM program also provides a tool for verifying  NSP: 800 rec fica on of deficiencies quickly and efficiently. The posi ve feedback and many lessons-learned from this pilot program have led to further As the SA con nued to increase program expansion in Year III. efficiency, the number of delivered reports rose to eventually reach 3,238 for Year II which exceeded Addi onally during Year II, the SA began the the program target by over 15%. The program development of “road maps” for ensuring that peaked during the third quarter with 963 CMORE data and systems are integrated with the inspec on reports completed, prior to a planned exis ng ministry systems. This is cri cal so that decline in the fourth quarter that coincided with ministry databases may be populated with the vast the observance of the Islamic holy month of amounts of informa on collected by the SA and Ramadan and Eid-ul-Fitr. that informa on seamlessly integrates with the systems used in regular opera ons. This task will In general, the programs con nued to show signs con nue through Year I I I as the program of improvement both in terms of the quality of approaches comple on in August of 2014. delivered projects and the strength of the implemen ng ministries in tracking and rec fying devia ons. Specifically, all four programs had above average grades for the second half of Year II, with IRDP having the highest average score. One of the key developments during the year that can be seen as contribu ng to the con nuing program improvements is the submission of weekly devia ons by the SA to the ministries and a monthly review of the overall devia on tracker. This tool was implemented across all programs by the second half of the year and has provided a concise, clear way to manage project devia ons. 5 What is CMORE? C MORE is an integrated set of tools that allows field-based staff to be assigned tasks, collect informa on in the field through smart phones, and electronically send it back to a database where supervisors review the data and generate reports. Addi onally, CMORE provides a means for tracking deficiencies, analyzing trends over me, and organizing regular repor ng. The informa on collected can then be displayed in a variety of ways, including geographical layouts, data tables, and reports. Supervisory Agent Capacity Deficiency Building Summaries Feedback Inspection Monthly & Reports Quarterly Reports 7 Education Quality Improvement Program – Ministry of Education Third Party Monitoring HIGHLIGHTS E EQUIP, under the stewardship of MoE, is tasked with the construc on of educa on facili es for Afghanistan's youth. The SA's EQUIP TPM team completed 873 inspec ons and reports during Year II, over 9% above the target, During the year it became apparent that the ministry could benefit from more regular repor ng of cri cal deficiencies. In order to achieve this, the SA developed and implemented a weekly update of cri cal deficiencies for MoE. This mechanism and inspected a total of 300 schools. allows for more mely rec fica on and a simplified means of tracking problems as they arise. The During Year II, although poor quality workmanship ministry has been proac ve in addressing and failure to follow design on CC (Community deficiencies and the overall quality of construc on Contracted) projects noted, improvement in the should demonstrate this over me. quality of construc on observed under the EQUIP program. Generally, construc on of NCB (Na onal Compe ve Bid) projects rated higher than the projects awarded under CC and projects, where Con nuing in Year III, the SA plans to complete 700 inspec on reports the provincial engineers had more ac ve quality focusing on newly awarded projects. assurance inspec ons, rated high. The SA con nued working closely with MoE during Year II and while the focus was on overall program improvement, specific a en on was given to cri cal items. Examples of the cri cal issues are:  Deficiencies in structural elements such as walls and roofing systems.  Devia ons from the approved scope of work (i.e., a change in design).  Unsafe working condi ons or construc on issues that may result in danger to the beneficiaries. 9 Ministry of Education - Monitoring Training Center HIGHLIGHTS A s a result of Year I findings, documented while working closely with MoE engineers on the EQUIP program, the MTC at the MoEwas implemented in Year II to increase the monitoring skills and capacity of their Infrastructure Services Department (I S D) engineers and to teach them monitoring techniques using the latest tools and technology also used by the SA. Selected MoE engineers received one full month of intensive classroom and prac cal training on construc on monitoring, including use of the CMORE mobile and cloud- based suite of tools. A er comple ng the ini al training in November 2012, MoE engineers worked closely with S A staff to perform inspec ons in the field as well as data processing within the MTC facility in order to strengthen their skills. The SA trainers provided con nuous training comple on. For this metric, students scored throughout the rest of Year II, including tandem 51.47%, on average, over the first two months of evalua ons of MoEd field engineers, quality training. Over the final month of training, this control (QC) checking of previous inspec ons from rose to 95.40%, an amazing improvement to prior months, and evalua ons of MoE office nearly perfect. engineering staff. The Year II class concluded in August and the graduates will con nue performing inspec ons The MoE MTC team completed 403 inspec ons in Year II, the highest number for MoE. Addi onally, a new class of 15 engineers ever completed by a MoE team over a similar period. Overall, including SA will join the program in Year III and three quality control reports, the program completed 628 inspec on reports in Year engineers from the Year II class will assist in the II. training to further build local training capacity and sustainability within the Ministry. While the quan ty of inspec ons is an important figure, the strongest measure of student success is the improvement in the percentage of correct entries each trainee had on inspec on reports from the beginning of the program through 11 Irrigation Restoration and Development Program – Ministry of Energy and Water Third Party Monitoring HIGHLIGHTS W ith the success of three TPM programs in Year I, the World Bank added IRDP to the ARTF Monitoring Program at the start of Year II. The SA completed 317 inspec on reports in Year II, bes ng the target by nearly 6%. Soon a er inclusion in the TPM program IRDP clearly demonstrated its strength as the best performing of the ARTF construc on programs in terms of quality. IRDP had the highest average score for Year II. IRDP projects are generally constructed well and seem to have a high level of buy-in from the beneficiaries. Both are at least par ally a ributable to the importance placed on irriga on projects in the heavily agrarian society of Afghanistan. This importance seems to lead to higher quality projects, increased maintenance, and a longer lifespan on projects. Some of the best prac ces observed from this program include:  Oversight by MoEW and the facilita ng partner, the UN FAO, is ac ve and strong, leading to be er performance by subcontractors.  MoEW and the SA developed a robust system for iden fying, tracking, and correc ng errors. This deficiency tracker has since been implemented across all programs. Key Highlights In the la er part of Year II, the SA incorporated hydrological and meteorological sta ons in the  IRDP canals are regularly constructed to a IRDP work plan at the request of the World Bank. high standard. These inspec ons provided an important assessment of the data collec on sta ons and  The IRDP TPM team and MoEW developed generally the sta ons rated highly. a weekly deviation tracker which is now used for all programs. Going forward, the IRDP TPM program will complete 200 inspec ons in Year III. In the first  Maintenance of canals is very strong and quarter of Year III, the focus will be on inspec on reflects the importance communities place of remaining hydrological and meteorological on irrigation projects. sta ons and then the focus will shi to canals.  Hydrological and meteorological equipment is generally in excellent condition. 13 Irrigation Restoration and Development Program – Ministry of Energy and Water Community Monitoring Program HIGHLIGHTS C ommunity Monitoring (C M) plays a different role than tradi onal TPM in the World Bank's ARTF monitoring program. TPM provides for a wide range of projects to be monitored by an SA engineer while CM provides a high degree of monitoring on individual projects and in real me. The program must be able to track procurement and be flexible enough to add CMs as new projects get underway. While SA TPM monitors thoroughly inspect sites as many as four rec fica on and tracked construc on progress on mes throughout a year, community monitors are a real me basis to allow the MoEW to provide providing individual observa ons on a daily basis oversight of the contracts as necessary. In throughout the life of a project. This near real- me addi on to the quality assurance the community feedback helps ministries to quickly iden fy monitors provided, they also allowed the devia ons and verify proper rec fica on in a cost- communi es to become ac ve stakeholders by effec ve way. In essence, the reach of the communica ng their concerns through the ministries is much wider with a CM program. Community Monitors to the MoEW and the World Bank. Throughout Year II and going into Year III, The first CMs received formal training for one the community monitors helped the Bank verify month at the Champion Technical Training Center the execu on of IRDP projects and provided a in Jalalabad. However, it quickly became apparent means for the Ministry Staff to increase quality that the program would need a more flexible assurance while improving rela onships with the approach to grow beyond the pilot program. As members of the communi es. new CMs join the program, the SA team provides local, on-site, one-on-one training on mobile With the success of the pilot program, the SA is applica ons, construc on standards and safety, expanding the CM program in Year III to include and the project requirements. This shortens the more IRDP canals and poten ally the addi on of training me and provides flexibility to add CMs as EQUIP schools. From a pilot size of 20 CMs, the new projects begin. This change was a result of program will now have as many as 100 CMs in Year one of the lessons learned during this very posi ve III. pilot program. These lessons include: During Year II, the Community Monitoring Program used Community Monitors, selected  The program must be able to track procurement and be flexible with input from the CDC in each beneficiary enough to add CMs as new projects get underway. Tracking community with ongoing canal construc on. procurement and construc on startup is cri cal. This will allow for the Under the guidance of educated supervisory maximum return on the invested training and equipping of community engineers located at the IRD offices in Kabul, monitors by fielding new community monitors early enough to follow Community Monitors in their home communi es, construc on through all phases. IRD, in close coordina on with submi ed regular observa ons through smart MoEW, will iden fy new projects as soon as the procurement process phone technology. The IRD supervisory engineers is complete. That will allow selec on and training of Community then analyzed the observa ons sent by the Monitors so they are available at the start of construc on. Community Monitors, used the observa ons to access the quality of canal construc on and  The training should focus on explaining key aspects of construc on and prepare monthly reports for the World Bank. In use of mobile tools with an emphasis on regular repor ng of addi on to the reports provided to the Bank, the straigh orward, clear observa ons. processed observa ons and devia ons were provided to the MoEW on a weekly basis.  Observa on frequency and requirements should vary by project needs Technical Coordina on mee ngs between IRD rather than being a daily or weekly requirement. and MoEW/ UNFAO were held on monthly basis to discuss the feedback coming from the  Cri cal observa ons shared real- me with the Ministry have the Community Monitors. This process iden fied poten al to be a strong contributor to their quality assurance deficiencies in a mely manner to the MoEW for program. 15 National Emergency Rural Access Program - Ministry of Public Works / Ministry of Rural Rehabilitation and Development Third Party Monitoring HIGHLIGHTS T he rural roads and bridges program, N E R A P/A R A P, is unique in that it is managed by both the Ministry of Public Works (MoP W) and the Ministry of Rural Rehabilita on and Development (MRRD) with responsibili es split between heavier-use secondary roads overseen by MoPW, and more rural ter ary roads administered by MRRD. During Year II, the SA's NERAP TPM team completed 434 inspec ons of both secondary and ter ary road sec ons and bridges, nearly 25% more than the 350 required under the contract. As is common with road construc on in Afghanistan, construc on efforts under NERAP con nued to be challenging in Year II. However, NERAP also saw the greatest improvement on quality over Year II and signs at the end of Year II point to an even stronger roads and bridges program in Year III. Some key highlights from the year include:  MRRD ini ated Opera on and Maintenance The SA works closely with the NERAP ministries (O&M) programs for many of the completed and submits devia on reports on a weekly basis. ter ary roads. These reports have been all-inclusive with minor and major deficiencies noted and reported. To  MoPW and MRRD have greatly increased the help assist the ministries further, the SA will strength and completeness of designs under modify this to focus on major deficiencies in Year the NERAP follow-on program, Afghanistan III. Rural Access Project (ARAP).  MoPW and MRRD have included a centrally- Con nuing in Year III, the SA plans to complete 450 inspec ons of the funded O&M program in ARAP. NERAP follow-on program, ARAP. These inspec ons will focus on new construc on projects as well as the new centrally-funded  Bridge projects nearly always receive the maintenance program to monitor its efficacy in maintaining the vast highest marks and generally score above inventory of roads and bridges under the responsibility of the average. ministries.  Revisited projects, those monitored both early and late in Year II, averaged improvements of nearly a full grade with many falling between 'average' and 'good'. 17 National Solidarity Program – Ministry of Rural Rehabilitation and Development Third Party Monitoring HIGHLIGHTS T he Na onal Solidarity Program (NSP) is unique in that it is a community-based program that sees the villages plan and implement projects. A total of 840 NSP inspec ons were conducted in Year II by the SA's TPM team, 5% higher than the planned 800. These inspec ons demonstrated rela vely strong results for NSP. Even with the long history of NSP, the program con nued to make improvements in Year II of the ARTF Monitoring Program. Some of the notable achievements include:  Distribu on of construc on quality cer ficates to emphasize quality and operability.  Crea on of a micro hydropower O&M manual and training of 1,850 operators.  Crea on and publica on of water system Irriga on nsporta on O&M guidelines with distribu on expected to Tra reach 3,000.  Publica on of the Revision 6 of the NSP Engineering Technical Manuals, which covers construc on and O&M for the ministry's Program Management Unit. ra l Developme upply & Sanit Ru nt rS a te Wa on Key Highlights  Women were consulted on and involved in NSP projects a majority of the time.  Early NSP projects were constructed well Power and quality continues to strengthen in recent years.  All five monitored sectors findings demonstrated high quality construction with good overall grades. 19 Impact of Afghanistan Reconstruction Trust Fund Monitoring and Feedback on Government of Afghanistan Ministries T he SA works closely with the Government of Afghanistan's line ministries responsible for implemen ng each ARTF construc on p r o g ra m . T h e S A c o n d u c t s m o n i t o r i n g inspec ons, analyzes the data, and reviews the Ministry results and trends with the ministries on a Implements monthly basis. In Year II, the SA began sharing key Corrective devia ons with the ministries on a weekly basis to Actions allow for melier and less costly rec fica on. The results of this partnership have improved program delivery by increasing the quality of construc on, the meliness of devia on correc ons, and helped to create many new standards by which each program operates. Project Monthly Inspections Review of Findings Some examples of key developments in Year II include:  Development of weekly devia on reports and monthly upda ng of a devia on tracker for each program.  Increased design quality for projects in the NERAP follow-on program, ARAP.  Crea on of an O&M program for MRRD roads in NERAP and that of a centrally-funded O&M program in ARAP.  Crea on of O&M manuals for micro hydropower projects and water and sanita on projects within NSP, as well as training based on these manuals.  Revisions to designs and specifica ons based on SA feedback.  Distribu on of SA reports to provincial and local ministry en es to improve program delivery.  Increased a en on paid to worker safety, the par cipa on of women in the project process, and other social issues. 21 Afghanistan Reconstruction Trust Fund Monitoring Program World Bank Contract No. 8004370