ICRR 14205 Report Number : ICRR14205 IEG ICR Review Independent Evaluation Group 1. Project Data : Date Posted : 11/07/2013 Country : India Is this Review for a Programmatic Series? Yes No How many operations were planned for the 2 series? How many were approved? 1 Series ID : S108258 First Project ID : P108258 Appraisal Actual Project Name : E-delivery Of Public US$M ): Project Costs (US$M): 150 150 Services L/C Number : Loan/ US$M ): Loan /Credit (US$M): 150 150 Sector Board : Public Sector US$M ): Cofinancing (US$M): 0 0 Governance Cofinanciers : Board Approval Date : 03/31/2011 Closing Date : 06/30/2012 06/30/2012 Sector (s): Central government administration (41%); Information technology (33%); Sub-national government administration (19%); Banking (7%) Theme (s): Managing for development results (70%); Other accountability/anti-corruption (20%); Other public sector governance (10%) Evaluator : Panel Reviewer : ICR Review Group : Coordinator : Jiro Tominaga Iradj A. Alikhani Christopher D. Gerrard IEGPS2 2. Project Objectives and Components: a. Objectives: The development objective of the Development Policy Lending (DPL) series was increasing access to online services by citizens in their localities (Program Document (PD), paragraph 52). The DPL was to accomplish this objective through supporting policy and institutional actions in two pillars : Higher emphasis on coordination . This will contribute to government's "push" for providing more services online; and Increased outreach to citizens which will "pull" a greater number of services online (PD, paragraph 52). This DPL was the first one of a series of two single tranche DPLs . The government decided not to request the second loan in May 2012. Based on the guidance note on programmatic DPLs in the ICR guidelines issued by the Bank, this ICR Review assesses the achievements of this loan against the objectives set for the entire program. b. If this is a single DPL operation (not part of a series), were the project objectives/ key associated outcome targets revised during implementation? No c. Policy Areas: There were eight policy areas clustered under two pillars . Seven of the eight policy areas had seven prior actions. Six policy areas had seven indicative triggers for the second DPL (PD, pp. 18-26). Pillar 1. Higher emphasis on coordination 1.1. Strengthen states' institutions in e -governance : Slow implementation of the National e-Governance Plan (NeGP) was of concern to the government . The government's e-readiness assessments indicated uneven capacity in states and departments . The prior action in this area was the development of a comprehensive policy for the institutional strengthening program and commencement of its implementation by assisting state governments in preparing and implementing e -governance projects. The indicative trigger for DPL 2 was the approval of a policy for public -private partnerships in e-governance projects and issuance of templates for model concession agreements. 1.2. Development of technical standards : Technical standards help ensure sharing of information and interoperability of data across e -governance applications. The prior action in this area called for the government to adopt a policy for the use of open standards in e -governance. The indicative trigger for DPL 2 was the government adopting technology standards including for interoperability and security . 1.3. Improve Inter -Agency coordination and monitoring of e -governance : The PD (paragraph 66) noted the need for a high degree of coordination among ministries and agencies at different levels of government for the success of NeGP. It also emphasized the importance of accurate and up -to-date performance information on e-governance activities. The prior action in this area called for the inter -ministerial committee of senior officials to adopt a comprehensive framework for monitoring the implementation of the NeGP . Pillar 2. Increased outreach to citizens 2.1. Improve access to services by using mobile platforms and increasing the pace of internet penetration : The government's aim of providing "web-enabled, anytime, anywhere access " to information and e-services was found to be constrained by limited access to broadband services, especially in rural areas . The prior action in this area was for the government to adopt a policy framework for the delivery of basic financial services using mobile phones. The indicative triggers for DPL 2 were for the government to (i) approve the National Broadband Plan to establish broadband in all panchayats, and (ii) adopt a policy to facilitate the setting up of an additional 150,000 broadband-enabled service access points in panchayats . 2.2. Facilitate increased participation of users in design and evaluation of e -governance projects : The government aimed to make e-services "user-centric" by increasing citizens' participation in the design and evaluation of Mission Mode Projects . The prior action in this area called for the government to strengthen its e-governance impact assessment strategy and to begin implementation . The indicative trigger for DPL 2 was for the government to adopt administrative guidelines that incorporated comprehensive stakeholder involvement and performance measurements. 2.3. Improve service orientation of government processes and officials : User surveys revealed that computerization of existing manual processes through e -services was not having a substantial impact on customer convenience, cost reduction, waiting time, and corruption . Re-engineering of existing processes and a new orientation of public officials emphasizing customer service, knowledge sharing, cooperation, and transformation of processes were found to be important in this context . The prior action in this area was for the government to develop and begin implementation of a reorientation program for state government policy makers and officials in e-governance. The indicative trigger for DPL 2 was for the government to begin project -level implementation of the process re-engineering guidelines, and common platforms and components . 2.4. Electronic Service Delivery Law : The Program Document notes that there needed to be a stronger legislative foundation for the major transformation of service delivery anticipated through NeGP, to commit government agencies to the necessary reforms and to establish a common framework for the NeGP . There was no prior action for this policy area . The indicative trigger for DPL 2 included a draft Electronic Service Delivery bill to be formulated and submitted to the Union Cabinet . 2.5. Uniform and predictable verification of e -service users : To deliver electronic services, various types of identification need to be automated . A national verification framework was considered to be required to verify the identity of the online service user . The prior action in this area included the government to issue guidelines for use of digital signatures in e-governance. There was no indicative trigger for DPL 2 in this policy area. d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: The loan of $150 million was appraised in February 2011, approved by the Board on March 31, 2011, and became effective on July 5, 2011. The loan was disbursed in one tranche on December 13, 2011, and closed on June 30, 2012 as scheduled. 3. Relevance of Objectives & Design: a. Relevance of Objectives: The program objective to increase citizens' access to online services was relevant to the third pillar of the FY09-12 Country Strategy, "Increasing the effectiveness of service delivery ." The country strategy emphasizes the focus on the creation of governance and institutional arrangements that promote an enabling environment for results (including through demand-side accountability mechanisms such as beneficiary and civil society involvement), strengthening capacity for publicly provided services, and enhancing private sector participation . This program includes all of these elements in its design . The support for e-governance is included as a theme that cuts across the three engagement areas in the new FY 13-17 Country Partnership Strategy (e.g. paragraph 63). The relevance of objective is rated substantial . b. Relevance of Design: With regard to the policy matrix and the causal linkages between policy actions, triggers, indicators and end-of-series outcomes, reasonable links are established in most areas . For example, the development of technical standards for e-governance (Policy Area 1.2) links well with the intended outcome, which is to improve reliability and inter-operability of government systems . The outcome indicator for this policy area requires both the adoption of standards and certification of systems' compliance with approved standards, covering both the development and enforcement of the standards . The causal chain in Policy Area 2.1 is quite direct -- the selected indicator measures the improvement in access to electronic services through the increase in the number of broadband-enabled connections nationwide . In some areas, particularly those that involve improvement in the quality of services, the end -of-series outcomes and corresponding indicators have slight gaps . For example, Policy Area 2.3 aims to improve service orientation of government processes and officials . Business process re-engineering is an important action in this area and the results indicators monitor the number of government processes re -engineered. However, it is not clear how the Government is ensuring the quality of re -engineering undertaken in the relevant government agencies (center and state). Another part of the intended outcome in Policy Area 2.3 was to enhance the service orientation of appointed and elected officials . The indicator monitored the number of states whose officials have completed reorientation or leadership programs . This is likely related to the attainment of the intended outcomes, but it does not fully capture the qualitative improvement aimed for in these policy actions . These qualitative changes require time before it becomes visible, but the results framework within a DPL program/series may provide too short a timeframe to monitor the progress . Thus, part of the challenge in the design appears to stem from the choice of a DPL as the instrument to support this program. The ICR indicates a somewhat mixed picture about this . It argues that the DPL was a well-suited instrument to address the need for a broad national policy framework for coordination between central ministries as well as between the center and state -level institutions (paragraph 38). At the same time, it points out that the suitability of the DPL instrument may not have been adequately vetted during the preparation period due to the lack of time (paragraph 70). This review agrees with this mixed assessment given the diverse set of challenges this loan tried to address. The DPL was likely useful for addressing the gaps in the broad policy framework identified through sustained dialogue between the Government and the Bank . However, many of the policy areas in the program aimed to achieve institutional changes, involving changes in practices and incentive mechanisms in government agencies . Institutional challenges pointed out in the Program Document included the need to address capacity constraints at the agency level to develop, implement and monitor the activities; develop indices and ways to monitor and evaluate progress of activities implemented by agencies at different levels of government; increase user participation in design and evaluation of e -government projects; and improve government processes and service orientation of officials . The task team seems to have provided on-demand, complementary analytical and technical assistance apart from the DPL (ICR: paragraph 75) and made some positive contributions . But addressing these institutional challenges tend to require sustained effort over the medium term. Based on these considerations, the relevance of design is rated modest . 4. Achievement of Objectives (Efficacy): All prior actions for the release of the tranche were met prior to Board presentation . Those policy actions included for DPL 1, but not included in prior actions, were steps toward achieving the triggers for DPL 2. Most of the indicative triggers for DPL 2 were met (thus, the steps to be taken to achieve these triggers were also met ). Achievements in each of the eight policy areas were as follows . Pillar 1. Higher emphasis on coordination 1.1. Strengthen states' institutions in e -governance : The trigger for DPL 2 has been achieved. However, it is not clear whether the other indicator, 180 e-services in low-income states, was met as the data is not reported in ICR. Similarly, whether or not the policy action envisaged for originally planned DPL 2 -- the Department of Information Technology to establish project development support fund and Expert Cell on PPPs for e-government projects -- was achieved is not known. Nevertheless, the end of series outcome indicators --1,500 e-services and 227 e-services through PPP mode -- were either exceeded or almost met as of September 2012 (1,776 e-services; 225 through PPP mode). 1.2. Development of technical standards : The indicative trigger was not fully met -- the policy on interoperability and security in e-governance had not been adopted at the time of ICR writing . The ICR is not clear whether the policy action for planned DPL 2 (Government to adopt a framework for certifying systems' compliance with approved technical standards ) was taken. However, the end-of-series indicator reported in the ICR requires certification for compliance with the technical standards (targeted for 150 e-services), so it is likely that the intended step was made. The end-of-series target was substantially met with 143 e-services offered through certified systems. 1.3. Improve Inter -Agency coordination and monitoring of e -governance : As well as the prior action, the policy action planned for DPL 2 was taken -- the Government is utilizing the M&E framework to improve monitoring and coordination of e-governance through Program Management Information System, Monthly Progress Report, and Prime Minister-appointed Expert Group on NeGP. The end-of-series indicator, 250 e-services monitored was substantially met as 246 e-services were monitored at the time of ICR writing . Pillar 2. Increased outreach to citizens 2.1. Improve access to services by using mobile platforms and increasing the pace of internet penetration : The first indicative trigger was met as the National Telecom Policy 2012 was issued. The second trigger needed slight modification. Instead of expanding Service Access Points, the government decided to establish 150,000 Government Access Points at the village level . The policy action for the planned second DPL was for the Government to adopt a policy for delivery of services on the mobile platform . The ICR reports that a framework for mobile governance has been developed to facilitate the use of mobile phones and related mobile-applications for access to public services, especially in rural areas, despite the cancellation of DPL 2. The first end-of-series indicator was not met given the delay in 3G service expansion due to the irregularities identified in the 3G spectrum auction process (ICR, paragraph 43). As of September 2012, there were 15.08 million broadband subscriptions . The second indicator, 250,000 broadband-enabled Service Access Points established in panchayats was not met . 95,649 Common Service Centers were rolled out with 80,733 connected. The third indicator, 100 services delivered using the mobile platform seems to have been met as the ICR reports 113 e-services operational. 2.2. Facilitate increased participation of users in design and evaluation of e -governance projects : The indicative trigger was met. The end-of-series indicator, 200 e-services incorporating user perspectives, was not met. At the writing of the ICR, 50 services under 5 projects would have incorporated user participation in design and impact assessment in the financial year . 2.3. Improve service orientation of government processes and officials : The indicative trigger to begin project-level implementation of the process re -engineering guidelines, and common platforms component, was essentially met according to the ICR . Business process re-engineering formed an integral part of the NeGP Operational Guidelines issued. According to the Guidelines, the business process re -engineering and change management plans need to be defined in detail in individual project reports . In addition, the NeGP program guidelines provide for setting up of Dedicated Project Teams for each mission mode project . Under the guidelines, a separate group under a Process Manager and domain specialists is to be set up to address the business process re-engineering, change management, and legal and process quality issues . The first end-of-series indicator, 752 e-services delivered through re -engineered processes was not met as there were 246 services at the time of ICR writing (additional 200 services were expected to have been delivered by the end of the fiscal year ). The second indicator, officials of 35 states and union territories complete re-orientation/ leadership program, has presumably not been met as the achievement reported is that 20 states participated in the Specialized Training on e -governance program. 2.4. Electronic Service Delivery Law : The trigger for DPL 2 was met. At the time of the ICR writing, the bill had been introduced in the Lok Sabha and referred to the Parliament Standing Committee on IT, which tabled its report to Lok Sabha in August 2012. This policy area does not have an end -of-series outcome. 2.5. Uniform and predictable verification of e -service users : There was no indicative trigger for DPL 2 in this policy area. The policy action intended for DPL 2 -- for the government to begin verifying users of e -services -- has been met. The end-of-series outcome indicator -- that users could be verified by service delivery systems anywhere and anytime for 50 e-services -- was not met. There were 18 services meeting this requirement . In sum, out of twelve end-of-series indicators (including those for low-income states and PPP mode for Policy Area 1.1), five were either met or almost met at the time when the DPL series closed . The results among the indicators in Pillar 1, which addresses the challenges among the supply side of e -governance services (e.g. service development and institutional coordination ), seem to have had more success than the other pillar (increased outreach -- the demand side). The overarching development objective was to increase access to online services by citizens in their localities . On balance, it is likely that the number of e -services and access points have increased, but it is not known how the usage of these services by citizens in their localities is progressing. The ICR and the third party impact assessments referenced in the ICR indicate that for activities in the areas where the end-of-series indicators were not fully met, implementation appears to be ongoing and objectives are likely to be reached. Some additional measures have been taken after the DPL was closed, building on the actions and analyses made during this DPL implementation . Considering these points, the efficacy of this loan is rated substantial . 5. Efficiency (not applicable to DPLs): 6. Outcome: This loan addressed important challenges of enhancing access to and effectiveness of public service delivery . It supported several policy actions to increase citizen access to online services . Institutional changes that were found to be critical in increasing citizen access to e -governance services require sustained efforts over the medium term. The team provided technical advice and analytical inputs prior to and during DPL implementation . Dialogue between the government and the Bank in increasing the effectiveness of NeGP continued after this DPL closed. Based on the assessment on relevance and efficacy, the overall outcome rating of this loan is rated moderately satisfactory . a. Outcome Rating : Moderately Satisfactory 7. Rationale for Risk to Development Outcome Rating: The NeGP is a priority program of the Government of India . As such, it is likely that the program receives substantial attention from the government with or without Bank involvement through the DPL series . The ICR also reports a considerable level of commitment by the government as demonstrated in its overall investment of more than US$12.6 billion in creating citizen-centered service delivery systems . However, as noted in the ICR, the challenge of adhering to the NeGP concept of centralized initiative and decentralized implementation is substantial. It requires effective coordination at the center and strong implementation capacity in the field . Maintaining the momentum for program implementation across the country is challenging . Another issue is whether re-engineering will result in a less bureaucratic, simpler and efficient system . Based on these considerations, the overall risk to development outcome is rated modest . a. Risk to Development Outcome Rating : Moderate 8. Assessment of Bank Performance: a. Quality at entry: By the time of project preparation, the Bank had been in dialogue with the government for six years . The Bank had been interacting with the counterparts over many years, providing inputs in various areas including the institutional arrangement and the monitoring and evaluation system . Technical support provided prior to and during the preparation phase seems to have been recognized by the government counterparts (ICR, paragraph 66). According to the task team and the ICR, the Bank's advice in strengthening the monitoring and evaluation system for the NeGP has been particularly useful . At present, the government and the Bank continue to engage in dialogue on issues related to e -governance and NeGP. Discussions on possible state-level operations are also ongoing . These developments show the government's willingness to maintain access to Bank's expertise in this area . However, as noted in the previous section, the choice of instrument may not have been the most suitable for addressing key challenges identified for this operation, due partly to the lack of time for preparation. Balancing these aspects, the quality at entry is rated moderately satisfactory . at-Entry Rating : Quality -at- Moderately Satisfactory b. Quality of supervision: During the implementation phase, the Bank provided a range of ad hoc, on -demand technical support (ICR: paragraph 75), playing the role of technical partner for the government in implementing the NeGP . However, the ICR also points out that the Bank could have been more proactive in helping the government advance Mission Mode project implementation, develop strategy for the NeGP in low -income states, and institutionalize tracking and reporting of DPL progress . Based on available project records, the Bank conducted one formal supervision mission . The focus of interaction with the government during much of the period was on whether the government would like to proceed with the second DPL . The quality of supervision is rated moderately satisfactory . Quality of Supervision Rating : Moderately Satisfactory Overall Bank Performance Rating : Moderately Satisfactory 9. Assessment of Borrower Performance: a. Government Performance: The government has been highly committed to successful implementation of the NeGP and has devoted substantial financial resources to the program . Institutional arrangements have been established for this highly ambitious program. The challenge remains to be coordination with states, particularly in helping lower -income states accelerate program implementation . Progress seems to have been made with various actions supported by this loan such as the interoperability frameworks and digital signature law . Although the second loan in the DPL series was cancelled largely due to the IBRD lending limit, the government sustained the progress in the program. Government Performance Rating : Moderately Satisfactory b. Implementing Agency Performance: The implementing agency at the federal level, Department of Electronics and Information Technology (DeitY), has been instrumental in developing and implementing the NeGP as a whole and the program supported by this DPL. It deployed staff to design, develop and accelerate the NeGP implementation, serving as technical resources for various departments . The program implementation for this DPL did not encounter major obstacles. Implementing Agency Performance Rating : Satisfactory Overall Borrower Performance Rating : Moderately Satisfactory 10. M&E Design, Implementation, & Utilization: a. M&E Design: The NeGP has created an elaborate institutional framework for implementation monitoring . At the center, the Prime Minister's Committee on NeGP was established to provide leadership, prescribe deliverables and milestones, and monitor the overall implementation of NeGP . DeitY was tasked with the oversight function associated with monitoring and evaluation of NeGP implementation at state and central levels including infrastructure and technical components . In the states, an Apex Committee headed by the Chief Secretary provided implementation oversight and inter -departmental coordination. b. M&E Implementation: The Program Management Information System (PMIS), which has been made available online, is being used to track performance of Mission Mode Projects, infrastructure development, and other critical aspects of the NeGP . Monthly Progress Reports (MPR) includes review and analysis of MMP performance using standardized templates. Similarly, the status of core infrastructure is also reported through the MPR . The Prime Minister has appointed a NeGP Expert Group which reviews each of the MMPs in order to identify critical operational, legal, financial and institutional issues and bottlenecks . c. M&E Utilization: The ICR indicates that monitoring reports are available in central and state departments involved in NeGP . However, there is no information available in the ICR about how these monitoring reports have improved decision making by the relevant departments . It is also not clear how the quality of e -services are being monitored. M&E Quality Rating : Substantial 11. Other Issues a. Safeguards: There are no issues reported with respect to compliance to the Bank's safeguards policy . b. Fiduciary Compliance: There are no issues reported with respect to fiduciary compliance . c. Unintended Impacts (positive or negative): No additional impacts reported. d. Other: 12. 12. Ratings : ICR IEG Review Reason for Disagreement /Comments Outcome : Moderately Moderately Satisfactory Satisfactory Risk to Development Moderate Moderate Outcome : Bank Performance : Moderately Moderately Satisfactory Satisfactory Borrower Performance : Moderately Moderately Satisfactory Satisfactory Quality of ICR : Satisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 13. Lessons: The ICR includes useful lessons, particularly regarding the need for creative instruments and approaches to support the Government of India . This is relevant for efforts to support electronic governance as they involve long-term engagement to help institutional strengthening and short -term actions to establish inter-sectoral policy frameworks. The new lending instrument, Program-for-Results, might be able to address some of the challenges, but this review agrees with the ICR that an innovative and creative approach is needed for engaging with a sophisticated client on issues that are as complex as e -governance. 14. Assessment Recommended? Yes No Why? This DPL supports an ambitious program by the Government of India to leverage the information and communication technologies for effective delivery of public services . There have been a number of projects with similar objectives in several countries, for example, Sri Lanka and Ghana . It might be useful to undertake more in-depth analyses of several operations focused on electronic delivery of public services . An important aspect of such an assessment would be to examine whether these operations helped increase the transparency, accountability and accessibility of public services to citizens . The efficacy of DPL versus investment lending could also be reviewed. Similarly, the issue of the adequacy of the depth and breadth of re -engineering of processes in India and their impact on service delivery could be assessed in more detail . Finally, improvement in the quality of public services can only be assessed some time after project closing and a PPAR would be an ideal instrument to for such an assessment . 15. Comments on Quality of ICR: Given that the ICR covers program areas widely and provides useful information, the ICR quality is rated satisfactory . However, in some areas, it could have strengthened its analysis of outcomes achieved and opportunities missed. In particular, further analyses would have been useful on some of the questions it raises including (a) the potential benefit from more detailed assessment regarding the choice of lending instrument and (b) the need for further proactivity in Bank supervision . Additional descriptions on how the policy actions supported by the DPL helped meet the intended results would have been useful, too . a.Quality of ICR Rating : Satisfactory