Executive Summary of Evaluation Name of Evaluation CASA Mid-Term Review Date of Final Report of Evaluation August 2012 Dalberg, Angela Hansen, Naoko Koyama, Veronica Chau Author of Final Report of Evaluation veronica.chau@dalberg.com Date of this Executive Summary July 14, 2014 Author of this Executive Summary Luis Domenech, IFC CDI Department, ldomenech@ifc.org Number of pages of this Executive Summary 5 Modifications from original Executive Summary Paragraphs’ rearrangement. Without content’s modification. Executive Summary Approved for public disclosure by Anna Hidalgo, IFC-ECRIO, AHidalgo@WorldBankGroup.org Background IFC Advisory Services, in accordance with donor agreements and internal monitoring and evaluation requirements, has commissioned Dalberg Global Development Advisors (Dalberg) to conduct an independent mid-term review of the Conflict Affected State in Africa (CASA) Program. CASA’s business model is a comprehensive program consisting of three components: (1) Project Support1, or the facilitation of tailored and coordinated Advisory Service (AS) projects, (2) Funding, or the provision of funds to support the implementation of Advisory Service (AS) projects, and (3) Knowledge Management (KM), or dissemination of IFC tools, lessons learned, and best practices for private sector development in fragile and conflict-affected states (FCAS). CASA’s current business model aims to enhance the delivery of IFC’s Advisory Services in fragile and conflict-affected states (FCAS) as measured by2: (1) an increased number of projects, (2) enhanced project outcomes based on improved relevance to the local context, (3) increased diversity across Business Lines. CASA was initiated by IFC Private Enterprise Partnership for Africa (PEP Africa) in response to the growing focus within the development community and the World Bank Group on private sector development in conflict-affected countries. CASA was launched in 2008 and became active in Central African Republic (CAR), Democratic Republic of Congo (DRC), Liberia and Sierra Leone in 2009. Three additional countries were launched in 2010 – Burundi and Côte d’Ivoire – and in 2011, South Sudan. The mid-term review, takes place at an opportune time, as the Program has almost four years of experience to examine for lessons learned. Key review findings will inform recommendations for adapting CASA’s business model as it moves into its second phase. Objectives 1 Note: In CASA’s initial program proposals and documentation and the Terms of Reference for the review, this component was referred to as “Integration”. For this review it will be referred to as “Project Support” to better describe CASA’s intention. 2 This is based both on a review of CASA program documents (PDSA, donor reports etc.) as well as conversations with members of the initial team which developed the CASA program. Page 1 of 5 The objectives of the review are (1) to inform IFC CASA donors, management, staff, and other key stakeholders on the relevance, additionality, effectiveness and impact, efficiency, and accountability of the Program; and (2) to inform management of lessons learned and potential improvements to service delivery for the remainder of the current Program cycle, as well as for any potential continuation or expansion phases. In addition to structured review criteria three strategic questions were addressed: (1) Was the CASA model a good investment for the donors and for IFC? (2) What are the determining factors for success in CASA countries? (3) Is the available evidence robust enough to answer these questions effectively? Analysis The review covers each program component, all seven countries, and the period from Program inception in mid-2008 until the end of December 2011. Recognizing that CASA serves as a facilitation mechanism and does not directly implement advisory programs; the review focuses on the Program’s performance and does not focus in detail on the performance of the projects it suppo rts. CASA’s performance will be reviewed with regards to the Program’s objectives and targets as stated in the Program’s Approvals (PDSAs), supervision reports (PSRs), and other reports. To the extent that CASA supported projects are reviewed, this will be a portfolio-level analysis of projects’ performance as measured in their PSRs. In all, the review included CASA and Private Enterprise Partnership for Africa’s portfolio (including a desk review of over 85 internal and external documents) and interviews with 91 CASA stakeholders and experts. Conclusions and Recommendations Key Findings Relevance and Additionality CASA is highly relevant based on industry best practices for private sector development in fragile and conflict affected states. By focusing on private sector development (PSD) in FCAS in Africa, CASA addresses one of the most important elements in post-conflict reconstruction (as per the World Development Report 2011). Moreover, according to stakeholder feedback, no other agency is addressing PSD in FCAS as comprehensively as IFC through CASA. While private sector development is critical following a conflict, supporting that development in this context is exceptionally challenging and requires a unique approach. To this end, CASA’s model successfully applies the best practice principles developed by the Donor Committee for Enterprise Development. Our assessment of CASA’s relevance is based on stakeholder perceptions: relevance and additionality criteria received the highest ratings from the majority of stakeholders interviewed.3 A full one hundred percent of IFC Business Line staff rated CASA’s relevance and additionality ‘high’ or ‘very high’, affirming CASA’s unique value to the existing IFC Advisory Services in FCAS in Africa. Other stakeholders also considered CASA’s relevance high; over eighty percent of local stakeholders (i.e., government and private sector counterparts in CASA countries) and over seventy percent of donor agencies interviewed rating CASA’s relevance ‘high’ or ‘very high’. 3 Dalberg analysis of stakeholder interviews. Page 2 of 5 Effectiveness/Impact CASA’s project support and funding components help generate more AS projects and sustain the quality of existing projects. During the review period, the IFC AS portfolio in CASA countries increased in volume while the overall PEP Africa portfolio shrank. This occurred without sacrificing project quality; the CASA portfolio’s overall project rating remained in line with the broader PEP Africa portfolio. It is difficult to isolate CASA Coordinators’ contribution to the evolution of project volume and quality, due to numerous exogenous factors such as country context, donor interest, and most importantly the coinciding contribution of IFC Business Line staff. Nonetheless, feedback from IFC Business Line staff provided evidence that CASA significantly impacted portfolio performance: eighty percent of Business Line staff rated the effectiveness of CASA’s Project Support function ‘high’ or ‘very high’. Closer analysis of project-level feedback from stakeholders also showed a clear link between CASA’s contribution, and project volume and quality. In particular, CASA Coordinators’ local presence and close engagement with local stakeholders were highly valued by stakeholders, leading to improved quality ratings for some projects. CASA’s funding through the Multi-Donor Trust Fund also played a key role in generating projects. The availability of funding allowed many projects to start; the flexibility of that funding allowed disbursements to be made quickly to accommodate small windows of opportunity. CASA has helped increase diversity of projects implemented across Business Lines but has limited ability to adapt standard products to suit FCAS needs. The CASA portfolio has shifted from a heavy Investment Climate (IC) orientation toward a more diverse structure. CASA Coordinators identified the need to go beyond standard product offerings, and while CASA funding did enable them to do so, a few challenges became apparent. First, CASA faces two institutional barriers that limit its ability to integrate across Business Lines: (1) lack of institutional incentives for Business Lines to engage in FCAS; and (2) a strong culture of business as usual. Second, CASA Coordinators have limited time to devote to strategic planning and innovation; this is partly because Coordinators are generally overstretched, but also because they appear to lack strategic direction and incentives to plan in accordance with such a direction. On the Project Support Continuum CASA is not well structured to play an innovation role. CASA Coordinators are highly appreciated by IFC staff, donors, and local stakeholders for the role they play in the day to day coordination and support of project delivery. However, Advisory Staff are concerned that Coordinators do not have the depth of technical expertise and understanding of the Business Line products to effectively engage in developing tailored products. CASA’s current structure is anchored in country-specific coordination and expertise which limits the potential for CASA to have an institutional influence. It is Dalberg’s opinion that this is best addressed through adapting the structure to include Business Line Champions which will give CASA a voice and legitimacy within the Business Lines where the key product decisions are made. Despite the delivery of several concrete outputs, CASA has not fully met stakeholder expectations relating to Knowledge Management. Stakeholders widely expressed interest in learning from CASA’s experience, yet CASA has been relatively inactive and/or irregular in publishing its lessons learned and best practices. CASA’s KM activities have included the development of the Conflict Analysis Tool, a FCAS- Page 3 of 5 focused M&E methodology, a training course for operational staff working in the CASA context, and a FCAS Practice Group for IFC staff that serves as a global platform for knowledge sharing. However, many stakeholders are not actively engaged with these efforts and outputs, as they may or may not meet their specific needs. As a result perceive CASA as being inactive. CASA’s effectiveness varies across countries. Due to differences in program duration, and complexity of individual country dynamics, it is difficult to draw general conclusions on success factors. Nonetheless, the Dalberg Review Team identified some country-specific and Coordinator-specific factors that may affect CASA’s effectiveness in certain countries. Country-specific factors include political stability, private sector viability, and donor support. Coordinator-specific factors, based on stakeholder feedback, are the quality of the Coordinators’ relationships with local stakeholders (usually influenced by their length of stay in country) and their ability to navigate IFC’s internal systems. Both sets of factors enhance the potential for CASA’s/IFC’s success. Efficiency and Accountability CASA is operating efficiently toward its program and funding targets. Using only 60 percent of its budget, CASA has achieved or surpassed most of its program targets well within their time frames.4 Additionally, CASA’s initial investment in building a field-based platform via its Coordinators attracted additional bilateral funding of US$ 15 million for project expansion. CASA’s human resource approach may not ensure the business model’s sustainability. The Program needs to ensure that the role of CASA Coordinator is attractive enough to ensure recruiting success and rewarding enough to motivate over the long term. During its first phase, CASA had a clear strategy for recruiting Coordinators that met the Program’s specific capability needs; specifically it found IFC staff with a combination of country context exposure and knowledge of AS offerings, which were willing and able to experiment with the CASA model. While this short-term strategy was largely successful, a longer term strategy for recruitment and retention is yet to be defined. In order for CASA to continue attracting staff, the career trajectory of Coordinators needs to be defined. Retaining Coordinators who have the right expertise and experience is critical, as the primary determinant of the Program’s success is the effectiveness of its Coordinators. Strategic Recommendations Business model recommendation CASA has achieved significant results to date, but more may have been possible if the program had engaged with the Business Lines more effectively. To improve CASA’s collaboration with the Business Lines, the current “country-level” business model should be expanded to include CASA champions within each Business Line. These champions will facilitate greater learning from CASA’s experience, give CASA access to deeper technical/product insight, and build the key relationships needed to enhance effectiveness. 4 According to CASA PSR FY12Q2, CASA has spent US$5.7 million out of the total budget of US$9.6 million for all program expenses other than project funding budget from its inception to FY12Q2. Page 4 of 5 Component recommendations Project Support: The primary proposal for improvement of this component relates to the business model, as described above, however additional recommendations include: • Establish a dedicated budget for hiring short-term consultants (STCs) and support staff as needed, in order to alleviate the capacity constraints on CASA Coordinators. This support would enable CASA Coordinators to spend more time developing strategies for Project Support. • Regularly update and ensure consistent development of Country Engagement Strategies that detail the processes for identifying, designing and implementing projects. Funding: Overall, CASA’s funding should remain flexible, but some recommendations for improving the funding structure include: • Formalize CASA’s resource allocation (project funding and staff) to 1) potential future CASA countries and 2) new products tailored for the CASA context. Knowledge Management: CASA needs to invest significantly to enhance the KM component, building on the initial “tool development” phase and strengthening its role as a thought leader in FCAS. Accomplishing this requires hiring a full-time KM Coordinator who works to provide key stakeholders with CASA expertise and insights. Cross-cutting recommendations: In addition to the recommendations for the business model and individual program components, overarching recommendations include: • Scale-up foot print by establishing CASA programs in new countries. • Advocate for wider adoption of the CASA model within the WBG and push for the institutional flexibility required for the model’s effectiveness. • Improve M&E and finance support functions to increase accountability and enable fact-based strategic planning. Instituting these recommendations will likely increase CASA’s reach and impact. Copyright and Legal Disclaimer The material in this publication is copyrighted. IFC encourages the dissemination of the content for educational purposes. Content from this publication may be used freely without prior permission, provided that clear attribution is given to IFC and that content is not used for commercial purposes The findings, interpretations, views, and conclusions expressed herein are those of the authors and do not necessarily reflect the views of the Executive Directors of the International Finance Corporation or of the World Bank or the governments they represent. The views expressed in this publication are those of its authors and do not necessarily reflect the views of the wider institution. Some of the information used in this document may come from publicly available sources such as company websites and publications. The Lessons of Experience series does not represent a commitment by IFC to require projects it finances to take certain or all of the actions specified in this publication. Instead, any issues arising in an IFC-financed project will be evaluated and addressed in the context of the particular circumstances of the project. Page 5 of 5