AUGUST 2011 64872 ABOUT THE AUTHORS CHRISTOPHER WALSH is the Communications Officer for the Water and Sanitation Program A Roadmap for Tracking Knowledge Results – (www.wsp.org), a global partnership administered by the Experiences and Challenges World Bank, where he leads a global team of communications specialists. He was previously the From fostering expertise to packaging and sharing lessons communications specialist with the energy unit of the World Bank’s Africa region, and has also from the field, the knowledge work of the World Bank managed communications for Standard & Poor’s. Group takes many shapes. But unlike lending operations, YEHUDE SIMON is the WSP’s Communications knowledge work is not always subject to the same Officer for Latin America and the Caribbean Region since results monitoring processes. For example, the publishing 2009. He has previously worked for the Communications Unit at Save the Children: Latin of a report has often been used as the only indicator of America, and as a consultant for Johns Hopkins University a successful knowledge product. During a program-wide Communications Center in Peru. SHERYL SILVERMAN effort to develop and implement a results-based management has been a member of the WSP Communications team since 2008. framework for its technical assistance work, the Water and Sanitation Program (WSP), a During that time she has focused primarily on Web-based tools. multi-donor partnership administered by the World Bank, discovered that applying the same SARA SULTAN is a Communications and Knowledge Management framework to its knowledge work helped demonstrate the value of knowledge by drawing Consultant in the WSP Communications Team. She is a connection between knowledge inputs, outputs, outcomes, and impact. This SmartLesson enthusiastic about social collaboration, knowledge describes lessons from a technical assistance program’s experience applying a results exchange and their various dynamics. framework to knowledge work. In addition to demonstrating value, this helped better APPROVING MANAGER Jacqueline Devine, Acting define different stages of knowledge work, measure progress, and better manage program Manager The World Bank Water and Sanitation Program communications and knowledge management. The authors would like to acknowledge Amy Grossman for also contributing to this SmartLesson.  Background Lessons Learned In 2008, WSP began to develop a program- 1) Incorporate communications and knowl- wide results framework to articulate the edge work early in the design of a results results from the technical assistance provided framework. to the 24 countries where it works. Central to WSP’s mandate is providing water and After delving into the development of the sanitation knowledge and thought leadership knowledge component of the results to sector constituents. Given how critical framework, staff identified many activities harnessing and delivering knowledge is for related to communications that naturally fit WSP, a component was included in the results into the knowledge work of the program. For framework to explicate the outputs and example, one of the primary roles of indicators that support the program’s communications staff was to disseminate knowledge work. The following are some of knowledge products to external audiences. the key lessons learned from WSP’s experience: As the framework developed, it became SMARTLESSONS — AUGUST 2011 1 evident that communications and knowledge management are not, and should not be independent of one another. Pros and Cons of Using Excel vs. Web Tools Rather, for a comprehensive approach, it was necessary to position one as a function of the other. To this end WSP identified a knowledge objective - to strengthen the sector knowledge and advocacy to improve water and sanitation services for poor people - and then identified four outputs that supported this objective: knowledge generation, dissemination, evaluation, and advocacy (See Figure 1). Although this component of the framework gained traction among management, it was challenging at first to gain program- wide acceptance because its development had occurred separately from that of WSP’s overall results framework. 2) The framework is evolutionary and iterative. 3) Learn to walk before you run. Through discussions and brainstorming with staff from each Having the right tools matters but start with the basics. region, WSP identified various indicators of progress in the Once the initial results framework was developed, WSP four outputs. Some of the indicators track purely turned to tracking the indicators, which were created in quantitative data, such as the number of visitors to WSP’s consultation with staff. A results “tracker� was developed web site, the number of views on WSP’s YouTube channel, or as an Excel spreadsheet (see Figure 2) and staff time was the number of “re-tweets� from WSP’s Twitter stream. dedicated to track these indicators. Other indicators were a combination of quantitative and qualitative measures, such as “number of sector interventions WSP staff learned that just as the framework is an iterative identified that use or were influenced by WSP knowledge.� document, the tracking mechanism is as well. Complicated Having identified the four outputs, WSP began a soft rollout and expensive websites or instruments were not necessary to of the results framework. It was clear as implementation get started. Keeping it simple allows for enhancements to be began that the results framework would continue to evolve made until it is ready for migration to a more robust platform. as staff began using it. Not only did the framework continue to evolve, but it was also adaptable for regions, if warranted. Once staff began adding information to the tracker consistently, For example, in Latin America, a results champion worked maintaining it in Excel grew challenging in terms of time spent with staff to develop a manageable set of indicators relevant entering the information. At this stage, it may prove helpful to specifically for that region. With a results framework that explore other options such as an online database that staff can better reflected their work, staff there almost immediately access to input data and monitor progress. began regular contributions to tracking indicators. 4) To demonstrate knowledge-sharing results, it’s The results framework also needed to evolve as new necessary to invest in tracking. technologies and methods were implemented in WSP. For example, Twitter indicators were absent from the first Investing in a “results framework� does not necessarily version of the framework, but as WSP began to increase its require financial resources. Time is perhaps the most focus on social media, it has added five indicators related to valuable investment WSP made for the development of Twitter, such as number of followers and re-tweets. the knowledge portion of the results framework. Figure 1: Knowledge portion of results framework 2 SMARTLESSONS — AUGUST 2011 Figure 2: A screenshot of the WSP Results Tracker The communications and knowledge Conclusion management team spent roughly five to seven hours a week for six weeks mapping In the past, communications and knowledge out WSP’s activities, grouping them into activities have not generally been measured emerging categories, and identifying against pre-identified indicators. After all, to indicators and responsible parties for each, explicitly identify indicators that can be used to giving the framework a strong foundation track progress in communications and to build on. knowledge is to expose an area traditionally perceived as an art to rigorous measurements. The group also benefited from meetings This can be both, frightening and exciting— with a Monitoring and Evaluation Specialist, frightening because naming indicators also who had worked on WSP’s overall opens an activity to failure, and exciting because framework. The list of carefully identified it provides an opportunity to substantively activities and indicators were then plotted demonstrate progress, success, and/or lessons. into an excel sheet, giving the framework a working platform. The program allocated By taking the time to identify and track 15 percent of one staff member’s time to appropriate indicators and corresponding centrally track the various activities. assumptions, WSP found that the knowledge Analytics and tracking tools helped when component of the framework could be used to they were available but field staff input was more effectively and comprehensively articulate essential. its work. WSP has found that tracking communications and knowledge work can help DISCLAIMER strategically prioritize and manage activities. SmartLessons is an awards program to share lessons learned in development-oriented advisory services and investment operations. The findings, interpretations, and conclusions expressed in this paper are those of the author(s) and do not necessarily reflect the views of IFC or its partner organizations, the Executive Directors of The World Bank or the governments they represent. IFC does not assume any responsibility for the completeness or accuracy of the information contained in this document. Please see the terms and conditions at www.ifc.org/ smartlessons or contact the program at smartlessons@ifc.org. SMARTLESSONS — AUGUST 2011 3