36989 No. 102 / March 2006 REPORT CARDS AS A TOOL FOR EMPOWERING COMMUNITIES IN THE FIGHT AGAINST HIV/AIDS IN CAMEROON: A WORK IN PROGRESS The use of report cards in a HIV/AIDS prevention program in Cameroon appears to be a promising tool for galvanizing community interest, discussion and action. Local HIV/AIDS committee members are evaluated by communities who vote on the effectiveness of committee activities. As a result of the report cards, most communities replaced some of the members of the local HIV/AIDS committee and the content of the action plans to fight HIV/AIDS. The report card tool also facilitates discussions on a very sensitive and previously forbidden topic. Printed report cards are now available in 6,000 communities who could use the report cards to evaluate other activities or community services. Introduction transfer responsibility for local- level monitoring and evaluation (M&E) to the beneficiaries. One of the greatest challenges facing societies threatened with the spread of HIV/AIDS is to find To the best of our knowledge, it is the only large- ways to motivate people to change their behavior scale application of the report card approach to to prevent contracting the virus. Building this HIV/AIDS in the world. Elsewhere, report cards motivation entails the kind of communication have been used largely as "citizen report cards" to which generates awareness so that people provide feedback on client satisfaction with understand the dangers posed by the disease and delivery of water, health, education, electricity what can be done to avoid infection. The key is and other services. In the MAP, report cards are to inspire people to act on their own behalf. This being used both for community self-evaluation is the ultimate aim of the report card approach and for communities to evaluate program being implemented as part of the Multisectoral performance at the local level. AIDS Program (MAP) in Cameroon. The report card approach in Cameroon shows To empower people in the fight against early signs of success. It creates a forum for HIV/AIDS, thousands of community grants under public discussion on a very sensitive topic, the MAP need to be effectively managed at the increases awareness of the disease, and, most community level through participatory processes importantly, instills a sense of responsibility that enhance the accountability of local among the people ­ a realization that they are the HIV/AIDS committees to their communities. ones who can and must act to prevent themselves Most HIV/AIDS operations conduct random from being infected by HIV/AIDS. It encourages supervision of community action plans and annual the community assembly to tell the truth, to audits of 10 to 20 percent of the communities. In recognize that AIDS is still progressing or that 2003, the National AIDS Committee in Cameroon affected family members with AIDS are still (CNLS) decided to test self-evaluation tools that rejected. It increases project staff accountability would entrust the communities to monitor to the people, provides feedback by ordinary activities and evaluate them. The objective is to people on the effectiveness of the local community action plan to fight HIV/AIDS, and own report card approach based on his previous supports community evaluation and decision- experience as an extension officer. He recruited making processes. Still, this approach is in its social development specialists to support this infancy. Report cards are experiencing growing effort.1 pains as they begin to take hold. The report cards were introduced in 2003 as a tool to help with the local management of the MAP. Genesis of the Report Cards The report cards were not seen so much as an M&E instrument but as a capacity building The use of report cards in the MAP began as part activity to empower the communities to control of an effort to reach out and involve people at the and improve their action plans. The first year was community level, and to improve the spent largely on design and on finding a place in effectiveness and governance of community the MAP agency to house this novel approach. grants in fighting HIV/AIDS. CNLS used MAP CNLS decided to give responsibility for the report funds to provide grants to the communities to cards to the Local Response Unit of CNLS. In stimulate local actions to fight AIDS. NGOs April 2004, report cards were included, along sensitized communities and assisted them in with quantitative evaluation, as part of a setting up a Local Committee against AIDS. community self-evaluation system led by local Each committee worked with the community to program administrators known as Municipality prepare an action plan. Each community received Focal Points (MFPs). Report cards are now a a small grant (a maximum of $2,000 per year) to mandatory component of the participatory implement its action plan. These grants are assessment process that takes place every six managed by the committees. The MAP quickly months prior to requests for new six-month scaled-up: it began with 866 communities in allocations. 2002, was reaching 3,000 communities by the end of 2003, and expanded to 6000 communities in 2005. The Report Card Methodology This was the first time in Cameroon that grants The report card approach in Cameroon adopted its were used at the community level on such a current features during a February 2004 massive scale. CNLS gave communities workshop. Report cards are the second of the significant discretion in the activities they could following three stages. undertake based on identification of local factors affecting propagation of AIDS and access to The first stage, quantitative evaluation, now treatment. However, CNLS came under criticism called monitoring, records and assesses the for misuse of funds at the community level with progress of each planned activity in terms of stories of some individuals using funds for small actual achievements. allowances. Project management became concerned about the soundness of committee The second stage, qualitative evaluation, assesses management and about monitoring the large sums the impact of program activities at the community of grant money being administered by the level as perceived by community residents. communities. CNLS did not want to spend scarce Report cards are used to determine people's level resources to monitor or control community grants. of awareness about HIV/AIDS and to assess the effectiveness of actions against HIV/AIDS by The World Bank task team played its "knowledge tracing changes from year to year. The questions bank" role and informed CNLS that Care International was using report cards at the 1 community level in Malawi. The news triggered We gratefully acknowledge the support received from interest in Cameroon. CARE Malawi could not the Norwegian and Finnish Trust Fund for Environ- mentally and Socially Sustainable Development to send an experienced team to Cameroon, so the assist with the design and early implementation phase Bank task team leader helped CNLS design its of the report card initiative from 2003 to 2005. 2 can vary from one community to another but they · Are you satisfied with the work of the generally follow guidelines prepared by CNLS. local HIV/AIDS committee? They cover the disease itself, how to prevent it, and how well the committee is helping to prevent The three cards depict one, two, or three stars. it. The community representative votes one of the three cards, choosing the number of stars to Four groups of ten persons each ­ girls, boys, quantitatively represent a qualitative response. women and men who are not members of the For example, regarding the second theme, committee and who are not involved in the awareness of danger and risk associated with implementation of the action plan ­ are HIV/AIDS, the associated question is: "Have you purposively selected to represent the diverse changed your behavior due to HIV/AIDS?" If the ethnic groups, socioeconomic strata, and person votes with a one star card, that means he or neighborhoods of the community. The total she has changed behavior only slightly or not at sample size of 40 respondents represents about 5 all. If a three star card is selected, that denotes to 10 per cent of the average community. Three major behavior change. A hat is passed around to cards, color-coded for each of the four collect the cards, which are placed face-down to demographic groups (yellow for girls, green for assure confidentiality of responses. boys, pink for women and blue for men), are used to provide responses on each of the following 12 The third stage, analysis of the quantitative and questions on nine HIV/AIDS-related issues. qualitative evaluation results, is performed jointly by project staff and community members. They 1. Knowledge of AIDS complete the revision of the community's · Do you know how HIV is transmitted? `problem tree' and `vulnerability map'. Ideally, · Is there HIV/AIDS in your community? this final phase of the process entails a period of 2. Awareness of HIV/AIDS danger/risk group discussion based on the results of the · Have you changed your behavior due to monitoring and report card voting. This phase HIV/AIDS? should enable community members to gain a full 3. Quality of services provided appreciation of the degree to which they have met · Does anyone visit your community as planned goals for each of the identified activities part of the fight against HIV/AIDS? related to HIV/AIDS prevention so that they can · If so, are you satisfied with these visits? better identify further action to satisfy perceived 4. Promotion and use of condoms needs. · Do you have access to condoms? · Do you use condoms during casual sexual encounters? Implementation of the Report Card Approach 5. Voluntary blood-testing · Do the members of your community As of early 2005, after one year of implementing undertake voluntary blood tests? the report card approach, it had been applied in 6. Access to treatment 858 communities in all ten of Cameroon's · Do people in your community infected provinces, with the majority (55 percent) of with HIV have access to treatment? communities in the Southern Province. The 7. Stigma distribution of the 858 communities among the · ten provinces is as follows: Are people with HIV rejected by their families or by the community? · South ­ 469 8. Management of financial resources for the · South West ­ 122 fight against HIV/AIDS · Center ­ 63 · Is the money that is meant to be used for · West ­ 60 the fight against HIV/AIDS well spent? · North West ­ 55 9. Role of the local HIV/AIDS committee · Littoral ­ 30 · Adamaoua ­ 25 3 · East ­ 15 Another problem is the risk that the report card · North ­ 14 process will become more of an end in itself than · Extreme North ­ 5 a means for action. During the community meetings, there was a brief discussion following The total number was expected to increase the presentation of results, but little or no group significantly during 2005 as self-evaluation discussion when the results to all 12 questions becomes standardized as part of all community were tabulated. While the discrete findings on action plans. By October 2005, CNLS had each of the report card topics are important, just produced 6,000 sets of cards and guidelines on as important is the opportunity for reflection by how to use the cards, and disseminated them to community members on what these findings mean the communities. and what actions might be taken to remedy the situation, assigning responsibility wherever Thirty people have received basic training in possible. Report cards are a platform for group report card techniques: five managers from the deliberation and decision-making, a tool for Local Response Unit, 20 heads of provincial governance and empowerment, far more than a units, and five MFPs. The objective of this mere survey instrument. training was to enhance the effectiveness of the application of the report card tool at the The strengths of the approach outweighed these community level. drawbacks. In some way, this game of cards ­ on a topic which is associated with fear, shame, and sadness ­ was able to attract people's attention Observations on the Report Card Process and raise their awareness in ways that might have been far more difficult with a lecture by a health In early 2005, a team of World Bank consultants professional. The use of report cards seemed to visited three communities in the Southern lighten up and make more accessible a heavy, province, two rural and one peri-urban, during the formerly forbidden, subject. Several people in meetings to vote on the topics covered by the these communities said words to the effect of report cards. In each community, discussion was "Before these meetings I did not know what lively and keen interest was shown by many of HIV/AIDS was. Now, thanks to coming together those present. There were a number of apparent in this way, I do." problems. There were also some clearly positive features. Equally as important, the report card voting sessions provided a forum where community The most glaring shortcoming was the low level members could openly address HIV/AIDS-related of community participation in the report card issues together, thereby developing activity. Initially, there were relatively few communication with peers, family members and community participants not on the HIV/AIDS other age groups on an issue that is difficult to committee. Committee members voted and failed discuss. For example, communities often discuss to mobilize the community to vote. In one closing bars at an earlier hour, sexual relations community, only nine people voted, and all were between teachers and students, and taking actions members of the committee. CNLS quickly to influence local factors associated with corrected this situation as committee members are HIV/AIDS. Communities developed collective not allowed to vote on their own performance. In responsibility for action to mitigate the risks all of the communities, it took so long to collect involved and assist those affected by the disease. the cards, count them, and present results after each vote, that many people simply lost interest. The most stimulating exchange that took place in CNLS tried to correct this problem by asking the these communities concerned the issue of stigma facilitators to do the voting quickly, playing it like (topic 7) regarding the care of orphans in the a social game, and to pay more attention to the community. One man disagreed with the results third stage, the analysis of the quantitative and of the votes which indicated that most people qualitative evaluation results. were caring for affected community members. He 4 asserted that people in the village only cared for · The self-evaluation approach generally their own families and no one else. Many others needed further adaptation to the Cameroon hotly contested this. It turned out that there were context so that communities could better five orphans of AIDS-infected parents who were understand it. receiving community support. The heated · The questions needed simplification and good discussion appeared to galvanize the people's will translation into the local vernacular. to increase their collective responsibility and · The report card tool needed proper assistance to orphans and others affected by the administration by the MFPs during the self- disease. evaluation sessions. · Other key actors needed to be included ­ Refining the Report Card Approach MFPs, committee members, and peer educators (pairs educateurs) ­ along with After the first year of using the report card, it community members as evaluators. became clear that several issues were not being sufficiently illuminated by using only yes/no Much of the refinement of the report card questions. Accordingly, a short, qualitative approach comes down to improved training for interview guide was added to address these issues. the MFPs, the persons primarily responsible for implementing the program in each community. 1. Voluntary blood testing (theme 5) MFPs need to select participants for the report · Do members of your group get blood card activity who are not members of the tests? HIV/AIDS committees and who fairly represent · If not, why are people reluctant to be the diversity of the community, animate the self- tested? evaluation sessions, manage the voting 2. Actions undertaken by the community to expeditiously, keep the community members reduce HIV/AIDS prevalence (theme 2) present at the evaluation sessions informed about · Describe the nature of these actions and results on an ongoing basis, and express the how the members of the community questions correctly in the local language. Proper participated in realizing them. training of local MAP staff (the MFPs) charged 3. Role of the local committee (theme 9) with administering the report card program is · What are the reasons the community is clearly a major need that was underestimated satisfied or dissatisfied with the actions of during the program's conception. Problems the local HIV/AIDS committee? encountered during observation of the report card meetings in the communities have generally been This short interview guide was to be administered resolved satisfactorily as follows: by the Municipality Focal Point (MFP) to three · HIV/AIDS committees are seeking increased members of each of the four aforementioned community participation in the meetings and demographic groups not previously selected for are aware that they should not take part in the the report card voting, thus expanding the total voting themselves. number of persons reached by the qualitative · Community meetings are prepared in advance evaluation to 52 per community. Each interview to ensure sufficiently broad representation (by was expected to take ten to fifteen minutes. locale, ethnicity, socioeconomic level, etc.), and meeting times are set to maximize community Beyond the issue of yes/no questions, there were a attendance. number of other aspects of the report card · Vote results are now presented in large enough approach which were problematic during this first letters to be read throughout the assembly hall. year or so of experience. · The MFP, who leads the meetings, is trained to accurately translate the voting procedures and · MAP managers responsible for oversight of topics into the local vernacular. the report card methodology needed to become more familiar with the approach. 5 Suggestions for Improving the Program might say that if refreshments are served people will attend primarily for that reason. The analogy Beyond refining the report card approach itself, may be children who receive school lunches. other issues are now being addressed to improve They are more apt to go to school, but while there the program. These may be organized around two they will also go to class and learn. Similarly, broad themes: recognition and integration. providing modest refreshments may draw people to the report card sessions, but once they are there Recognition they will participate in the discussions that should enhance their wellbeing. HIV/AIDS does not lend itself to open, free discourse. The topic is too intimate, private, and Integration potentially embarrassing and judgmental. To induce constructive dialogue about HIV/AIDS, Propagating messages aimed at changing peoples' careful thought should be given to some sort of behavior to reduce the prevalence of HIV/AIDS is material inducement or recognition. Two not easy. To do so effectively requires all the opportunities for such recognition are the peer help available: to strengthen communication in educators (pairs educateurs) and the report card whatever way possible, but also to enhance the meetings. transmission and acceptability of the message. The message itself, to be palatable among The educators are the foot soldiers, the local ordinary persons, may best be cloaked as part of a itinerant teachers, of the HIV/AIDS health holistic, multi-sectoral development program profession. They go door to door in the villages rather than as an isolated HIV/AIDS intervention. (where they reside) talking to people about the Stronger communication can come from forging meaning of HIV/AIDS: the nature of the disease, strong links between the HIV/AIDS community risks associated with it, prevention strategies, and program, including the self-evaluation with its so on. They are the key source of information, report card component, and local government. especially about what the HIV/AIDS committees This strengthening can also entail sound analysis are doing. They motivate people to attend the with proper use of qualitative and quantitative community meetings. At present, they receive no research techniques. remuneration. Given that they are drawn from the villages, they are relatively poor. Many have had In one community, during the report card voting to resign from this service due to the high procedure, the local mayor made an unannounced opportunity cost of the time they spend on this visit. He had heard of the meeting and wanted to voluntary educator role. Their precious time is come and lend his support to the cause of fighting needed to earn income for their families. Opinion HIV/AIDS. This was an articulate, hands-on, was unanimous in the villages visited, as well as respected mayor. When he spoke about the among the representatives of partner donor importance of the HIV/AIDS program, people agencies (GTZ, UNAIDS, UNDP), that the peer listened. After he left, people were more attentive educators should receive some remuneration, both to the proceedings than before he came. Clearly, to retain them in this important function and to collaboration between local government give them the recognition they deserve. authorities and the HIV/AIDS program is one way to strengthen its effectiveness. The other opportunity for granting recognition as Perhaps more basic and obvious than this inter- an incentive for engagement on HIV/AIDS is the institutional integration is the need for closer semi-annual meeting at which report card voting collaboration between the Local Response Unit takes place. In many parts of Africa, certainly in (LRU) and the Operations Unit within the Central the south of Cameroon, community gatherings Technical Group (CTG) of the National traditionally are occasions where food and drink HIV/AIDS committee (CNLS). The CTG has a are provided. This refreshment not only monitoring and evaluation unit which, until now, encourages people to attend the event but also has been disassociated from the LRU, so there is demonstrates respect for their attendance. Critics 6 very little analytical basis upon which to assess People appreciate this process whereby local progress and identify areas needing improvement. HIV/AIDS committee members report to them on program activities. Those entrusted to manage Taking just one example among many, a key issue the program are made accountable to the people facing the community members, especially in the who, after all, must be the key actors in avoiding outlying rural areas, was the cost of transport to infection from this pandemic. Where this self- the city where the blood-testing facility was evaluation has taken place, community members located. People were said to generally accept the now claim that they are better informed and better need for testing but many did not go because of understand the use of funds employed in the fight the cost of two trips, one for the test and a later against HIV/AIDS. one for the results. The people of one community said they had found a solution to this problem by Social monitoring is also well received. People convincing the government to drive a mobile are empowered by participating and being testing unit to their community. Sound, consulted on the main issues of the HIV/AIDS comprehensive analysis of peoples' willingness program. Shortly after the first self-evaluation and ability to pay for transport to testing centers, sessions had taken place in 2004, a number of and of the cost of bringing testing units to local HIV/AIDS committees were replaced as a villages, would be most useful to policy formation result of communities expressing dissatisfaction regarding blood testing. The institutional with them. The main reason was the integration recommended here may well soon embezzlement of funds targeted for HIV/AIDS happen as one of the next steps for the report card prevention activities. Generally, the self- component is to transfer it to the monitoring and evaluation process is becoming accepted and a evaluation unit of the CTG. part of community life. The final recommended integration may well be It is difficult to provide any definitive assessment the most important. Common sense tells us that of the effectiveness of an innovation like report people feel uncomfortable discussing HIV/AIDS. cards when it is little more than a year old. This The topic is just too personal and, increasingly, difficulty is compounded by the lack of too painful, as loved ones die of this disease. It is qualitative and quantitative analysis done on the likely that a significant number of people do not program to date. What can be said in this participate in meetings on HIV/AIDS because of progress report is that the report card approach this discomfort and embarrassment. The report does seem to be making positive, noteworthy card approach for self-evaluation may well play a contributions to HIV/AIDS prevention in useful role in all development sectors ­ health, communities where it has been applied. This is a education, agriculture, etc. This integration would promising form of social accountability at the make sense because all sectors are affected by community level. HIV/AIDS but also because, presented as part of a development package, people would likely feel People seem to appreciate and to be drawn into less timid addressing issues associated with the engagement in the difficult discourse on disease. Consideration is being given to HIV/AIDS by the innocence and even fun of the incorporating HIV/AIDS issues in this manner report card game. They appear to enjoy seeing into a Bank-supported community-driven immediate feedback on their responses to development (CDD) program in the near future. questions. Awareness of the disease has increased as has community mobilization to take collective remedial action. The people have Conclusion: The Potential of Report Cards for gained ownership of their own self-evaluation HIV/AIDS Prevention work. They feel a sense of power by influencing actions on HIV/AIDS and judging the quality of Overall, the self-evaluation process and its report health and support services. With the corrective card instrument have been well received by the measures recommended above ­ relating to communities, the municipalities and CNLS. preparation of the communities, training of the 7 meeting leaders (MFPs), better use of the local vernacular, recognition, and integration ­ it appears that this report card approach may well be internalized in the HIV/AIDS program of Cameroon in such a way that it will provide a sustainable contribution to the prevention of this disease. This note was prepared by Lawrence Salmen (SDV), Marcel Bela (CNLS), Anne-Jeanne Naude (consultant) and Jean Delion (AFTS2, and task team leader of the MAP in Cameroon). Additional copies can also be requested via e-mail: sdcommunications@worldbank.org. 8