FINAL EVALUATION OF THE TRANSPARENCY OF THE MAURITANIAN EDUCATION BUDGET (TOME) PROJECT (P162168) Mohamedou Ould Lekweiry Jean-Benoît Falisse 25 February 2022 Executive Summary This report is the final evaluation of the Transparency Of the Mauritanian Education Budget (TOME) project conducted by the NGO Ecodev in the Islamic Republic of Mauritania between 2017 and 2021. At its core, the project implemented social accountability mechanisms in forty pilot schools (twenty in Nouakchott South, Moughataa of Riyadh, and twenty in Tagant, Moughataa of Tidjikja), through an internet platform for information sharing and establishing committees including different local education stakeholders (municipalities, parents, and teachers). These committees were set up at school level ( Comité de Suivi Participatif de l’École, CSPE), national level (Comité National de Plaidoyer, CNP), and later, following an identified need, at departmental level (Comité de Suivi Participatif Départemental , CSPD). Like the other projects in the third group of Global Partnership for Social Accountability (GPSA) beneficiaries, the aim was above all to bring about a change in wider scale dynamics and inspire new practices beyond the project itself. It is these aspects that the report focuses on: what dynamics have TOME helped (or not) to instill? What explains the changes observed (or the absence of such changes)? What can the project teach us about how social accountability dynamics can be implemented? The evaluation is based on a review of various reports produced by the project, recent documents from the Mauritanian Ministry of Education, an analysis of data from the Medrassety.net platform used by the project, twenty-six individual interviews and six focus groups (including participatory drawings). Evaluation of the Project’s Objectives The first part of the evaluation looks at how the three key intermediate indicators of the project were, or were not, achieved. For the first indicator, ‘capacity building and partnership framework for civic monitoring of primary education’, the success of the strategy adopted depend s on the perspective adopted. If the expectation is that social accountability dynamics will be effective and lead to significant educational change in each school, then a lot of work remains to be done. If, however, the idea is just to demonstrate the viability of a concept, then the picture is much more positive  - especially given the difficult circumstances. It seems difficult to think of an approach that would have been more effective in setting up a new dynamic. The interviews and secondary data show that the sensitization of the main stakeholders and the initial training worked very well (adapting to the context of the pandemic was, however, challenging - and very much in line with what has been reported in other parts of the world). Many field actors were highly enthusiastic about putting into practice the social accountability principles that had generated great enthusiasm in the training and awareness-raising phase. Ultimately, one crucial element clouds the picture: the budgetary issue. Despite its efforts to seek funding and support for the conceptualization of school action plans, the TOME project could not ensure that the schools and committees it trained were allocated additional resources. These financial resources would have enabled the new dynamics initiated in the schools to be genuinely implemented - tension results from this. 2 On the second indicator, ‘generation of citizen feedback through the process of social accountability and collaborative engagement with public sector institutions’, the project has achieved clear successes; especially with the enactment of a new legal framework and then the promotion of a new accountability institution, the School Management Committee (COGES), which is a direct continuation of the CSPEs piloted during the project. The ideas promoted by TOME, first and foremost social accountability and collective action of different parties in favor of education, seem to be fully supported by all actors  - including those for whom they mean having to reconcile with new parties (e.g., in the administration). Early advocacy engagement with the authorities and the construction of an exemplary experience (a success story) are the main elements used by the project to generate collaborative engagement. At the school level, significant challenges remain  - notably with the lack of resources to turn community feedback into actions - but the strategy of demonstrating the added value of the CSPE model with a test school and involving the authorities in the process is working. The evaluation shows, however, that it is vital to qualify the link between citizen feedback and collaborative engagement at two levels: firstly, the contribution of feedback (and of TOME in general) is sometimes direct, sometimes more diffuse, and secondly, citizen feedback is not always the only way in which the TOME project affects public measures. With regard to the third outcome, ‘facilitating knowledge and learning to strengthen the effectiveness of social accountability interventions and project management’, the TOME project is a new and highly iterative process in the Mauritanian context, from which there is plenty to learn. The project developed beyond just a platform, setting up a whole ecosystem of social accountability, with training and new committees. The COVID-19 crisis, in some ways, forced a return to a more platform-centric approach. The general understanding of TOME and the principles of social accountability is generally good and aligned with the project's objectives. Still, it is also quite clear that for many field actors, there is a need to see very concrete changes (including material changes) to embrace the approach fully. Elements Facilitating the Project The TOME project is, at first glance, a radical innovation in the Mauritanian educational landscape, which does have any community participation and accountability initiatives. However, a closer look at the context shows that the idea is embedded in different time frames and realities, which together provide a window of opportunity to understand better why and how the central idea of TOME - the set-up of participatory school committees and more collaborative school management - was able to take place in the pilot schools and spread to the national level. First, considering the long history, the idea of collaboration echoes religious and traditional principles and, therefore, makes sense for everyone. Second, within the education sector, the conditions are ripe at several levels: (1) a cycle of education sector reform is underway, (2) the idea of greater decentralization and devolution in the sector has already been launched and is gaining momentum, and (3) the sectorial crisis is such that actors, who feel stuck, are ready to experiment with something new to get out of the deadlock. TOME’s original theory of change echoes these points, even if it anticipates more resistance to the idea of social accountability from the administration than there was (probably because of the aforementioned education crisis). This theory of change also identifies the importance of a space for discussion, 3 which undoubtedly played a role. However, we also find that this space must be conceived beyond the formal space: the positioning of TOME and its staff as anchored in civil society, but also very connected to both the administration and the population, is key to creating a coalition, a mobilization, and a real interface between the state and the citizens. Sustainability The TOME project aimed to demonstrate the possibility of a new model of civic engagement and participatory governance in schools. With the enactment of the COGES law, but also through the strong interest of national (parliamentarians), local, and civil society actors (in particular with the National Federation of Parents, the Fédération Nationale des Parents d’Elèves de la République islamique de Mauritanie [FENAPRIM]), it seems that a new field of possibilities has opened up in the education sector. The recent World Bank Education Sector Support Project in the five poorest wilayas (regions) of the country (PASEB II) has taken up the support for the COGESs, and incorporated the need for support to their budgets. This is, of course, an encouraging sign and should ensure the system piloted by TOME is implemented and monitored for a guaranteed five-year period (2020–2025). However, these positive elements should not obscure the fact that the social accountability dynamic promoted by TOME remains fragile and still requires support. The sector remains highly centralized and national discussions (such as the national consultations) address the subject of social accountability without making it a central issue. People we met who had not been involved in TOME, both on the side of the partners and the administration, often struggle with the concept or even with the idea of COGESs. In terms of on-the-ground achievements (local/school and departmental), the enthusiasm of committee members is certain, but the sustainability of the newly created institutions (in the form of the COGES or otherwise) also remains fragile. Even with a trained population, the lack of means (budget) to follow up on plans and decisions risks making committees not supported by partners - including those with a legal existence - empty shells or at least less central actors than conceived by the new laws (and by the TOME project). Contribution to the GPSA Theory of Action What does the TOME project tell us about the GPSA Theory of Action? The general theory seems to be validated on many points. In terms of the hypotheses of what allows the dynamics to be put in place, we note the importance of (1) the willingness of the actors, including the government, to put in place a new dynamic, which in the Mauritanian case comes from the realization of the impasse and the difficulties of the education sector, which are an observation shared by all at the beginning of the TOME project; (2) the development of the project within a framework of institutional reform, which in the case of TOME is above all the development of the ideas of decentralization and putting the education sector at the top of the government’s agenda (and even more so after the Presidential election of 2019); and (3) the formation of a broad coalition that supports the education sector in developing approaches that are more inclusive of all the key stakeholders. TOME is a project that is, very clearly, at the beginning of a possible broader change. Some of the actions being taken bear fruit and are inspiring. Various steps towards a medium-term effect are visible, including the scaling up of a key mechanism for accountability and the setting up of multi-stakeholder reflection processes. However, it is too early to speak of long-term effects or spontaneous collaborations. Things 4 worked extremely well in the one school where the whole package of interventions could be delivered, but, so far, many school stakeholders see changes in processes and relationships rather than significant changes in school performance and learning levels. The TOME experience allows us to refine or formulate some additional hypotheses on the TOME Theory of Action, notably that: (1) the demonstration of the value and effectiveness of social accountability approaches can be done based on a small number of pilot experiences  - in many ways it is mainly the example of one school that TOME uses to push and promote its approach; (2) important questions about sustainability arise if the accountability and participation approach are without means: the stakes are de facto low as there are no real resources to manage (no budget) beyond the rational use of already very scarce material and human resources. The risk is then that enthusiasm and energy will run out; and (3) a condition for change is the ability to mobilize broadly (which the GPSA Theory of Action already talks about), but the TOME project shows how this ability is not just a matter of setting up formal frameworks - beyond committees and formal meetings, what makes the difference with TOME is its team that links up and facilitates the relationship between actors, including the national education administration, the federation of parents’ associations, municipalities, and teachers. To do this work, the TOME team relies on a network developed before and during the project and on a project team composed of members with work experience (and contacts) in the different institutions involved in the project. 5 Contents Executive Summary ………………………………………………………………………... 2 Acronyms and Abbreviations ……………………………………………………………… 7 1. Introduction …………………………………………………………………………….. 8 2. Methodology…………………………………..………………………………………... 8 Objectives of the Report 8 Evaluation Questions 10 Theoretical Framework 10 Methodological Approach 10 3. Context ………………………………………………………………………………… 12 General Context 12 Sectoral Context 12 General Presentation of the TOME Project 15 4. Theories of Action and Change………………………………………………………. 17 GPSA's Theory of Action and Change 17 TOME's Theory of Action and Change 18 5. Project Implementation ………………………………………………………………. 22 Phase 1 22 Phase 2 24 6. Achievement of the Project’s Objectives …………………………………………. 24 Capacity Building and Partnership Framework for Civic Monitoring of Primary Education 25 Generation of Citizen Feedback through the Social Accountability Process and Collaborative Engagement with Public Sector Institutions 34 Facilitating Knowledge and Learning to Strengthen the Effectiveness of Social Accountability Interventions and Project Management 44 7. Localizing the Theory of Action ……………………………………………………. 47 An Idea with a Long History 47 A Favourable Political Context 48 A Trend That Speaks for Itself 49 Multiple Positioning 49 A Strong Team 50 8. Challenges and Sustainability ..……………………………………………………… 52 Weaknesses and Challenges 52 A Changing Context, but One in Which Significant Challenges Remain 55 The New World Bank Funding 57 New Areas to Explore 58 9. Conclusions: Looking Back at the Theory of Action …………………………….. 59 10. Recommendations …………………………………………………………………….. 60 References ......………………………………………………………………………………63 Annexes ……………………………………………………………………………..............63 6 Acronyms and Abbreviations CNP National Steering Committee (Comité National de Pilotage) COGES School Management Committee (Comité de Gestion de l’École) CSPD Participatory Departmental Monitoring Committee ( Comité de Suivi Participatif Départemental) CSPE Participatory School Monitoring Committee (Comité de Suivi Participatif de l’École) FENAPRIM National Federation of Parents of Students of the Islamic Republic of Mauritania (Fédération Nationale des Parents d’Elèves de la République Islamique Mauritanie) GPSA Global Partnership for Social Accountability MP Member of Parliament (Député) PASEB Basic Education Support Project (Projet d’Appui au Secteur de l’Education de Base) SPA Students’ Parents Association (Association des Parents d’Elèves) TOME Transparency of the Mauritanian Education Budget WB World Bank WBG World Bank Group 7 1. Introduction This report is the final evaluation of the Transparency Of the Mauritanian Education Budget (TOME) project conducted by the NGO Ecodev in the Islamic Republic of Mauritania between 2017 and 2021. At its core, the project consisted of implementing social accountability mechanisms in forty pilot schools through the establishment of committees including different local education stakeholders, including municipalities, parents, and teachers. Like the other projects funded in the third group of Global Partnership for Social Accountability beneficiaries, the project's aim was to bring about change and inspire new practices beyond the project itself. It is these aspects that the report focuses on: what dynamics did (or did not) TOME bring about, what explains the observed changes (or lack thereof) and what can the project teach us about how social accountability dynamics can be put in place? While the report addresses project implementation issues and returns to the level of achievement of activities, it is not a study of project impact in the typical sense of the level of achievement of narrowly quantified objectives internal to the project, but rather an analysis of the dynamics developed through the project. Thus, the report seeks to understand the theory of action which has made this project a success - at least judging by the changes in approach in the education sector in Mauritania. What explains why the ideas that TOME promotes have clicked with some of the country’s law and policy-makers? And how do we explain a certain level of frustration - or at least a sense of something unfinished - among some (but not all) at the grassroots level? Section 3 sets the general context for the study, reviewing the general and sectoral background, and then the GPSA and the TOME project Theories of Action are presented. The general course of the project is described in Section 5, before Sections 6,7 and 8 – the core of the report –  address the main evaluation questions, namely: Did the project strategy contribute to the expected outcomes and impacts? If so, for whom, to what extent and in what circumstances? (Section 6) To what extent and how did the sector system enable or hinder the implementation of the project? What, if any, contributions have the project made to strengthening these sector systems and addressing bottlenecks and other implementation gaps? (Section 7) Under what conditions are the results sustainable? What is the risk that the results achieved are not sustainable? (Section 8) 2. Methodology Objectives of the Report The main objective of this final evaluation is assessing the effectiveness and sustainability of the project, while paying attention to context and processes. The specific objectives are: 1. To generate learning and knowledge about the conditions under which the project has operated and under which it will be able to sustain its results. The aim is to inform 8 improvements in theories of change, strategies, programs, and social accountability projects by key project stakeholders. a. To what extent and how has the sector system enabled or hindered the implementation of the project? What, if any, contribution has the project made to strengthening these sector systems and addressing bottlenecks and other implementation gaps? b. Under what conditions are the results sustainable? What is the risk that the results obtained are not sustainable? 2. Demonstrate the results and social return on the investment made in the project in a credible and transparent manner. Accountability is a multidimensional concept: upwards (donor and government), downwards (target communities and beneficiaries), and horizontally (between project partners). More specifically: a. Did the project strategy, including course corrections due to changes in context and learning, contribute to the expected results? If so, for whom, to what extent and in what circumstances? b. What unintended outcomes (positive and negative) were produced (including spillover effects)? What effects, if any, did they have on the project’s theory of action and assumptions? 3. To what extent do the results validate the GPSA Theory of Action and its adaptation to Mauritanian education and governance contexts through the project? A central objective of this final evaluation is to understand how the project aligns (or not) with the GPSA Theory of Action. The theory of action is the key to the evaluation and should be central. It requires a holistic assessment of all project components, including how they interact to achieve results (or not). The analysis therefore includes, but is not limited to, the assessment of the following: a. To what extent, why and how have the project’s lessons fed into broader reform efforts, including those led by government, World Bank Group (WBG) dialogues, operations and country and sector strategies, and other development partners? The GPSA is particularly interested in sustainability through the partial adoption of project lessons and approaches by public sector institutions, WBG operations and strategies, development partners, among others (see outcome 4 of the GPSA Results Framework in Annex D). b. With regard to the GPSA Theory of Action, to what extent and how did the project implementation process and the achievement of or contribution to the identified results align with the hypothesized path as described in the GPSA Theory of Action flower illustration? Are there examples of this project’s journey – from grant receipt to contribution to long-term outcomes – that do not align with the path described in the GPSA Theory of Action and related assumptions? If so, what explains this divergence? 9 Evaluation Questions The evaluation questions will therefore be as follows: a. To what extent, under what circumstances and for whom has the project contributed to the expected medium- and long-term results, in line with its theory of action and Results Framework? This first question covers the results of the project. b. Did the project outcomes include any unintended results (positive and negative)? If so, what were these unintended outcomes and what effect, if any, did they have on the project’s theory of action and assumptions? c. With regard to the GPSA Theory of Action, to what extent and how did the process of project implementation and the achievement of or contribution to the identified results align with the hypothesized path as described in the flower-shaped illustration of the GPSA theory of action? Are there examples of this project’s journey – from grant receipt to contribution to long-term outcomes – that do not align with the path described in the theory of action and its related assumptions? If so, what explains this divergence? d. In what ways, if any, is the project or any of its components likely to be sustainable? As discussed in the GPSA MERL guide, the GPSA is particularly interested in sustainability through the partial adoption of project lessons and approaches by public sector institutions, WBG operations and strategies, development partners, and others (see Outcome 4 of the GPSA Results Framework). Theoretical Framework The methodological approach borrows from different conceptual frameworks. At a fundamental level, it is a realist evaluation (Pawson & Tilley, 1994). This is an evaluation that attempts to reveal the functioning of different mechanisms (which underlie the results) by taking into account the particular context in which the project took place – the educational situation in Mauritania between 2017 and 2021, including the local specificities of the schools in the wilayas of Nouakchott South and Tagant. The realistic evaluation thus ultimately provides a complete picture of what happened. In addition to the realistic elements, the evaluation borrows from the contribution analysis method (Befani & Stedman-Bryce, 2017), which aims to verify the elements of a theory of change by testing each step. Here, our starting theory of change will be limited, but we hope to identify one or more theories of change in the preliminary phase and test and refine these ideas – these theories will necessarily be a contextual update of the broader GPSA Theory of Action. A Baseline Study was carried out at the beginning of the project, and although the implementation of activities was iterative, we will refer to this baseline. Methodological Approach The research was divided into different phases that allowed (1) to adjust the general orientation of the research to the GPSA framework and the project framework (ensuring that the questions 10 that are, in practice, the research questions are also the questions that the GPSA and TOME Theory of Action use); (2) to adjust the research questions and tools to ensure that nothing is missing; and (3) to avoid unnecessary work and to focus resources on essential aspects. In the preliminary phase, we conducted orientation interviews with the former TOME project leader and a World Bank official in Nouakchott. The main objectives of these interviews were to (1) understand the development of TOME and its context from the perspective of some of its key actors; (2) identify bottlenecks and achievements, as well as dynamics, in the different elements of the GPSA Theory of Action; and (3) identify possible documentary sources. In the preliminary phase, the documents produced by the project were also analyzed to understand the different stages through which it passed and to understand better the (implicit) theory of change on which it is based. This also involved (1) exploring the channels through which the project’s ideas may have circulated, including the online and offline p articipatory tools promoted by the project and the exchange and learning mechanisms that may have existed within the project. It was also (2) to explore the development or otherwise of aspects that are not necessarily easily quantifiable (and therefore might not be reflected in the project indicators), such as budget monitoring capacity and information analysis and sharing (two aspects central to the project’s objectives). In the second phase, twenty-six individual interviews and six focus group discussions were conducted following a general interview guide (Annex B) and echoing the developments outlined in the first phase. It was, of course, not possible to talk to everyone. We sought to meet people involved at all three levels of the project - municipal, departmental/intermediate, and national - and in the widest possible range of positions (see Annex A). They are not identified by name in the report, to give as much freedom as possible to the people we met. We, therefore, quote the people interviewed, mentioning their function (and sometimes their geographical area); our aim here is to describe the general dynamics. For the local level, we selected three schools in each wilaya, ensuring this included both schools where the project was judged a success and schools where the project was more difficult. The idea is to see, by comparison, what elements make a difference. We then conducted individual interviews and focus groups with parents, teachers, student representatives and representatives of the municipality. In the first part, the committee members draw a picture of the school as they see it, placing the different actors in it. The idea of the drawing is to have a basis for explaining the relationship between the actors and the most important elements. In the second part, the discussion starts from the drawing and covers the issues outlined in the interview guide. In addition to these interviews, we analyzed the data available online on the project platform, Medrassety.net. Initially, we had planned to analyze the minutes of the CPSE meetings, but these were often not available, making this task impossible. It was, however, possible to analyze reports made by the TOME project on action plans and the level of organisation of CPSEs and other committees. 11 3. Context General Context1 The GPSA is a multi-donor trust fund established in 2012 to contribute to the citizen engagement agenda by providing strategic and sustainable support to CSOs to reflect the voice of beneficiaries, promote greater transparency and accountability, and achieve stronger development results. The GPSA contributes to country-level governance reforms and improved service delivery by (a) generating knowledge, networks and funding to strengthen the capacity of civil society to engage in evidence-based social accountability; (b) helping World Bank teams and government counterparts integrate social accountability more strategically into their programs; and (c) leveraging the experience, knowledge and resources of external partners to enable the World Bank to scale up its engagement in this area. The GPSA provided grants to forty-one civil society organisations from over thirty-four countries in total to address critical governance issues in low- and middle-income countries through four global calls for proposals and other forms of support. Following the third GPSA call for proposals in 2016, TOME or ‘Transparency of the Mauritanian Education Budget’ was selected and approved on 13 March 2017. The project focuses on improving the transparency of the primary education budget by monitoring and evaluating its implementation and quality through social accountability tools in forty schools in the wilayas of Nouakchott South (twenty schools) and Tagant (twenty schools). The NGO Ecodev implemented the TOME project in partnership with the FENAPRIM. The GPSA-supported project was to use collaborative social accountability processes to contribute to better educational outcomes. Previous internal and external evaluations to the school authorities had already identified the following constraints: teacher shortages, lack of teaching and learning materials, and poor school management. The project sought to address the last point, i.e., the main inefficiencies and bottlenecks in school management, to improve access to and quality of education, and to determine how the education budget could be used more effectively. Sectoral Context In line with the GPSA approach, we identify three levels of context: (1) the general context of the education sector in Mauritania, which is the background to our research; (2) the context of education policies in Mauritania; and (3) the context of the practical (and actual) implementation of these policies. Our study is mainly interested in the difference between these last two levels of context, which is where the TOME project fits in. Education in Mauritania2 Despite sustained investment over a long period, Mauritania has not yet achieved universal access to primary education. While the gross enrolment rate in primary education has been over 100 percent for the past decade, the net enrolment rate for the primary school age group (6–11 years) has remained below 80 percent, well below the average for sub-Saharan Africa and the Arab countries. Serious gender gaps exist in the Mauritanian education system, particularly in school management. Gender parity has been achieved in primary school enrolment, where girls accounted 1 From the Terms of Reference of the evaluation 2 Adapted in part from the Terms of Reference and the Baseline Study 12 for 50.1 percent in 2018. However, as far as teachers are concerned, women accounted for only 33 percent of primary school teachers. Women are especially absent from school management, accounting for only 4 percent of school headteachers. The low proportion of women among teachers and headteachers may help explain the gender gap in learning outcomes. Mauritania was ranked 150 out of 157 countries in the 2018 World Bank Group Human Capital Index (HCI). In 2020, Mauritania’s HCI score is 38, which means that a child born today will be 38 percent as productive at age 18 as they could be if fully educated and healthy. Mauritania’s score and rank are low, mainly because of the country’s education problems. The COVID-19 crisis is still weakening the education sector. The pandemic has forced nearly one million learners out of classrooms, with the closure of public and private educational institutions, from primary schools to universities. Given the significant economic and social impact of COVID-19, the government announced the closure of all educational institutions on 14 March 2020, two days after the first confirmed case. Schools reopened on 1 September but closed again for several weeks between 4 December 2020 and 11 January 2021. The periods of closure have impacted human capital accumulation, with disproportionate effects on vulnerable populations, particularly girls, who are at risk of not returning to school and have less access to the means of communication that can be used in alternative education. At the same time, the new government since the election of the new President of the Republic (who took office in August 2019) has expressed a strong will to improve access to and quality of education in the framework of human development, to increase social cohesion and to ensure sustainable and inclusive growth. Education Policies The General Report on the 2021 National Education Sector Consultation Days traces the history of education policies in the country (MEN, 2021). It describes the post-independence period: “In Mauritania, and in relation to this issue [education], there has been unanimity in recent years about the worrying deterioration in the quality of the national education system and its inability to meet the major challenges that continue to confront the nation. The many reforms (1959, 1967, 1979, 1999) implemented since independence have all focused on linguistic issues, being limited to bureaucratic recipes which, instead of solving problems, have often created them, sometimes with consequences affecting even the national unity that the education system was supposed to consolidate and preserve. The 1979 reform aimed to systematize teaching in national languages (Arabic, Pular, Soninke and Wolof) throughout the education system. The measure was applied immediately for Arabic speakers, while for the other three linguistic communities, a transitional regime was instituted during which children would be taught in a so-called 'bilingual', but essentially French-leaning, stream, pending experimentation with national languages by the language institute created for this purpose. The transitional stage has lasted for twenty years, with the result that separate education has been de facto enshrined. It is true that the 1999 reform corrected this situation by unifying the system through the introduction of bilingual education. Diagnoses were made, strategies and action plans drawn up, and significant resources mobilized by the state with the support of its development partners to consolidate the achievements and support the implementation of this reform. But, more than twenty 13 years later, the results are far from satisfactory. In particular, the system's quality is poor and continues to deteriorate.” The Baseline Study of the TOME project continues to unfold this story (Lekweiry, 2018): “The Mauritanian government, with the support of the World Bank and other development partners, has developed an ambitious ten-year National Program for the Development of the Education Sector (PNDSE) to serve as an instrument for implementing the reform. The PNDSE has completed its first phase, PNDSE I (2001–2011) and the second phase, PNDSE II (2012–2020), is currently being implemented. In addition, since 2015, Mauritania has adopted a new education policy reflecting the government’s long-term vision for the development of the education sector and its desire to pursue the 1999 reform and achieve the 2030 Agenda for Education for All. This policy is part of the Strategy for Accelerated Growth and Shared Prosperity (SCAPP) and is inspired by the Education 2030 Framework for Action and aims to achieve Sustainable Development Goal 4 (SDG4). Today, the results of this reform show that although the State’s efforts for education (4 percent of GDP) have enabled significant progress to be made in terms of access to education (100 percent [official] enrolment rate), the quality of learning has not followed, as shown by all the evaluations carried out since 2003 by the National Pedagogical Institute, which show that the level of acquisition of pupils is constantly deteriorating. The World Bank’s 2018 Education Service Delivery Indicators (ESDI) survey in Mauritania reveals that the average Mauritanian Grade 4 student achieves 34 percent of the overall score in mathematics, 50 percent in Arabic and barely 14.2 percent of the total points in French. The causes of the learning crisis lie in four key factors: children unprepared to learn, unqualified and unmotivated teachers, inputs that do not often reach the classroom, and management that does not affect learning.” (MEFS, 2006). The last point is particularly important to consider when understanding the TOME project. In Mauritania, the management capacity of schools is very weak, as they have no autonomy of decision and no budget of their own. In practice, this means that school headmasters cannot help teachers solve problems, provide pedagogical advice, let alone set objectives that prioritize learning.” The World Bank in Mauritania3 “In terms of the project’s alignment with the broader work undertaken by the World Bank Group (WBG), the Mauritania Education Sector Support Project (P163143) represents the main intervention under Mauritania’s Country Partnership Strategy (CPF) for years 18–23 in support of objective 2.2 on improving access and quality of general education. It also addresses the cross- cutting theme of strengthening governance and institutions. The Mauritania Education Sector Support Project (P163143) takes into consideration. It complements other WBG interventions, including Development Policy Operations (DPOs: P167348, P171238), additional funding for the Public Sector Governance Project (P165501, parent project P146804), and Education Budget Management Transparency (TOME project, P162168), to ensure coherence and effectiveness of 3 Adapted in part from the Terms of Reference and the Baseline Study 14 WBG support. The CPF is aligned with the priorities of the government’s Strategy for Accelerated Growth and Shared Prosperity (SCAPP) 2016–2030 and builds on the WBG’s comparative advantage.” Ecodev The TOME project is implemented by the organisation Eco-développement (Ecodev) which is an association (under Mauritanian law) created in 1999. Ecodev’s mission is to ‘engage in the development of its country’, and to this end its initiatives improve the living conditions of the population. The association also contributes to the protection of natural resources and the environment. Thus, it participates in the implementation of local development, food security and environmental protection projects, and the promotion of access to basic services (electrification, sanitation, waste management, etc.) in several regions of Mauritania. After more than twenty years, it has recognized know-how in various development sectors, and has been implementing social accountability initiatives since 2008. These projects include, among others (1) ‘Jeunes dans la cité’ (Youth in the City), funded by the French Embassy in Mauritania to build the capacity of young people to carry out social accountability interventions; and (2) ‘Promotion of local development in the Moughataa of Monguel’, funded by the European Union to build the capacity of citizens of the Moughataa (department) of Monguel to implement social accountability interventions in the field of education. In addition to the TOME project, Ecodev is also engaged in a five year (2019–2024) $22.5 million integrated school feeding and nutrition program entitled ‘Food for Education and Nutrition for McGovern-Dole Children – The Future Belongs to Us!’ This is funded by the United States Agency for International Development (USAID) and the United States Department of Agriculture (USDA) and implemented by the Mauritanian government in collaboration with Counterpart International and local partners. The program targets 127,100 students in 209 schools in the Brakna and Gorgol regions and involves schools, parent-teacher associations, community members and NGOs. General Presentation of the TOME Project4 The TOME project was implemented between April 2017 and June 2021. It is a project that promotes social accountability to improve student learning. Its objective is to improve budget transparency in basic education by monitoring budget execution and assessing the quality of services provided at the local level, using online and offline social accountability tools. The main indicators of the project are: (1) increased collaboration between stakeholders at the local level to generate information that will improve basic education in targeted schools; and (2) increase the number of actions informed by citizen monitoring to improve budget transparency and education services. In addition to the students and parents’ associations of the forty target schools (twenty in Nouakchott Sud (Moughataa of Riyadh), twenty in Tagant (Moughataa of Tidjikja), the project also targets the National Federation of Parents’ Associations of the Islamic Republic of 4 Adapted from the Terms of Reference and the Final Report of the TOME project 15 Mauritania (FENAPERIM) and the Directorate of Basic Education of the Ministry of Education, which can use the information/data produced by the project to advise on the design of policies and improve school results. It is also expected that the Ministry of Finance and the Ministry of Economic Affairs and Development will use the results of the project to improve budget allocation and strengthen the control of educational expenditure. The project has three main components: 1. Capacity development and partnership for civic monitoring of basic education. This component includes the following activities: a. Develop and implement a communication and information strategy (including a web platform, regional advocacy workshops and radio programs) on collaborative engagement, social responsibility, budget monitoring and service delivery. b. Organize training sessions to build the capacity of basic education stakeholders (including partner CSOs and relevant government counterparts) on social accountability and service delivery and budget monitoring to be active and constructive actors in the partnership framework to be developed by the project. c. Develop, in collaboration with key partners (student parents’ associations, CSOs and government counterparts), participatory monitoring tools and guidelines for planning, implementation and monitoring (including data collection and analysis and feedback) to evaluate the implementation of the local basic education budget. d. Pilot participatory monitoring tools in selected schools and document pilot results, lessons learned and best practices. 2. Generate citizen feedback through social accountability processes and collaborative engagement with public sector institutions. a. Organize a series of meetings and workshops to implement a collaborative engagement framework with executive and parliamentary government counterparts to promote proactive disclosure of budget and service delivery information in basic education. b. Monitor and analyze the budget by: i. organizing sessions on participatory monitoring of basic education budget implementation in target schools with key stakeholders; ii. developing a user-friendly platform for the dissemination of information on the basic education budget. c. Carry out participatory monitoring cycles of education service provision through: i. information gathering sessions in target schools; ii. workshops and meetings to consolidate data and provide information to government counterparts; iii. follow-up sessions on information gathered, problems identified and actions agreed to address the issues/problems, with government counterparts and service providers (school stakeholders). 3. Facilitate knowledge and learning to improve the effectiveness of social accountability interventions and project management. 16 a. Knowledge sharing and learning: Develop mechanisms to capture and facilitate the exchange of knowledge between stakeholders, by: i. designing indicators for lessons learned during project implementation; ii. developing tools and mechanisms for learning and sharing among social accountability practitioners (including the Mauritania Boost database), and sharing these through the media, events/meetings and printed materials; and iii. implementing peer learning exchanges between the beneficiary, parent-teacher associations, local governments and the Ministry of Education. 4. Theories of Action and Change GPSA’s Theory of Action and Change5 “The Global Partnership for Social Accountability ’s (GPSA) Theory of Action supports a new generation of collaborative social accountability processes, which engage citizens, communities, civil society groups and public sector institutions in joint and iterative problem- solving to address poverty and improve service delivery, sector governance and accountability. One of the key lessons from the GPSA and global experience is that social accountability is most likely to be effective and scalable when it complements broader government policies and programs, including service delivery systems. By engaging with both civil society partners and the public sector, and by building on existing service delivery systems (e.g., programs, policies, channels, decision-making arenas, as well as the frontline), the GPSA addresses the need for, and capacity for, collective multi-stakeholder action. GPSA-supported coalitions develop the capacity to engage meaningfully and collaboratively in policy-making, implementation and service delivery processes. To build multi-stakeholder civil society-led compacts, civil society groups use GPSA advice and guidance, information on public sector reform efforts and national systems, insights into the practice of social accountability in relevant contexts, and other resources. A key outcome is that civil society partnerships and relevant public sector counterparts engage in collaborative social accountability processes that include individuals, communities and other groups in society, many of whom are generally excluded from shaping their own futures and participating in government. Collaborative social accountability can provide and strengthen platforms for collective citizen action. These multi-stakeholder pacts are a means of strengthening interactions that provide actionable information to decision-makers and change their preferences, incentives and ideas to achieve priority development goals at the local level. The GPSA expects these compacts to help address the immediate or systemic causes of local development priorities. They use social accountability mechanisms to address barriers to improving service delivery for all whether this is through systems strengthening and/or improving frontline service delivery. 5 from reference documents (GPSA, 2019) 17 In addition, World Bank teams support meaningful engagement between civil society and public sector institutions, including helping to identify service delivery entry points and social accountability opportunities for improving development outcomes. World Bank sector teams help open the door to engagement with governments by applying their unique experience in sector reform efforts. They can also help civil society groups identify concrete opportunities for community input into programs, policies and service delivery processes. Through this improved environment for engagement, civil society and public sector institutions are implementing collaborative social accountability processes that, unlike previous generations of social accountability, complement public management, service delivery chains and national systems with community-led actions. It is the synergy between the work of civil society and coalitions within the public sector -which can gain new information, insights, knowledge, legitimacy and resources through joint action  - that enables collaborative social accountability processes to contribute to more effective and sustainable development policies and, therefore, results. The GPSA expects that governments beyond individual projects will take up elements of the collaborative social accountability processes and their lessons. Over time, and through their shared experience, civil society, government and development partners will seek to adapt the knowledge gained from collaborative processes. They may sustain or scale them up through programs or policies that can be applied in other localities or sectors, sometimes beyond the duration of ASMG support. The nature of the GPSA grant is to make small experimental investments with the potential for scale-up and sustainability. The TOME project adapted the mandate of the GPSA to the context of service delivery in the education sector in Mauritania. By engaging a variety of stakeholders (FENAPERIM, Ecodev, NGOs etc.), including government at the central level and decentralized services, the project used collaborative social accountability mechanisms to overcome barriers to good quality education and student retention, with the aim of helping to improve service delivery in education.” TOME’s Theory of Action and Change The documents produced during the development of the TOME project do not explicitly mention a theory of change. General ideas are mentioned, however, and these are based on the GPSA Theory of Action (still under development at the time), and also, (as pointed out by one of the initiators of the project) on various ideas on the notion of human capital that were circulating at the time and would later be included in the 2018 World Development Report. An Iterative Approach In our interviews, Ecodev team members involved in the implementation of TOME were happy to talk about a theory of change that is iterative during the project, because it is not based directly on well-described previous experiences. TOME, and with it the idea of citizen and participatory committees at school level (and basic social services in general), appears to be a novelty in Mauritania. The road ahead is certainly not clear-cut for its initiators. 18 The general framework for action diagnoses four dimensions that are interconnected and explain the weakness of the Mauritanian education system; in brief and to avoid repetition, these are (1) pupils who are often absent, (2) material resources that are very limited, (3) teachers who are poorly trained and not very present, and (4) the governance of schools. The former head of the TOME project summarized the vision of the project as contributing to this fourth dimension. “The fourth factor, and this is where we come in as TOME, is school management. These schools do not have the autonomy to manage themselves. They don’t have the legal personality to be autonomous, and they don’t have a participatory management board that represents the stakeholders of the school at the local level. To have a contextual approach, you have to take a school at the local level, and this school and the factors that determine cycles and learning are largely absent at the moment. So that’s the analysis of the local pivot.” The Ecodev president added that the way in which this local analysis is carried out must be inclusive, laying the foundations for another pillar of the project: “I was going to say that the SPA – which is all parents – had to make a big effort, a big plea, to be accepted by the school management staff first, and then to get the school management staff to communicate in a timely manner with them about the service the school is providing, etc. [With the project, we] set up a more inclusive body. We had [in these bodies] schools, the students themselves, the parents, which gave legitimacy to this committee. […] In any case, this committee was more inclusive, more representative of the actors of education at the local level. ” The Theory of Change of the Baseline Study and the 2018 Reflection Document The project’s Baseline Study does not use the terminology theory of action or change, but it does refer to key ideas linking TOME project interventions (and the idea of social accountability) to outcomes. Just over a year after the project began, in June 2018, a document entitled ‘Analysis of the feasibility of introducing a social accountability process to improve learning for all Mauritanian pupils’ was produced by TOME project staff (ECODEV, 2018). It updates and deepens the considerations of the Baseline Study. This reflection document draws on the experience already gained, and the discussions initiated at the project level and refers directly to ‘Learning to Achieve the Promise of Education’ (World Bank, 2018b), in particular in its diagnosis of the causes of low learning achievement including its diagnosis of the causes of the learning crisis in the country, as well as another World Bank report, ‘Opening the Black Box: Contextual Drivers of Social Accountability’ (Grandvoinnet et al, 2015). The first part of the reflection document gives an overview of the situation, returning to the four dimensions already mentioned. The general assessment is clear, and not surprising given what has already been said: “all the dysfunctions at school level are the result of systemic obstacles of a technical and political nature that prevent the education system from putting learning at the center of its action”. At the technical level, the emphasis is on the coherence of the elements of the education system, while at the political level, the personal interests of the different parties involved in the education system are emphasized, which are not necessarily consistent with the objectives of promoting learning. Part two of the reflection document explicitly addresses the underlying theory of action ‘improving learning for all through social accountability’. This TOME Theory of Action builds 19 directly on the 2018 World Development Report (World Bank, 2017) by identifying three strategies: (1) better assessment of student learning to make more informed and strategic choices; (2) a close relationship between teaching and learning in the classroom based on evidence; and (3) an alignment of stakeholders’ interests. The paper explains before introducing TOME that “Citizen data, community engagement and social accountability, and multi-stakeholder coalitions can support the implementation of these strategies ”. Here we see the three pillars of TOME’s Theory of Action, which diagnoses Mauritania’s weaknesses in Table 1. 20 Table 1: Main Contextual Factors Influencing Social Accountability Enabling factors Adverse factors • Mauritania joined the Global Partnership for Social • Administrative culture and State action (MEN, MEF, Parliament) Accountability (GPSA) in May 2015 standards based on compliance • Government commitments stipulated in the with procedures, not results. National Strategy for Accelerated Growth and • Highly centralized management Shared Prosperity (2016–2030) on civil society, and lack of school autonomy governance and transparency, and increased decentralization. • Existence of 8,695 national NGOs and 55 • Weak capacity of civil society, in Citizens’ action (FENAPRIM, recognized international NGOs particular FENAPERIM and • Government commitments stipulated in the networks working in the education National Strategy for Accelerated Growth and sector Shared Prosperity (2016–2030) to strengthen civil society in order to make it more dynamic and more structured so as to contribute effectively to citizen CS Networks) control of public action. • Existence of FENAPERIM and civil society networks recognized by the MEN and involved in the field • Existence of a space for consultation and dialogue • Low involvement in the interface between civil society platforms and the state development and implementation citizen State- • Existence of an informal space for consultation and of public policies dialogue between FENAPERIM and the MEN • Lack of a law on the right of access • Relative access to available public information, to information (Citizens’ data) especially on education and learning • Lack of reliable information co- Information • Freedom of the press and full liberalization of the produced by citizens and the state audio-visual space on public services, especially education. • Acute awareness on the part of civil society • Lack of a collaborative engagement Civic mobilization (collaborative (FENAPERIM and networks) and the State (MEN, strategy between civil society Parliament) of the seriousness of the learning crisis (FENAPERIM and education NGO and the need to find a solution networks) and the state (MEN, • Strong common will among education officials at Parliament) all levels and those in civil society (in particular • Lack of multi-stakeholder FENAPERIM) to work together to find the best coalitions to mobilize stakeholders engagement) possible solutions to improve learning for all in the education system to improve learning for all Source: adapted (abridged) from: Analysis of the feasibility of introducing a social accountability process to improve learning for all Mauritanian pupils (ECODEV, 2018) Without repeating the details of the document ’s analysis, it is important to see that the TOME project aims to act directly on the weaknesses by establishing a framework for partnership and 21 collaborative engagement between the different stakeholders. Table 2, taken from page 17 of the document, summarizes this: Table 2: TOME Accountability Tools Key elements of social Local level Departmental level National level accountability State-citizen Participatory School Departmental Participatory National Advocacy interface Monitoring Committee Monitoring Committee Committee Information Participatory School Departmental Participatory National Advocacy Monitoring Report Monitoring Report Report Civic Mobilization School Action Plan Departmental action plan National Action Plan (Collaborative (budget transparency and Engagement) apprenticeships) In sum, and to conclude, we have an iterative theory of change, which will evolve over the course of the project, but which is primarily structured around three mutually reinforcing axes that link the actions of the project to educational improvement: • Strengthening the interface between the state and the citizen, through the establishment of committees. • The production of information, notably via the school’s participatory monitoring bulletin via a series of sheets - this is only possible when the interface has been strengthened. • Civic mobilization. As the paper points out “information or the existence of a state- citizen interface does not necessarily lead to action by policy-makers to improve education. Coalitions for social accountability between citizens and the state are often needed to mobilize and engage decision-makers to find effective solutions to identified problems together”. 5. Project Implementation The course of the TOME project can be summarized in two phases: April 2017 to April 2020; and April 2020 to the end of the project (this period is marked by the COVID-19 pandemic). Phase 1 This phase was characterized by the set-up of the partnership framework, its operationalization through capacity building and the provision of social accountability tools. At the level of the partnership framework, the project established (1) a Project Steering Committee whose role is to guide, facilitate and monitor the implementation of the project through a Technical Monitoring Committee and (2) a National Advocacy Committee (NAC) whose role is to support the project team in sensitizing decision-makers on the project’s approach and the consideration of its results for scaling up. Later, but still during this first phase, the advocacy committee was operationalized through the creation of a parliamentary group in the National Assembly to support 22 the project’s efforts to generalize the project approach to all schools in the country. The TOME project team provided the secretariat of the steering and advocacy committees . At the departmental and local levels, the project distributed the key actors (teachers, headteachers, students and parents’ associations, municipality, etc.) between newly created participatory committees - the Participatory School Monitoring Committees (Comité de Suivi Participatif de l’École, CSPE) and started capacity-building sessions on social accountability and related tools. The tools sought to mobilize all school stakeholders (headteachers, teachers, students, parent- teacher associations, elected officials) and involve them in the participatory diagnosis of their school, the definition of an action plan, and the implementation of this plan to improve learning in the school. These tools also enabled information to be passed on from the school to the departmental and national authorities, as well as to be fed back to the database, as shown in Figure 1. Comité National de Gouvernement – MEN (National) Plaidoyer Annuel Projet TOME Comité de suivi participatif départemental Services déconcentrés (Régional) Semestriel Ecoles (local) trimestriel CSPE CSPE CSPE Figure 1: Information Flow Mechanism Source: Baseline Study, TOME Following the operationalization of the various committees, the project team noted the absence of a link in the partnership framework, particularly at the regional level. They then set up this link, known as the Participatory Departmental Monitoring Committee (Comité de Suivi Participatif Départemental, CSPD). 2018 was marked by the operationalization of the CSPEs and CSPDs through the implementation of the CSPEs in ten schools in Riyadh and ten schools in Tidjikja as well as in the two CSPDs in Riyadh and Tidjikja. The CSPEs and CSPDs in these schools received a two-day training on the concepts of social accountability, social accountability tools and the participatory approach by two expert trainers. Subsequently, the project team assisted the various CSPEs in diagnosing their school and drawing up their action plan. 23 In the academic year 2019-2020, the same process was conducted by the project team for the remaining ten schools in Riyadh and the remaining ten schools in Tidjikja, thus completing the process in the forty target schools of the project. In addition, during the same year, a school budget was allocated to the Fodé Diaguily school in Riyadh in the wilaya of southern Nouakchott (via a partnership with a local association). This unprecedented experience has led the project managers, Ecodev and its partners (FENAPERIM, the municipalities of Riyadh and Tidjikja) to want to generalize this experience. In this context, a project consultant drew up a procedure’s manual for the school budget. Ecodev, FENAPERIM and the communes of Riyadh and Tidjikja mobilized contributions for establishing a support fund for the quality of basic schools, known as FAQEF. However, this fund did not see the light of day, as we will explore further later. Phase 2 This 2020-2021 phase is marked by the onset of the COVID-19 pandemic. It was characterized by the closure of schools. Indeed, the emergence of COVID-19 in Mauritania from 14 March 2020 and the measures taken by the government to close all schools in the country, to isolate regions from each other and to ban public meetings had a direct impact on all project activities. To reduce the impact of COVID-19, the project team (1) set up WhatsApp groups at the level of all participatory monitoring committees to continue the exchange from a distance and (2) accelerated the implementation of the TOME digital platform, Medrassety.net. This platform has been implemented in all TOME schools and allows exchanges between all local stakeholders and the project team. At the national level, the TOME project’s experience with COGESs contributed to and/or influenced government policy by setting up COGESs in all public schools in the country, capitalizing on the experience of participatory monitoring committees in forty schools in the wilayas of Tagant and Nouakchott South. The COGESs were an opportunity for the TOME project to share its experience with the Ministry of Education, thus impacting on education policy. 6. Achievement of the Project’s Objectives In this section, we address the first evaluation questions: did the project strategy contribute to the intended outcomes and impacts? If so, for whom, to what extent and under what circumstances? To answer these questions, we describe the project’s achievements, paying particular attention to the adaptations and the different actors' perceptions of the process. The project’s monitoring and evaluation matrix is useful for recalling the project’s objectives, but our ambition here is not to monitor the activities in detail (the Final Report of the Beneficiary of the GPSA Grant produced by Ecodev already does this work; ECODEV, 2021). We aim to understand what transformations took place and how they happened. This section is structured in three subsections that correspond to the three intermediate results that Ecodev had defined as necessary to achieve its final result: to improve the transparency of the basic education budget by monitoring and evaluating its implementation and quality through social accountability tools in the target regions of Mauritania. 24 Capacity Building and Partnership Framework for Civic Monitoring of Primary Education The TOME project trained 255 actors in social accountability methods. It engaged a total of 260 people in participatory monitoring committees at the local (school) and regional levels, as well as at the national level with the TOME Project Steering Committee and National Advocacy Committee (ECODEV, 2021). Table 3, taken from the Final Report using the project indicators, shows this information: the target values are largely exceeded, and the interim reports indicate that these training sessions are taking place within the time frame foreseen by the project. Table 3: Achievement of Intermediate Outcome Indicators 1 Selected intermediate outcome Unit of Measurement Cumulative Values indicators Target achieved Values in in 2021 2021 Intermediate outcome indicator: Number of actors trained 240 255 Mastery of social accountability using social accountability tools tools Intermediate outcome indicator: Number of actors structured 250 260 Level of structuring and and involved in the engagement of stakeholders in the different committees social accountability process. This training is described by the different actors – teachers, parents, headteachers and mayors –  as “important” and “inspiring” and took place, for the most part, and as planned, before the COVID-19 related school closure period and in particular during the project conference in October 2019. The project platform, Medrassety.net, hosts the training modules, including specific tools such as citizen scorecards, participatory budgeting, budget planning, and expenditure tracking. Creating a Space for Dialogue and Collaboration Whether they are from students parents’ associations (SPAs), schools, or municipalities, the actors in the field all describe the TOME project and the training received as opening up new perspectives for collaboration. The drawings created collectively during the focus group discussions illustrate the emergence of a collaborative space: in five out of six drawings, the actors who appear at the center of the school include the CSPE, the SPA, and sometimes even civil society (see Annex C for an example). They echo the content of the discussions that followed and emphasize a key change: with the TOME project, more actors feel (and are) involved in the school, find a place in it, and have established a healthy relationship with other actors involved in the school. A headteacher in Tidjikja explained during a focus group: “The TOME project has trained us on how to value the relationship between the different educational actors and the role of each one. This training has made our relationship fluid and homogeneous.” 25 TOME enabled a range of local actors to rethink their role at the school and their relationships. For SPAs whose members were often untrained, the project is described as an essential change – as one SPA member in Tidjikja reports:“We were absent, we knew very little about the role that has been assigned, and we were not fulfilling it properly. With the intervention of the TOME project and the various training sessions we have received, we are now aware of our role, and we are carrying it out properly.” The people we met were unanimous: the relational space was much more limited before TOME's intervention. The reasons why things are stuck are to be found at different levels. As in many other examples of social accountability, service providers (in this case, teachers and headteachers) do not want ‘users’ and civil society (in the broad sense of the term, in this case, parents) to interfere in their affairs. The fear is often that these actors will become ‘policemen’ who control them. Actors traditionally located outside the school, such as the municipality or SPAs, are also unsure of how to engage in a different relationship with the school, and within the school there is no obvious model for rethinking this relationship either. TOME allowed a new space to emerge. It has at least two facets: first, there is the formal space, i.e., the participatory monitoring committees, and then there is a less formalized space which is simply the new connections created between parties. There is a clear relationship between the two, which are mutually reinforcing over time. Still, it is essential to emphasize that the efforts to create a formal space through training and meetings ultimately allow this informal space to emerge. The local participatory monitoring committee (CSPE) is, by far, the most emblematic achievement of TOME from the point of view of the field actors. “I think that what worked best in TOME was the setting up of the management committees and the many training sessions they received”, explains a teacher. The Final Report of the TOME project (ECODEV, 2021) describes the committee as follows: “The aim of these committees is to establish school management based on a participatory approach. Their main mission is to improve the quality of learning, increase access to school, fight against disparities, manage textbooks and school supplies, and contribute to the monitoring of attendance and the regularity of teachers ’ presence in the school. These supervision and monitoring committees include the territorial administration, communes, parents and student representatives. ” In practice, CSPEs are a powerful instrument to bring new actors into the school smoothly, as different parties are present in the committees. The executive secretary of Ecodev also explains that the model of a committee composed exclusively of parents and trying to play the game of social accountability did not work: “they had to make a lot of effort, a lot of advocacy to be accepted by the staff in charge of running the school first and then to get this staff to communicate in a timely manner with them on the service provided by the school, etc. ” A committee with diverse membership solves many of these problems. The question of the level of functionality of CSPEs can be approached from different angles, and we will return to this later in the report. However, two elements should be noted at this stage. 26 Firstly, some of the participants in the focus groups mentioned local problems (at school) which have local responses and require consultation. This consultation takes place within the CSPEs, which therefore discuss exact and concrete subjects, as the president of the SPA (from the Fodié Diaguily school) explains: “I am part of the school’s supervisory committee, and we often meet to discuss ideas related to teaching, the behavior of pupils, and the provision of tools and food for pupils.” In at least some schools, the CSPEs are functional: CSPEs take up real problems and deal with them. Secondly, and related to this point, while we find places where committees talk about frequent meetings that seem to happen naturally, other committees do not seem to meet at all. Others seem to be more dependent on external actors to organize their meetings, as a teacher in the municipality of Tidjikja confided to us: “meetings (usually four meetings a year) are organized between the school management and the parents under the supervision of the departmental inspectorate”. By taking a typical indicator of the functionality of a committee (do they meet), we see here a significant heterogeneity of situations that the rest of our report will confirm. Finally, the communication space created extends beyond the committees; communication is described as “direct and daily” by one SPA member, confirming the idea of an informal space developing. Communication can sometimes also take on the more formal character of written transmission if there is a “problem that the Mairie [Municipality] or the inspection needs to take care of”, explains a teacher from another school in the municipality of Tidjikja. The description of these communications is similar at the level of the school principals, as the principal of the same school explains: “I routinely communicate with the people mentioned in the drawing. They are partners in the school. It is very often in writing for official partners and through meetings for immediate partners such as SPAs, civil society.” New School Dynamics The focus groups, interviews, and reports of the TOME project mention several dynamics initiated by the establishment of participatory monitoring committees that can be grouped into three categories: (1) needs identification; (2) community mobilization; and (3) networking. Firstly, the tools prepared by TOME and the new dynamics at the school level will enable the implementation of a participatory diagnosis, i.e., the identification and description (and quantification) of needs and resources. Even if these needs were already felt and sometimes formulated by some of the actors, the setting up of the CSPEs will make it possible to reach a consensus and push the CSPE members who can act to do so. The chairwoman of one SPA explained: We identified together (including the pupil, the smallest among us) the needs of the school and made a study which was financed by the commune of Tidjikja. The rehabilitation work was carried out under our supervision, and the result was satisfactory for everyone. This is proof that this idea of transparency and discussion of budgetary information is applicable in our country.” Secondly, some research participants talk about activities that are not strictly speaking management or planning issues, but activities carried out directly and physically by the parties involved in CSPE. These are activities that are made possible because a new link is created, for example, as this teacher explains, the registration of students – an activity for which the TOME 27 project has provided guidance to schools: “This cooperation [the participatory monitoring committee] resulted in the registration of pupils whose families were deported from the Tevragh Zeina area (a residential area in Nouakchott) to areas adjacent to the school.” Thirdly, in some schools, the committee is what will enable the development of wider activity, using the network of CSPE members – which by definition is wider than just the teachers’ network. One school Headteacher explains the value of this networking as follows: “Yes, I participated in the activities of the school’s Participatory Monitoring Committee, and I also participated in those of the National Advocacy Committee, which convened all the partners. The graduates of National High School 1 made a grand gesture that should be emulated by other organisations. This gesture is a financial donation to a peripheral school (Diaguilly school), which is one of the schools targeted by TOME. Before this financial gesture, it had a low success rate of 2.5 percent in the first year of secondary school. The graduates of National High School 1 decided to donate MRU 500,000 [just under $1,400] to help the Grade 6e teachers and the supervisory committee (the inspector, pedagogical advisors, and the school director). And at the end of the 2021 school year, the school achieved a pass rate of 18 percent.” Beyond the Theory … the Pandemic At the time of our report, 80 percent of schools, including all pilot schools in Tagant, had publicly shared a school action plan on the project platform, which can be a tool for social accountability. We have so far given examples of positive dynamics instilled by the TOME project. However, we also find schools where the situation is much more mixed in terms of putting the TOME training into practice - probably partly due to exceptional circumstances. The project's third year was supposed to be the year of implementation of these training sessions; however, it was severely disrupted by the closure of schools and COVID-19 pandemic measures, which prevented field monitoring of the capacity-building process by the project team. The executive secretary of Ecodev summarizes the impact of the pandemic on the progress of the TOME project: “If the first two years were the years of communication, awareness-raising and the setting up of bodies at local and even national level, then the third year was to be the year of experimentation of the platform. Because it was the platform that was supposed to allow this information to be passed on in real time, i.e., the school directors or the parents, when they entered information on the platform, this information was accessible to the president of the advocacy committee at the assembly level, to the president of the national federation of parents, etc. So, we couldn’t set up the platform. Because of the pandemic, the closure of schools, the demobilization of actors, it was too much for us to do a real experimentation of the platform.” The president of Ecodev agrees: “The pandemic forced us to telework, which was impossible. Nevertheless, with the partners, we set up mechanisms that allowed us to keep a link between the teachers in the WhatsApp groups, the pupils and the parents. But it was more of a good dynamic to go quickly, to experiment, to have feedback quickly to make proposals too eventually, or to improve the texts and test one’s knowledge, especially the internal regulations and the statutes after the decrees.” 28 However, long-distance contact was maintained via WhatsApp groups, allowing exchanges and reflection to continue within the various participatory monitoring committees during the lockdown and school closure. The Medrassety.net platform was also launched at the same time, to maintain digital contact despite the above difficulties. How effective were these coping strategies? It is difficult to answer this question fully, but two important points should be noted. In terms of the level of effectiveness of the CLSPs, we, unfortunately, did not have access to data that would allow us to quantify broad trends, such as how often they met and what they discussed. With very few exceptions, the minutes are not publicly available (and, in fact, often seem not to exist). In the six schools we looked at more closely, the situation was variable, to say the least: in some schools, the committee is well established and fully functional, while in others, it has simply never met. It is difficult to understand the differences in performance between schools based on such a small sample but let us note here one element that is clear: of the six schools, the school that had only benefited from the training in a second phase is the one in which the committee seems to be least effective. In this school, the committee barely met, and the stakeholders do not seem fully committed to the project. In the other five schools, the situation is more variable and generally more encouraging. Nevertheless, almost everyone we met insisted that change takes time and that the project was very short, especially with the pandemic. Thus, the teacher representative of a focus group in a school in Tagant explains that: “The remark I made was that the TOME project is very limited in time. It would be better if it was sustainable and if all actors could benefit from refresher training.” Another source that confirms the wide variability in the effectiveness of the second phase of capacity building is the Medrassety.net platform. The impression is of a missed opportunity, despite the urgent training of the various parties targeted as the main users of the platform. The graph below shows a tool that is only partially complete and partially exploited: the tool for noting pupil or teacher absences did not seem to be used, while basic information such as class size or even the headteacher’s contact details were sometimes not included on the site. Possible explanations include the weakness of official school data in Mauritania combined with a low level of familiarity of stakeholders with both the IT tool (only 21 percent of the Mauritanian population used the internet in 2017, according to World Bank data) and the principle of data collection and sharing. 29 information sharing action plan contacts parents' office contacts monitoring committee contacts teachers number of pupils per class number of pupils absence of students* absences of teachers* contact details director 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Tagant Nouakchott Sud Figure 2 Indicators from Medrassety.net The platform looks more like a static information site than a tool for exercising a form of citizen participation. Of course, it is essential to bear in mind that the implementation of the platform suffered enormously from the pandemic, as the president of Ecodev explains: “[We lacked] the time to test it, to see what we needed to improve it, to make it more accessible, even in terms of IT development. I’m sure there is a lot to do, but, unfortunately, we haven’t had the time to develop the tool.” However, the platform's experience is not entirely negative: nine schools (almost a quarter) stand out with complete information because they make use of the platform to communicate. Public communications are interesting and show the different uses and audiences. One set of messages aims to inform the population of the progress made, mainly in terms of improving human resources, for example; this message posted on 14 June 2021 (in Arabic) “We are increasing it daily, but some days the absence is zero for teachers and students, God be praised” or this one, “yesterday, Tuesday, a memorandum was issued that some teachers have been transferred to our school”. The second category of messages is more informative, announcing visits or exams. The third and final category of messages is slightly more intriguing, denouncing material degradation in French and Arabic, for example, “lack of classrooms. Lack of tables and benches. Drinking water” or “the big door of the school wall must be replaced because of its fatigue due to the accumulation of sand on it before the Riyadh municipality removes it at the beginning of the current school year, may God reward them with good ones”. In a few cases, it is even about denouncing, sometimes by name, absent teachers. We have no information on the people behind these messages, but the aim likely is to use the platform to reach decision-makers. This is not to overstate the importance of the messages, which remain few, but the tool seems to 30 have some potential. All nine schools were still posting messages after the end of TOME, but not in the new school year. Interestingly, it is not clear whether the time of the training, i.e., whether the school was one of the schools that received the training in the first group (2018) or in the second group (2019), matters a lot, as confirmed by the Figure 3. partage d'information plan d'action contacts bureau de parents contacts comité de suivi contacts enseignants effectif par classe effectif scolaire absence des élèves* absences des enseignants* coordonnées directeur 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% second training group first training group Figure 3  Comparison by Implementation Time Source: Medrassety.net The main difference is in the sharing of information, which is obviously a crucial aspect, but in general, it is possible to invalidate the assumption that using the platform is only a matter of committees being in place for some time. Budget Management Tools Without a Budget? Beyond the pandemic circumstances, there is another major problem in moving from theory to practice on the (local school) budget issue. The project includes the idea of developing inclusive and deliberative participation in the school budget. The TOME project trains local actors, especially members of the CSPEs, in participatory budgeting. It also provides them with tools to facilitate this management, such as the invitation to open a bank account for the school’s participatory budget. However, almost all the actors interviewed expressed doubts about the realization of the participatory budget idea. There is no doubt about the merits of the idea or the principle – on the contrary, all actors welcome the idea of participatory and transparent budgetary governance. A departmental inspector explains, for example: “The idea of the project is a good one, I mean the clearer and closer the budgets are to the schools the better the management is. For example, there are so- 31 called school budgets, but they are not up to the mark, and we must involve the parents, the school management in the management of the budget for more transparency and reliability. ” At the level of the municipalities, for whom the idea of a participatory budget also means sharing their decision-making power over the school, the principle seems to be fully understood and valued, especially in Tidjikja, as a representative of the mayor explained in a focus group discussion: “For the idea of budget management is very useful, especially what we have learned in the workshops and seminars organized by the municipality and others. These training sessions allowed everyone to know their role. They showed us the methodology of managing budgets.” The problem is the validity of the exercise, as there is no increase in the directly managed local budget during the project period. Except for one school, Fodé Diaguily, which benefited from a pilot budget (MRU 200,000 from Ecodev and MRU 50,000 from FENAPRIM in February 2020), and despite the efforts of the TOME project, most schools remained without a significant local budget. In other words, the impression shared by many actors on the ground is that there is not a substantial budget to manage, which makes part of the participatory governance exercise (via CSPEs) confusing and even frustrating. This tension is, of course, a function of the exact budgetary situation of each school, but the trend is very clear. One point illustrates the difficulty in which the CSPEs find themselves: the bank charges that accumulate on an account that is not funded and, in fact, not or hardly used. The president of a SPA shares his experience, which echoes some others: “I once attended a meeting of one of the committees with the school director, but so far, we have not received any funding, and we have opened a bank account which is now suffering from accumulated taxes.” This leads to two attitudes. Some of the people interviewed, especially teachers who have benefited from numerous training courses and who have seen various aspects of their work change, remain confident in the approach and seem to show a certain amount of patience, as in the case of this teacher representative who spoke during a focus group in Nouakchott Sud: “I must say that the idea itself is excellent. The TOME project is the project responsible for the transparency of the management of school budgets, and as far as I know and as a teacher in this school, I have not seen any impact on that. It should be noted that school supervision committees have been set up at school level, but these committees do not meet.” Another group of people we met, especially those who were a little more distant from the day- to-day activities of the school, questioned the principle of the committee itself in the absence of a budget to manage. In the same focus group, the president of the SPA explained: “It is the CSPE that has the right to manage the school ’s budget, and this committee needs to be supported to be able to run the school properly; as for setting up a committee by itself and providing it with documents and papers, it seems to me that it is useless. There should be tangible support for the school to function successfully. ” The schools that seem to be more successful in terms of management and experience of transparency and debate around a budget seem to be those given a budget to manage or material resources, such as soaps and masks during the COVID crisis. For example, a teacher, member of the committee at Fodé Diaguily school (the only one that was supported at the budgetary 32 level), speaks of his experience: “In this context, I would like to point out that we have had a practical experience thanks to the donation made by the alumni of the National High School, represented in a substantial budget that has allowed us to raise the standards of the students and to increase the success rate at our school.” At Ecodev, the realization of the importance of providing a budget for schools comes early in the project. The TOME project team started thinking about it at the beginning of the implementation of CSPE. In September 2019, the project recruited an external consultant to help it better understand the situation, i.e., and to quote the terms of reference: “to determine the typical budget likely to ensure the functioning of the school in terms of pedagogical, administrative, financial and human resources management ” and “to determine the average threshold of an operating budget for a basic school with a complete pedagogical structure”. The expert’s Final Report (PAQEF, 2020) puts forward a three-phase scenario that would gradually aim to increase the decentralization of school financing to solve the lack of school budgets. It also proposes a standard budget structured around key objectives, decided and evaluated locally (by the CSPE) and budgeted by the CSPE, and formalized in a manual of procedures for the ‘Basic Education Quality Support Fund’ (Fonds d’Appui à la Qualité de l’Enseignement Fondamental, FAQEF). These action plans and budgets will be produced in all project schools but will not be funded during the project - except for the Fodé Diaguily school, which is used to demonstrate the concept's validity. According to TOME’s leader, the FAQEF could not be set up because the project's budget was too small. Moreover, World Bank procedures did not allow third-parties (schools) other than Ecodev and FENAPERIM to manage project funds, which meant that even the reallocation of funds to schools by Ecodev was not possible. However, this attempt to finance school action plans has created a positive dynamic in Tagant, as it has been taken up by the commune of Tidjikja that decided to set up a fund to finance action plans for the schools in this commune following the procedures laid down by FAQEF. The TOME project staff mobilized to provide technical support for these schools to implement their plan. Nevertheless, Tidjikja’s fund remained very limited (in budget and coverage), and even there, FAQEF and the planning exercises remained largely aspirational for the project's duration. They generated high expectations which, not being fulfilled in the end, seem to have turned into frustration. At the level of the managers interviewed, particularly at the level of the Ecodev chair, there is a clear perception that the budget is perhaps the most sensitive element of the whole project. The Final Report notes that: “The third major lesson comes from the success of the Fodé Diaguily school, following the budget allocated to it, and its positive impact on the quality of learning of the pupils in this school. For the project team, this experience is a lesson to be extended to other schools monitored by the TOME project in order to improve their performance.” The president of Ecodev explains the situation better than we can, and we conclude this subsection with his words: “The debate is about the school budget, the education budget and how it is managed. If we had this ambition at the beginning to say, at the end of the project, we should have an approach on the national budget, including school budget and more detailed 33 debates. […] So the budget is not there. We had this ambition, but unfortunately, we couldn’t make any progress on it […] We said at least let’s have the courage to say to each school, we’ll give them a million Ouguiyas a year [the equivalent of about $27,500], and we’ll make it available to them. Today, we are far from that. We are very far from that today, even though it is commonplace to say that each school receives a small budget. It is earmarked in the national budget, and each school receives this amount; each school has these millions of Ouguiyas at the operational level to improve its functioning, quality, and infrastructure. Maintenance can also be at the communal level.” Summary and Conclusion On the first indicator, ‘capacity building and partnership framework for civic monitoring of primary education’, the success of the strategy adopted depends on the perspective adopted: if the expectation is that social accountability dynamics will be effective and lead to significant educational change in each school, then a lot of work remains to be done, if, however, the idea is just to demonstrate the viability of a concept, then the picture is much more positive  -  especially given the difficult circumstances. It seems difficult to think of an approach that would have been more effective in setting up a new dynamic; the interviews and secondary data show that the sensitization of the main stakeholders and the initial training worked very well (adapting to the context of the pandemic was, however, challenging - and very much in line with what has been reported in other parts of the world). Many field actors were highly enthusiastic about putting into practice the social accountability principles that had generated great enthusiasm in the training and awareness-raising phase. Ultimately, one crucial element clouds the picture: the budgetary issue. Despite its efforts to seek funding and support for the conceptualization of school action plans, the TOME project could not ensure that the schools and committees it trained were allocated additional resources. However, these financial resources would have enabled the new dynamics initiated in the schools to be genuinely implemented - tension results from this. At the level of the project’s Results Framework, there is no problem: the committees exist, the training sessions have taken place and the stakeholders explain that they have developed their knowledge. The Results Framework indeed stops at the implementation of a process (and the project stops relatively quickly after the training sessions), the risk and the tension we point out are further down the line (see Section 8). Generation of Citizen Feedback through the Social Accountability Process and Collaborative Engagement with Public Sector Institutions Consultant: “What makes you most proud of this project?” TOME project manager: “It is the change in the configuration of school management committees that was decided by the ministry and which is a very strong sign, or at least a clear direction, that the public authorities want to give their school management policy an effective involvement [of the population and teachers]. This is something we are very happy about.” The second intermediate outcome is mainly at the municipal, departmental and national levels. The project indicators are again generally met and exceeded, as shown in the table below. Consolidated reports were produced at the departmental and national levels and the TOME 34 project noted a series of initiatives as a result of the citizen feedback, thus attributing these initiatives to the efforts of the TOME project and the dynamics it creates on the ground. Table 4: Achievement of Intermediate Outcome Indicators 2 Selected intermediate outcome Unit of Measurement Cumulative Values indicators Target achieved Values in in 2021 2021 Intermediate outcome indicator: Number of consolidated 7 7 Stakeholder co-produced reports co-produced information on budget transparency and quality of public education service Intermediate outcome indicator: Number of decisions taken 8 10 Responsiveness of government on the basis of citizen stakeholders based on citizen feedback to improve budget feedback. transparency and quality of public education service Questions of attribution are always complex and require caution in the absence of an elaborate framework (which we do not have). The Final Report notes, for example, a series of decisions taken based on citizen feedback, namely (1) the decree creating the COGESs; (2) the decree on truancy; (3) a new regulation on truancy; (4) the establishment of a monitoring mechanism for the COGESs; (5) the decree on the competencies of primary school teachers; (6) the regular evaluation of teachers to determine their level in Arabic, French, and mathematics; (7) the increase in resources allocated to education in the 2020 budget (+10.6 percent compared to 2019); (8) the introduction of distance learning; (9) the launch of 1000 COGESs; and finally (10) the introduction of a new five-year (2019–2024) integrated school feeding and nutrition program with partners Counterpart and Ecodev. To what extent are all these changes attributable to the efforts of the TOME project? The interviews reveal a series of linkages and allow us to trace the processes, but it is also useful to briefly consider the overall chronology of the project in perspective with these different changes. We do this in Table 5. Table 5: Timeline of the TOME Project and Changes in the Education Sector Significant Political and Administrative Changes in TOME Project Mauritania (affecting education) [→ imputed to TOME in the Final Report] 2017 April: launch 9 Nov: Implementation agreement with FENAPRIM 2018 —operationalization of the DSCCs —operationalization of CSPE in 20 schools 17 Jan: partnership agreement with the ministry June: analysis paper (theory of change) 16–18 July: participation in the validation of the education sector development project 2019–2021 2019 35 —operationalization of CSPE in other 20 schools → 1 March: decrees 2019-038 (absenteeism), 2019- 039 (COGESs), and 2019-040 (teacher levels) → five-year (2019–2024) integrated school feeding and nutrition program (Counterpart) → end of year: 2020 budget (+10.6% education) 22 June: new President elected (with a pro-education platform) 16 Oct: Resource mobilization workshop, launch of the idea of a Fund to support the quality of basic education (FAQEF) 5–10 Dec: NOC meeting 13 Dec: submission of problems Wali Nouakchott South 2020 5 Feb: first meeting of the Steering Committee 22/23 Feb: capacity-building days for parent-teacher associations 14 Mar: closure of schools (COVID crisis) → Setting up distance learning Setting up WhatsApp groups Implementation of the Medrassety.net platform → 13 July: launch of 1000 COGESs (ministry) 2 Sept: post-Covid plan, MRO 40bn for education 2021 June: closing Nov: national consultations The chronology allows us to highlight several things. Firstly, we can see that the project had to adapt to the closure of schools due to COVID-19 and reinvent itself by introducing WhatsApp groups and promoting the Medrassety.net platform among other things. We also see that what is possibly one of the project's greatest successes, the launch of COGESs, comes relatively early: the key decrees are taken in March 2019, when the CSPEs had just been implemented in twenty schools. Finally, it is also appropriate to add some nuance to the relationship between citizen feedback and the decisions taken, in view of the chronology and nature of the ‘decisions taken on the basis of citizen feedback to improve the transparency of the budget and the quality of the public education service’: a. Some of the decisions documented in the Final Report appear to be the direct product of an advocacy effort (in a positive sense). This advocacy is partly based on the experience of the citizen feedback, but the nature of the process is more complex, as explained earlier in this report. This is primarily about the COGES decree, the regulations on their operation, and their launch. To some extent, the evidence on teacher absenteeism could fall into this category, but it is difficult to establish this with certainty - it is not clear that it is primarily TOME’s advocacy and citizen feedback that leads to this change. b. The measures on absenteeism, teacher attainment and the nutrition program appear to echo wider issues, with probably other parties involved in advocacy. This is not to say that the TOME project did not matter here; it seems clear that some of the actions taken do reflect some of the learning and feedback from the project, but attributing these measures as being primarily due to TOME is probably risky (but again, this does not reduce the key contribution of TOME to these measures). 36 c. Finally, the TOME report also includes measures that are somewhat different. These are mainly in response to COVID, notably on distance learning or the increase in the education budget between 2019 and 2020 (an election promise). Here, unlike in the other two cases, it is not possible to say that the citizen feedback from the TOME project is a leading cause or even a contribution to the existence of these measures (in all likelihood, there would have been distance learning even without the TOME project). Rather, the way in which these measures manifest themselves has been influenced by the TOME project and the citizen feedback on, for example, how to maintain a link with parents and pupils when schools are closed. Putting Citizen Feedback on the Agenda Let us now look at the strategy that TOME used to put social accountability issues on the agenda in education. Table 6 shows the main interventions attributable to the project in this area and the key interactions that took place between the project and municipal, departmental, and national officials. Three different categories can be distinguished: (1) training and awareness-raising activities organized by the TOME project; (2) activities that TOME supported or facilitated, but under the leadership of a third party, typically a governmental actor at one of the three levels of power mentioned above; and (3) measures and initiatives taken by the authorities in response to TOME’s advocacy work. Establishing a causal link between these measures is not necessarily obvious: of course, the authorities are the only ones to decide, but these decisions are based on an experience and a conviction that TOME has helped to articulate. Table 6: Main Actions to Generate a General Pro-Social-Redeemability Environment Level Key Actions Municipal Training: official launch of the project in January 2018 Advocacy: ministry regulation allowing headteachers and COGESs to sanction absent teachers, mechanism for monitoring the operational performance of COGESs through the absenteeism rate of primary school teachers. Departmental Training: Departmental Participatory Monitoring Committees (CSPD) set up in Nouakchott South and Tagan. Facilitation: Production of 7 consolidated reports by these CSPDs National Training: official launch of the project in January 2018 Facilitation: Launch of the national program for the establishment of 1000 COGESs on 13 July 2020 by the Minister of Basic Education and National Education Reform from Tidjikja on the basis of the TOME project pilot experience. Advocacy: The Council of Ministers adopted Decree No. 2019-039 of 1 March 2019 on the creation in each public school of a School Management Committee (COGES) Decree No. 2019-038 of 1 March 2019 regulating the absenteeism of teachers in public primary and secondary schools 37 Decree No. 2019-040 of 1 March 2019 instituting a strategic review of the skills of primary school teachers in order to ensure a minimum level of competence of basic school teachers. Education sector expenditure has increased by 10.6 percent compared to the 2019 budget, representing 23.9 percent of the total state budget. The bulk of this subsection will explore these advocacy strategies, which have often relied on training/sensitization as a first step and facilitation as a second. Each of the elements of this picture counts, of course, but it is important to highlight one in particular, the establishment of Management Committees (COGESs) whose aim is to ‘manage school resources in a rational and transparent manner in order to achieve results commensurate with the resources invested ’ (TOME and Ecodev 2019). The COGES has, technically, a smaller scope of action than the CSPE, which is more of a monitoring body. COGES and CSPE will coexist during the TOME project, but there was little doubt in the minds of the people we met that COGES is indeed the new iteration, decided by the ministry (and not by an NGO) of CSPE. The decree setting up the COGESs, followed by the national program for setting up these COGESs, is an undeniable and important victory for the TOME project, which invites a very positive assessment of the achievement of the objectives project had set itself. The approach gives legal existence and creates a framework that facilitates the implementation of the COGESs, but it is also accompanied by a desire to make the implementation of the COGESs concrete and effective, which requires resources deployed on the ground. The national program launch on 13 July is an important first step. The president of Ecodev is optimistic: “The Ministry of Education has officially requested that the TOME approach be duplicated and generalized to all schools. And so it is in this sense that a document from the parents ’ federation was sent to the minister [ …] It is quite encouraging. It’s very good.” Before trying to understand how TOME has achieved this remarkable performance, it is worth noting that the people we met explained that they have more good news about the medium- term consequences of the project, notably through projects (potentially) to be financed by USAID and through the new project supporting education set up by the World Bank, PASEB II – in both cases, there is interest in supporting the functioning and development of COGESs throughout the country. For its part, the National Federation of Parents’ Associations talks about setting up seminars on transparency and action tools for its members. It links the TOME project directly to this approach (not yet implemented), notably by using parents with TOME experience as trainers for other parents. Strategic and Early Engagement with Key Stakeholders As soon as it was set up, the TOME project went to find the political decision-makers to involve and associate them with the project and, in particular, the constitution of the CPSEs. As the Final Report explains, the idea of “structuring local actors in Participatory Monitoring Committees at departmental and school level” will eventually emerge as “an effective way of concretizing the collaborative engagement of civil society actors (especially parent-teacher associations) and government (especially education authorities) in identifying and solving problems related to budget transparency and education quality together ”. 38 Rather than trying to convince the national level (the ministry) directly of the validity of the approach, the project worked from the ground up and from the concrete implementation of CSPEs. The former director of the TOME project was explicit on this point, emphasizing that this strategy would give him something to show for, and that this was a level at which the work was easier and at which he had more allies: “So I had the right combination … first of all because at the local level it’s easier. And by chance, I found people I know at the local level, among them some of my former constituents. So that counts. At the local level, it ’s not too big, it’s a school. Finding local actors who agree with me, including a mayor I know, has already allowed me to have the support of well-known personalities because schools can’t do anything without government approval. […] At the level of the ministry, we tried for months and ended up with an agreement to have legitimacy, so that they would accept that we could intervene in the field … even though we had started the project before we signed.” After the first year of the project, however, one step seems to be missing, as the project managers explain in their report: “In 2019, it was noted that there is a missing link in the structuring of the actors that could jeopardize the effectiveness of the collaborative process between the actors if it is not filled. These are the actors at the regional level: the Wali, the President of the Region, the Regional Director of National Education (DREN), the Regional Head of SPAs. To fill this gap, a process of consultation was initiated that led to the creation of a new structure called the Comité de Suivi Participatif Régional (CSPR), which brings together all these actors and monitors the targeted schools in each Wilaya (Nouakchott Sud and Tagant). This is one of the lessons learned in 2019 and has improved collaboration between stakeholders at the regional level.” At the same time, the project continues advocacy work at the level of national representatives, particularly those with interest in education (which is emerging as a priority for the presidency, as explained earlier and later in this report). The Project Steering Committee is built with support at the level of the National Assembly, as Ecodev’s executive secretary explains: “The advocacy committee was also present in the education groups in the National Assembly. They were themselves in a position to go beyond what was being done in terms of education policy at the national level, and who were therefore very interested. And so everywhere on the ground, they came down and did missions with us. That’s the difference with the classic steering committee.” As the Steering Committee chair explains, the project is gradually leading to national changes: “It is thanks to the project that the rectifying finance law and certain aspects have been ratified at the level of parliament, for example”. The same chairman of the Steering Committee and all the other actors we met also emphasized that, as with the establishment of the CSPEs, the pandemic seriously disrupted the strategy of raising awareness among the political elites that was being developed. As the former head of the TOME project explains it well: “We had set up the bodies, we started writing the reports of the participatory monitoring committees and barely after one school year … the pandemic! […] we had done a launching workshop with the Ministry of Education and so we had to go through this migration, I was going to say this mutation from CPSE to COGES. [... with the 39 political leaders] we no longer saw each other during the field missions, and so that broke the rhythm. […] we would have needed to put some more juice in, but the pandemic came. ” Building a Success Story Despite these difficulties, the results achieved by TOME are impressive. They are primarily based on the fact that an exemplary certified experience, a success story, could be presented. The former TOME project leader explains this strategy in no uncertain terms: “For me, the problem was very clear, we have a system that is sick, very sick. We are going to take a cell from its body. If we manage to heal one cell, we can heal the whole body. And so, I said to myself, if I take a few schools, and if we show that this allows us to change the lines, to improve the systems, we can propose this as a generalizable policy. To do this at scale. Because at the time, we were in the dark. We were learning, but we couldn ’t see how to get there. Let’s say that it was a very important trigger. Yes, because what we were doing locally was not at all typical. We had the idea of influencing all the politics of the country.” It is important to understand what this exemplary experience can be. As the previous subsection pointed out, the establishment of CSPEs is not without obstacles and, on the ground, the situation seems to be heterogeneous, to say the least: a general dynamic is in place, but some committees do not meet or meet only a little, while others are already doing impressive work. However, the TOME project team understands that highlighting a few success stories is somehow enough to get things moving. The Final Report of the project places particular emphasis on one particular school, Fodé Diaguily, where the results are clear and explainable in simple terms: “The second lesson learned in 2019 is from the experience of the Fodé Diaguily school, which shows the impact on the quality of learning of providing schools with a clean and transparently managed budget. The 23 percent improvement in the results of the pupils of the Fodé Diaguily school in the secondary school entrance exam shows the merits of providing each school with its own budget and transparent management rules. This lesson leads us to generalize this approach to all the target schools in TOME.” As we have already explained, Fodé Diaguily is the school where TOME was able to implement everything the project wanted to implement, particularly to accompany the training and awareness-raising with the provision of a budget in response to the formulation of an action plan. The project team has produced a description of the experience6. As is often the case in examples of social accountability, the important thing is to see that a citizen committee, when given the means, finds local solutions to the (necessarily local) problems it faces. In Fodé Diaguily’s experience, the fact that the funding for the local budget over which the committee has authority ultimately came from a local association - rather than from the World Bank- further strengthened the foundation of the TOME approach. The experience also invites a broader rethinking of the coalitions needed to sustain social accountability actions, and we will return to this point later. In the remainder of this section, we try to better understand the general perspectives on the TOME success story as put forward by those who set it up. The interviews show two seemingly 6 The following actions were undertaken with the budget: ‘1. Availability of management tools to teachers and headteacher; 2. Organization of permanent pedagogical support for teachers and headteacher; 3. Organization of support courses for pupils in the 6th year; 4. Motivational incentives for teachers and headteacher; and 5. Scheduling of daily breakfast for students. 40 opposing perspectives. On the one hand, most of those interviewed speak of undeniable and inspiring success. On the other hand, a minority of field actors express skepticism. i/ An Inspiring Success… Interviews with local project workers and national and regional project partners (MPs, mayors, etc.) describe TOME as a unique success story, inspiring the whole of Mauritania and moving the needle. The emphasis here is on the proof of concept, TOME shows that another management is possible. The president of Ecodev explains: “[With the project] The directors have understood the usefulness of having well-ordered tools, which allow them to follow the activities at school level, the presence of the teachers on a daily basis, the presence of the pupil, or the level of preparation of the lessons of these children. I think that will stay. The second thing. I say that this trust, even if it is fragile at times, which has been established between our parents and the school director. I think that this too will be found at different levels at times [if we come back in five years without more support]. We’re past that fear.” He goes on to explain that the speed with which this change has taken place is, for him, remarkable: “We have school management committees that are made up of headmasters, parents and teachers. The fact that the school headmaster becomes a support to the parents in the management of the school, that’s what has changed, and I think that it surprised me. As a parent, it surprised me a little bit in two years already.” At the grassroots level (schools), some of the interviews, especially those with school headteachers and teachers, confirm the impression of success. The emphasis is on three elements: the exemplary nature of the project, the change in the relationship between actors, and the change in mentalities. The exemplary and inspiring character of the project is described in terms similar to those used by the national actors. References to other inspiring schools are frequent, as are references from key persons, such as the headteacher in Tidjikja who explains: “The experience of this project can be duplicated in Mauritania and examples abound in Riyadh schools, such as the Fodié Diaguily school.” The people we met often refer to a relatively small number of experiences, but which are inspiring. At the level of the Ministry of Education, a manager involved in the Steering Committee explains: “Well for me, I have attended several meetings and I have noticed several things in this project, including the monitoring of pupils and their attendance, the monitoring of teachers’ absences, remarks on the infrastructure of the school, if it is windowless and prevents the pupil from being present. The monitoring of pupils ’ absences enabled the TOME team to note that many pupils were absent because they could not find breakfast in the morning. Based on this observation, the TOME team decided to set up a small canteen in any school where pupils are absent because of breakfast.” Interestingly, the change is attributed here to the TOME team, when it is an action taken by one of the committees. The reconfiguration of the relationship between actors goes hand in hand with the feeling that everyone can be an actor of change and has a role to play in improving the situation of schools, as the headmaster of a school in Tagant explains: “It became clear to us that this partnership [that promoted by TOME] is possible and only requires that everyone plays the role entrusted to them”. 41 Finally, the interviews also cover another aspect that testifies to an opening up of the field of possibilities: the change in mentalities and knowledge that facilitates the reconfiguration of the relationship between people. During a focus group, the president of a SPA summed up this perspective well: “Objectively, the project is one of the best projects in the field of education in our Moughataa; it has left tangible traces on the ground in terms of ideas and knowledge and in terms of changing mentalities”. And, the director of basic education at the Ministry of Education, speaking of the traces left by TOME in the schools, added: “There is one aspect that is very important to me, and that is the training of headteachers in the use of certain management tools and the training of teachers in classroom management tools. Both things are important. One has a very important return on the quality of school governance, and the organisation of the headmaster’s work and the other has an important impact on the level of student learning, because it allows the teacher to make better use of their teaching time.” ii/… And Some Misunderstandings As the former TOME project manager explains, students parent associations can be seen as the big winners of TOME. He explains: “Those who won in this process were parents. The association was, let’s say, very political, ‘it’s the education system that’s not working, the parents are not involved’, but they didn’t have the tools, they couldn’t say what was wrong. What we gave them were analyses […] They could talk about something, let’s say technocratic, which is not just politics [ …] We found a students parents’ association with only 65,000 members [for the entire country that counted around 1.76 million people below 15 at the time of writing this report], which is very weak. The government has everything to gain and therefore give them more power.” At the grassroots level, the fact that these new tools are there is not discussed, but the parents’ perspective is not always positive about TOME. Many were concerned that they would not see (enough) tangible changes, sometimes referring to budgetary issues. The president of the SPA of one school does not hesitate: “Personally, I have only seen words throughout the three years of the project since the workshop organized in Mauricentre [where Ecodev organized the meetings and training] and even the meetings at school level, we parents have never had a concrete result.” The pilot nature of the TOME project and the nature of the activities that focus on governance do not automatically lead to materially visible changes (rather than, for example, changes in relationships or communication). At the parents' level, this material dimension appears important, to use the words of a SPA president: “I confirm that the municipality and the inspectorate are enthusiastic, and we are very happy, but they ask for material things”. Part of this problem could be due to a misunderstanding of the project on the part of some people who expect financial means, as one headteacher explains: “We found that at the beginning the main idea/mission of the project was not well understood, the idea was not well clarified, because it was said that the project was financed by the World Bank of about $605,000, but afterwards we understood that this amount is for the management of the project itself and not for the schools. What we want now is to have practical actions through a second 42 phase that will be based on supporting the basic schools with budgets for its proper functioning.” A departmental inspector also agrees: “The TOME project is a very ambitious project, it was even believed that it would fund the schools, but this was not the case. In any case, the schools have benefited from its experience, especially in developing action plans and organizing the recurrent work in the school, namely monitoring. And this is a very important achievement in itself.” In other cases, however, the committee is simply not effective, as reported by a SPA president in a focus group, who is joined by other participants linked to the same school: “the committee has never met, but we intend to meet and try to achieve the objectives, inshallah ”. In these cases, the blame is often placed on the higher authorities who are described as lacking in will and transparency. The intuition of the project, the idea of participation and social accountability, however, is still considered to be generally feasible. An Approach Accepted By All These slight differences in the success of TOME (or rather, these different views on different aspects) do not prevent all actors from recognizing the importance of the TOME project. On this point, all the interviews converge. For example, the representative of the teachers of a school in Tagant who attended a focus group explained: “I have never heard any dissenting voices regarding the intervention of TOME. The project carried out by TOME was unanimously supported”. The reasons for this unanimous support are probably found in several places, and Section 7 returns to what might explain the general support and enthusiasm for the project. Two points should be noted at this stage: (1) that the project is intended to improve the school in the population’s interest is recognized by all; and (2) secondly, that this unanimous support is also strongly linked to the promise of a better tomorrow. The president of a SPA explains: “From what I have heard, the idea of the project is acceptable to everyone if it improves the performance of the school”. The question that arises, and which is not always immediately resolved by a pilot project, is whether the performance will be there to maintain the momentum - again, it is important to realize that the understanding of performance may vary from one stakeholder to another (particularly in terms of its materiality, as discussed in the previous subsection). For many actors, performance will necessarily involve refinancing, and this is the next step they have in mind, such as the president of a SPA in Nouakchott: “In our school, the parents were happy when this project came, and they approved it, because it is based on the cooperation between the school and the parents, and indeed the cooperation exists between all. If the TOME project finances schools, it will be a good foundation for the success of the Mauritanian school.” In another school in the other wilaya, another SPA president has a similar discourse, and we will conclude this subsection with his words: “These kinds of ideas and projects are popular 43 with everyone, but the problem is management, which requires strict monitoring; the project idea is good, but it needs to be financially supported for its continuity and sustainability.” Summary and Conclusion On the second indicator, ‘generation of citizen feedback through the process of social accountability and collaborative engagement with public sector institutions’, the project has achieved clear successes; especially with the enactment of a new legal framework and then the promotion of a new accountability institution, the School Management Committee (COGES), which is a direct continuation of the CSPEs piloted during the project. The ideas promoted by TOME, first and foremost social accountability and collective action of different parties in favor of education, seem to be fully supported by all actors  - including those for whom they mean having to reconcile with new parties (e.g., in the administration). Early advocacy engagement with the authorities and the construction of an exemplary experience, a success story, are the main elements used by the project to generate collaborative engagement. At the school level, significant challenges remain - notably with the lack of resources to turn community feedback into actions - but the strategy of demonstrating the added value of the CSPE model with a test school and involving the authorities in the process is working. The evaluation shows, however, that it is vital to qualify the link between citizen feedback and collaborative engagement at two levels: firstly, the contribution of feedback (and of TOME in general) is sometimes direct, sometimes more diffuse, and secondly, citizen feedback is not always the only way in which the TOME project affects public measures. Facilitating Knowledge and Learning to Strengthen the Effectiveness of Social Accountability Interventions and Project Management The third intermediate outcome is more difficult to assess quantitatively; the table below shows the intermediate outcome indicators. In the Final Report, the project highlights five key lessons shared with stakeholders on different occasions, which relate to (1) the structuring of actors  - including the establishment of an additional, regional level in the structure for framing and promoting social accountability (see previous subsections); (2) the promises of transparent budget management; (3) the need for a fund to support the schools ’ budget; (4) the (forced) possibility of moving some of the capacity-building online (via WhatsApp groups); and (5) the potential usefulness of the online platform. We will not go into these aspects again but note that what matters here is not the arbitrary number of lessons learned but instead what was learned and how. Table 7: Achievement of Intermediate Outcome Indicators 2 Selected Intermediate Outcome Unit of Measurement Cumulative Values Indicators Target Achieved Values in in 2021 2021 Intermediate outcome indicator: Number of lessons learned 5 5 Knowledge and lessons learned to improve project during the implementation of the management project 44 Intermediate outcome indicator: Number of communications 10 11 Sharing and dissemination of shared and disseminated lessons learned Therefore, this component is more challenging to approach in terms of the relevance of the implementation strategy. Clearly, and this was repeated to us during the interviews, the project has had an adaptive and iterative approach, and the various parties we met with emphasized this aspect. It seems important to show how TOME is an excellent case study to demonstrate how a local or even localized success is used as an example and helps to initiate significant change. However, this case study cannot be imposed from the top; it is patiently built through the engagement of different actors. The president of Ecodev explains: “We feel that when we take a ministerial order from above, a law is a piece of paper. But in practice, it’s complicated. What TOME has put in place is a committee that offers this functionality: the involvement of parents, giving them more weight, training parents.” In our interviews and focus groups, we found different, not necessarily mutually exclusive, understandings of the purpose and nature of the TOME project. However, it is usually possible to link these understandings to people's role in TOME; they illustrate a learning process in which everyone enters through a different door. Understandings of TOME For the initiators of the project, the objective is simple: to create awareness and instill a logic of collaboration and joint analysis of projects. To use the words of one of the project initiators: “Our objective was to raise the awareness of decision-makers. Don’t forget that TOME’s objective is not to solve the problem. It’s not about teaching the students and teachers, it ’s not about bringing the books, it’s about trying to create a dynamic. [The idea is that] at least they have an awareness of diagnosis in a different way from global and invasive diagnosis, more precise and detailed, around these four issues that are teachers, students, textbooks and school management.” Similarly, for public administration actors and authorities, such as the MP heading the National Advocacy Committee, the aim of the project is to raise awareness among the population and the authorities and to change mentalities, as he explains: “For schools and administrations, it changes the] role they can play in improving the achievements of their students. This is more time consuming and has made people aware of the role they can play. Because parents of students are not only responsible, but they can also play a big role in the future of children ’s achievements.” For teachers and headteachers, two of the categories that have benefited most from the TOME project’s training, there is a clear understanding of the TOME project (at least in the schools visited). This could be summarized by this teacher from Fodié who explains that the project aims to: “explain the best way to manage a school budget in a transparent way”. When trying to go deeper into the subject, we see that this transparency is understood in a very concrete way. It is embodied, on the one hand, by the existence of the committee and new relationships, and, on the other hand, by the activity of monitoring and sharing key information recorded, namely: staff (presence and absence), pupils (number of pupils), parents (number of pupils), 45 meetings (and their minutes), and pedagogical monitoring (school principal). Some teachers and headteachers also linked the TOME project to broader issues, such as the decentralization of education. At the level of the parents’ associations, the definition of the TOME project is often a little less clear-cut, possibly due to a lower level of literacy. One SPA president speaks of a “project that targets schools and tries to improve their situation ”, while a SPA president in Nouakchott explains her definition by referring to the concrete activities in which the SPA is engaged, namely: “We always hold meetings at the municipality on school issues such as the construction of classrooms and the role of parents in monitoring teachers and their attendance and absences.” A key element in discussions with parents is also what the project is not, i.e., the perception that TOME brings nothing in terms of materials or financial resources. It is not uncommon to see a certain frustration with this state of affairs. Some SPA presidents go further and do not seem to fully understand that the TOME approach does not include more material benefits: “I know that Ecodev always comes to see us and meet us, I have never benefited from anything, and I have heard some people say that this organisation is interested in education to improve it, I am not aware of anything else except that. I know that there are other NGOs that visited us at the opening, they provided the students with teaching tools, which we appreciate very much”. These slightly divergent understandings are not a problem in themselves, but they illustrate the difficulty of creating a learning framework given the different aspirations of each party. Summary and Conclusion With regard to the third outcome, ‘facilitating knowledge and learning to strengthen the effectiveness of social accountability interventions and project management’, the TOME project is described and experienced as a highly iterative and new process in the Mauritanian context, from which there is plenty to learn and experiment. The project soon developed as more than just a platform, and a whole ecosystem of social accountability, with training and new committees, was set up. The COVID-19 crisis, in some ways, forced a return to a more platform-centric approach. The general understanding of TOME and the principles of social accountability is generally good and aligned with the project's objectives. Still, it is also quite clear that for many field actors, there is a need to see very concrete changes (including material changes) to embrace the approach fully. 46 7. Localizing the Theory of Action “And so when the TOME project was launched, let’s say the policy cycles and the findings were all in the red. The government and all the actors knew that the system and its quality were not improving, but they didn’t know what to do. The World Bank had sounded the alarm in a report that focused on education and showed that this problem is not specific to our country, but a general problem.” Former TOME project manager The remainder of this report will focus on validating and further illuminating the (theoretical) fundamentals that underpin the work of the GPSA, building on the TOME project. This section looks back at the contextual elements that allowed both the principles of social accountability to gain local acceptance and CSPEs to emerge, but also at the elements that allow for a better understanding of how serious scaling up could be achieved in the space of only two years. This section, therefore, returns to the following evaluation questions: To what extent and how did the sectoral system enable or hinder the implementation of the project? In turn, what contributions, if any, has the project made to strengthening these sector systems and addressing bottlenecks and other implementation gaps? The questions posed by the GPSA approach focus on the sector system, which is, of course, also key, but we also believe that other elements beyond the sector framework are essential, and we address these in this section. We return to the most significant (and sometimes most original) without dwelling on the obvious - such as the management of the project or the competence of the staff involved. In particular, this section seeks to understand better the elements of what several interviews with national-level officials have described as a window of opportunity. An Idea with a Long History At first glance, the TOME project appears to be a radical novelty in the Mauritanian educational landscape, which is marked by significant centralization. The president of the CNP speaks of the general attitude of parents before the project: “Stakeholders, especially parents, do not pull enough. That is, they don’t hold those responsible to account. Sometimes it is also due to the lack of interest of certain populations, especially in rural areas, or even in education. [ …] what was missing in Mauritania was this kind of exercise [the TOME project]. ” Beyond the decades of centralized and sometimes authoritarian power, practices of consultation and discussion do, however, exist. It is, among others, the case in the Islamic (and Bedouin) tradition, which is cited in certain official documents, for example, in the synthesis of the national consultation days on education. The participants in our research do not mention this tradition outright, but they readily confirm it. These traditions are not explicitly mobilized by the TOME project, which relies more on more recent administrative and political concepts, such as the idea of decentralization or social accountability, but the interviews show that long- term dynamics are also at play. The starting point of the project should therefore be further qualified: it is correct that the project is very innovative in its approach, but at the same time, it echoes ancestral practices and traditions. The idea appeals to a certain common sense which, if not the political norm, is socially accepted and acceptable. 47 A Favorable Political Context From the perspective of Mauritania’s more recent political history, the TOME project also comes at the right time concerning two key elements: (1) the launch of a new cycle of education sector reforms starting in the 2010s and accelerating with the electoral program of the new president in 2019, (2) a new political will that sees the gradual establishment of a decentralization and devolution framework. Of course, it is not possible to separate the birth of the TOME project entirely from these changes: if the project was set up and approved by partners, it was also because its initiators perceived a window of political and administrative opportunity. However, this was far from guaranteed, and there is no shortage of failed reforms and frustrated political ambitions (in Mauritania and elsewhere). The dynamics that will allow TOME to be fully acceptable started in Mauritania around 1998, with the gradual establishment of local and participatory governance. The 1999 reform, which is underpinned by the National Sector Development Program, is part of this movement that will gradually make the idea of (social) accountability more acceptable and popular. The executive secretary of Ecodev explains: “[Accountability] is what is a bit new, it started to develop in the early 2000s. I prefer to place myself in 2002. It was really the beginning of these changes. With local authorities, we saw an elected official who was responsible for a territory and who was supposed to take care of its citizens and therefore talk to them, involve them, listen to them. That was at the beginning of the 2000s. [ …] But the questioning, that is to say questioning a mayor, questioning a person in charge to say what are you doing with this? Why can’t we do something else where this is? That’s new. This is new and still needs to be improved […] We are still in the early stages and it is not yet very solid this questioning of those in charge of the management of public affairs.” Technically, the country has moved towards greater transparency in public life, but, as respondents at the national level express it, there is still a lack of awareness of the rights and duties of everyone (for example, municipal councils, which are technically public, very rarely have attendees from the public). Nevertheless, as the president of Ecodev explains, social media have started to change things: “social media scare the administration. Why is that? Because they will criticize public action.” It is interesting to note that there is also an awareness that TOME also echoes a broader logic at a more local level. As one departmental inspector of national education (IDEN) explains: “It seems to me that the TOME project is based on important things such as the participatory committees in schools. These committees are very useful, which reflects the state’s intention to generalize this experience throughout the country.” To the potentially fertile, but little tilled soil of the long history is therefore added a certain political momentum which is easily identified and identifiable by the various stakeholders in the project and which gives the project a certain legitimacy. 48 A Trend That Speaks for Itself If the project was met with such enthusiasm, it is also because the failure of education sector reform policies in Mauritania is widely acknowledged and accepted, as the former TOME project leader explains: “I can tell you what allowed us to scale up: the elites and the politicians and the public policies were at an impasse, the government was at an impasse, the actors were at an impasse. Everyone was saying that the system was not working, from the President of the Republic to the MPs […] It’s as if there was an obvious diagnosis, but which was not clear. [Many said that] it’s because the teachers suck. But they don ’t. For the first time, we made them face the fact that it’s not the teachers that it’s something systemic. It’s about the teacher, it’s about the students, it’s about the school inputs, it’s about the management of the school and it’s about the overall governance of the system. It was a totally new diagnosis for the actors. They knew there was a problem, but let ’s just say that when we started, it was a good moment for us and for the system to listen to us. The timing was excellent.” The sector is also important. As our interviews at all levels confirm, there is agreement on the importance of education. As the president of Ecodev explains: “If we take education, everyone agrees: the role of parents, their involvement in schools. We have set up a participatory monitoring committee that everyone agrees on and that everyone accepts. What is missing is the capacity of citizens to do this exercise. But I think the administration is no longer adamant. And, of course, as long as there is no citizen control, it will do what it wants. But the citizen is capable.” This element, which is very present in interviews with officials who are trying (and have tried) to change the lines in the education sector in Mauritania, is correctly identified in the project’s initial theory of change, under the heading of ‘collaborative engagement: heightened awareness on the part of civil society (FENAPERIM and networks) and the state (ministries, parliament) of the seriousness of the learning crisis and the need to find a solution’. Multiple Positioning An element that is less present in the theory of change concerns the possible positioning of the actors of change. Education is the business of many parties, administration, parents, civil society, etc., but this categorization masks potentially more complex realities and the importance of actors who cross the lines. The project was able to benefit from a unique positioning of Ecodev, which was able to present itself not as just another NGO or just civil society but as an actor at the crossroads and a facilitator. Of course, Ecodev remains an NGO, but the TOME project actors benefit from a network and a base within the administration of the Ministry of Education (we will come back to this point) and also from the weight of the World Bank, which makes it possible to maintain pressure at the national level. It was this combination of support and previous experience and a project that could unblock things that convinced the ministry, as the former project leader explains: “In reality, the people who could have lost power were the people in the government and the Ministry of Education itself. But it has not been an obstacle for this project. Why? Because it’s the first project in the field. 49 Because it is the first project in the field that is innovative in Mauritania, and because it is financed by the World Bank outside the government.” This singular positioning and the transparency of the project will make it possible to win over, one by one, the actors who were initially less directly convinced by the project. The teachers ’ representative present at a focus group session explained: “Nobody hesitated [to support TOME], although at the beginning of the project teachers and some unions were a bit hesitant, and then we found out that the project is based on transparency, and I am not aware of any people or institutions hesitating to support the idea of TOME. ” According to the instigators of the project, it is also this positioning and capacity to bring people together early on in the project that strengthens the working relationship with the ministry and convinces it of the usefulness of the program: “when they [the ministry] came to see us halfway through, they realized that we could bring people together, we put 100 people in a hotel ”, explains the former project manager. This positioning inside and outside, in several spaces simultaneously, is what allows Ecodev to push the project, but it is also, in a way, the positioning that CSPEs and committees at higher levels seek to have. Ecodev’s executive secretary explains: “Outside, we were trying to structure the parents, to help them organize themselves better and to give them tools. But with time, we got inside the school. The committee that we had was brought in. It was extended to the teacher, to the students themselves, to the student representative and therefore to the debates.” One of the keys to TOME’s success is, therefore, its inclusion in different spaces: rather than simply confining itself to the space of a certain civil society (often in some form of opposition or at least claim towards the authorities), the project builds bridges and forges alliances very early on, in keeping with the very spirit and objective it has set itself. It is by being what it tries to build that the project gains in effectiveness. A Strong Team Finally, and closely related to the previous point, it also emerged from the interviews that the people who were at the heart of the TOME project were very important to its success, because of their individual qualities, of course, but also because they happened to be people who ideally combined insider and outsider positions, allowing them to connect with many of the key players in the project. At the head of the TOME project, and at its origin, is a former director of human resources at the Ministry of Education. He knows the workings of the public administration, the issues at stake, the key people and will, on the ground, reconnect with people he has worked with before. As for the rest of the staff coming directly from Ecodev, expertise also exists as explained by its executive director: “We stayed close to the debates and discussions on social accountability until the GPSA launched this call. And so, as we had this background somewhat associated with accountability, we made the proposal, and that’s how we were removed. We already had an interest in education, but we didn’t have an education project. We already had an interest in education, but you could say that we came in through accountability.” 50 He goes on to explain how the project team was able to mobilize and the work done in raising awareness among the parties: “...The National Federation of Parents played a very important role. Then we presented the project, the methodology of the project to the local authorities, to the local administration (the Hakem/Prefects). And so, after this great campaign of awareness and communication. We started by setting up consultation bodies in each Moughataa, what we called the participatory monitoring committee at the Moughataa level, which made it possible to share information in real time and to use it and possibly give feedback. [ …] National elected representatives came to reinforce the communication sessions with the elected representatives and regional directors of education, as well as the school headmasters, all of which were tools and common elements that helped, I would say. And that was not accepted immediately. There was always mistrust. And so, the project leaders had to do a lot of work to validate their assistant relationship manager […] to come to the committee meetings at the beginning to help them, to provide them with working tools, action plans, forms to fill in and to put everyone at ease.” It is difficult to establish what the counterfactual would have been with a less experienced and less connected staff, but let us simply note here that the exploitation of the window of opportunity that existed at the time of the TOME project in Mauritania and which is characterized, among other things, by the aspects described above (points A-D) was possible because the project staff were able, thanks to their experience, to identify and seize this window. Summary and Conclusion The TOME project is, at first glance, a radical innovation in the Mauritanian educational landscape, which does not count any community participation and accountability initiatives. However, a closer look at the context shows that the idea is embedded in different time frames and realities, which together provide a window of opportunity to understand better why and how the central idea of TOME, the set-up of participatory school committees and more collaborative school management, was able to take place in the pilot schools and spread to the national level. First, considering the long history, the idea of collaboration echoes religious and traditional principles and, therefore, makes sense for everyone, and second, within the education sector, the conditions are ripe at several levels: (1) a cycle of education sector reform is underway, (2) the idea of greater decentralization and devolution in the sector has already been launched and is gaining momentum, and (3) the sectorial crisis is such that actors, who feel stuck, are ready to experiment with something new to get out of the deadlock. TOME’s original theory of change echoes these points, even if it anticipates more resistance to the idea of social accountability from the administration than there was (probably because of the aforementioned education crisis). This theory of change also identifies the importance of a space for discussion, which undoubtedly played a role. However, we also find that this space must be conceived beyond the formal space: the positioning of TOME and its staff as anchored in civil society, but also very connected to both the administration and the population, is key to creating a coalition, a mobilization, and a real interface between the state and the citizens. 51 8. Challenges and Sustainability In this section, we look at the second part of the context-related evaluation questions, namely: Under what conditions are the results sustainable? What is the risk that the results achieved are not sustainable? Before proceeding, however, it is essential to note the ‘irreversible’ nature of the CSPE under its new name, COGES, due to its institutionalization by Decree 039-2020. As the director of basic education (at the Ministry of Education) emphasized during our interview, as we were discussing the introduction of the participatory approach as a tool to improve school governance capacities: “Indeed, the participatory approach is the approach that the department wants to generalize by taking into account certain considerations, such as the reduction of the members of the COGES’s executive board in conformity with the texts that organize it, but also in terms of the use of management tools by simplifying them to make them accessible to the future members of the COGESs, many of whom have very low academic levels.” Let us now look back at a series of potential weaknesses of the project, as noted in the field, before considering the evolution of the sector and the areas where synergies seem possible. Weaknesses and Challenges A Two-Speed Project? In addition to the enthusiasm of many of the interviewees, it is also important to point out the reservations they have about the project. At the level of teachers and headteachers, these reservations are limited and relate mainly to the fact that communication between the project and the schools could have been better at certain times. The project leaders acknowledged this weakness, explaining that a greater presence in the field would undoubtedly have improved the results. As reported by some headteachers, the problem is that “we noted a lack of communication between the project leaders and the actors in the field, which leads to unilateral decisions that are often not appropriate. We would have preferred that the decisions to be taken were discussed with the CSPEs before they were validated ”. It should be noted that this criticism comes amidst substantially more positive comments and that the pandemic context at the end of the project did not, of course, facilitate communication. Without neglecting these comments on communication, the issue does not necessarily appear the most central. More recurrent, however, were comments from SPA presidents who felt themselves to be in the minority in the TOME project, or at least envious of the apparently more substantial training that had been provided for headteachers. One SPA president in Tidjikja explained: “I think that TOME has not fully appreciated the role of parents, and the proof is that many training sessions were organized in Nouakchott without the SPAs being invited, and tablets were given only to school headmasters, whereas it was necessary to provide them to parents to enable them to carry out their monitoring and data mobilization activities. In reality, the priority was rather to involve the parents. It was the parents as partners who needed to be trained and equipped. ” 52 These comments should be treated with caution, however, as the parent federation, on the contrary, emphasizes the essential work of training parents and their associations. Ultimately, the project made choices in terms of training, and the school was the focal point. The criticism of TOME as a two-tier project is undoubtedly well-founded. Still, it does not contradict what was intended by Ecodev, which never claimed to train all actors permanently. The more fundamental question is whether an approach that focuses on training certain key actors is effective, and the answer seems to be positive because of the interest shown in TOME. However, it should be noted that apart from making many of its tools available in the public domain, the project has not proposed or tested a mechanism for scaling up and easily replicating training and awareness-raising. Is Facilitation Still Necessary? While the TOME project has successfully raised awareness, CSPEs (and, even more so, COGESs) are still in their infancy, and the interviews reveal a need for guidance or at least support. According to those who set it up, one of the project's weaknesses is that it did not have more field relays, i.e., Ecodev agents more directly present in the field. As an abundance of academic literature emphasizes (Mansuri & Rao, 2013; McCoy et al., 2012), participatory work requires that facilitators can act as relays between parties, as referents to settle any disputes, and as trainers when new people arrive in the system. At times, inspectors were able to play this role during the project, but it is not their vocation nor a permanent solution. The president of Ecodev talks about the usefulness of setting up a development and liaison officer: “If we had put agents in place, in any case, the idea that we are developing today is that a municipality that has thirty or twenty schools and that has to work on its education needs an agent dedicated to education. […] We could have found mechanisms to integrate the communes so that they would have agents dedicated to education. And of course, this would also have boosted everything we did and given greater sustainability.” The absence of such an agent, whether from an NGO project or the state, is a short-term threat to the project’s achievements, but at the same time TOME has prepared the ground - notably through the provision of tools (digital or otherwise) that could facilitate the rapid deployment of such an actor. Nevertheless, it is likely that the use of these tools, during and after the project, is highly correlated to the level of support after the training and awareness-raising period. This need for training and retraining is omnipresent in the CSPE action plans and regularly quantified. This points to an additional problem, even if a real person were to be present in each municipality following the project, would the organisation of training and support be possible without an additional budget? At the ministry level, for example, we observed that the idea of the COGES is not necessarily always understood, and there seems to remain a reluctance to trust the decentralized authorities fully. A newcomer to the Ministry of Education, who discovered the TOME project a few months before it ended, explained: “Moreover, we decided at the level of the department, very recently in the context of the implementation of the COGES, that although the commune insists on the fact that the management of the school is of its competence (taking into account its proximity, especially in the countryside), the minister has, in a preoccupation with 53 administrative continuity, to entrust the implementation of the COGES to the walis [the second administrative level, when the commune/municipality is the fourth].” Here again, the hesitations point to the need for greater support during and after the project. Finally, following the director of basic education we quoted earlier, we note that the tools developed for the CSPE (and COGES) can still be improved. As she noted, “the current tools will be difficult for everyone to use ” in view of the level of training of the members of the CSPEs, and an additional effort is therefore needed. Exemplary Schools That Are Not Quite Ordinary? One point that this evaluation was unable to determine due to lack of data is the extent to which the pilot schools are indeed representative of Mauritania. One of the interviewees pointed out the urban or peri-urban character of the majority of the schools, including Tidjikja, which, as its MP reminded us, is located in a rural area, but is also a historic town, founded in 1680, with traditions and a history different from that of rural areas. The question of the implementation of CSPEs and the continuation of the idea of social accountability, notably through COGESs, arises on two fronts. Looking at better-off students, what can be expected from public schools and can the participatory approach be a solution in this sector as well? The whole education sector is suffering, not just the public school. In a way, the private sector is perhaps the one that most needs the introduction of a participatory approach to monitoring and citizen feedback. As the National Reform Conference held at the end of 2021 pointed out, this sector is currently moving in the direction opposite to the public system, both in its approach to learning and in the curricula that is taught (which is not in line with the official curriculum). Considering poorer students in public school now, the question is the representativeness of the people involved in the COGES, of whom we have not been able to establish the profile (but it is known that the most marginalized are less likely to participate). There are also several questions about how the model might work with nomadic populations such as Mauritania. At present, this is not an issue, as COGESs are planned to be implemented in full-cycle schools only (which represent about 40 percent of schools), but the issue cannot be avoided in the medium term. Different Committees, Different Opportunities The TOME project has established not one but three distinct citizen-state interfaces, each operating at a different scale and, therefore, needing to be considered individually in terms of evaluation and performance requirements. Sections 6 and 7 have already discussed many of the elements related to the operation and implementation of these committees in practice. In this section, we discuss the specific challenges to the sustainability of each type of committee. At the most local level, we find the CSPEs set up during the project and which are really its core. Our evaluation shows that they have been functional, but how sustainable are they? Although the project documents distinguish the possible coexistence of these committees and COGESs, it is likely that in the COGESs will take over and absorb the CPSEs in the pilot areas, as they are established by law (unlike the CSPEs, which are just an emanation of TOME and 54 are unlikely to swell on their own without third party mobilization). However, the law alone will not be enough to bring COGESs into existence, as a large body of literature on community participation points to the risk of having committees that are empty shells, as their members do not know exactly what they should be doing and are meeting simply because they should be meeting (Falisse & Ntakarutimana, 2020). Support is needed. Our analysis of the CSPEs also points to another risk: the awareness-raising and training phases generated a great deal of enthusiasm and a real desire to change things, but the lack of resources, particularly a budget, makes some people doubt. We will come back to this, but the lack of a budget and, therefore, the limitation of the real stakes of the COGESs (and before them of the CPSEs) is potentially a threat to their long-term functionality (Osmani, 2008). At the departmental level, the committee is the most recent (of the three committees considered) and was created due to a need identified by the TOME project. The departmental committees played an important role during the project and seem to function well, as they meet a real need for coordination. In the documents we were able to consult at the Riyadh level, we find an active committee that synthesizes the needs of the schools and is able to follow up on the schools’ plans. The emphasis of these plans is very much on the critical lack of means (IT, but also human resources) and, to a lesser extent, on learning capacities, the openness of teachers, and the question of relations between family and school. Again, as in the case of the CSPEs, there is a risk as to the vitality of these committees in the medium term: a large part of the activity seems to be to relay and follow up on the schools ’ plans (with ambitions totaling MRU 76.79 million, or more than $2.1 million, for Riyadh alone in the documents consulted). Without a budget on which to act, the risk is that the departmental committee, whose role would be limited to exchanges without means, would weaken over time. Finally, at the national level, we find a National Advocacy Committee that did meet during the project but its activity was seriously diminished during the pandemic (it no longer met). This committee also responded to a need to move the needle at the national level and brought together people sympathetic to the project. It is not clear, however, that it was the committee ’s action that was at the root of the changes identified in Section 6, as the TOME project seems to have benefited from more individualized support and privileged relays in its efforts to promote the idea of social accountability. This is not to say that this committee is not useful, but rather to emphasize that it is the members in their individual capacity rather than the committee as a pressure group that seems to have mattered. In terms of sustainability, it was not clear that the committee would continue in its current configuration, especially as some of the legislative and policy changes it promotes were taken with the introduction of COGESs. However, the committee has provided a strong parliamentary alliance for education, and it is likely that the contacts made between members will continue. A Changing Context, but One in Which Significant Challenges Remain As we have seen, the Mauritanian context is generally supportive of TOME and the proposed innovations. The TOME project was designed in line with the Mauritanian government’s ongoing reforms and priorities in the education sector. Linked to TOME, either directly or more indirectly, government efforts include the development of the COGES as well as reforms to 55 increase the internal efficiency of primary education; the decentralization of certain functions to schools in the management of resources; the implementation of school improvement grants; the increase of financial resources for the acquisition of teaching and learning resources; and efforts to achieve a motivated and efficient teaching staff. On the face of it, this context should remain positive and encouraging. Still, some obstacles remain, first and foremost the delicate question of a potential school budget managed directly at the school level, which seems, as we have already pointed out, to be a sine qua non for a meaningful implementation of COGESs (what are COGESs if there is no real budget to manage?). Where could this budget come from? The director of basic education spoke of the difficulty of direct financing by students, which would also pose major problems of equity: “As technicians, we do not mind that students participate in the financing of the COGES, but until now, the policy of free basic education followed by the sector is contrary to this measure and therefore, this measure cannot be a source to finance schools. Thus, the sources of funding for the COGESs of the remaining schools are the small subsidies allocated by the department, which will henceforth be paid into accounts that will be opened in the name of the COGES and managed by it after the signing of the decree currently being prepared. Other sources of funding can benefit the COGES in the framework of support from technical and financial partners or civil society organisations such as TOME, for example, which intervenes in two zones (Moughataa of Riyadh, Moughataa of Tidjikja) or also the PASEB II project, which intervenes in six Wilayas of the country to support the COGES in the full-cycle schools of the two Hodhs, Assaba, Guidimagha, Gorgol and Nouakchott South.” Could the budget then come from the state? The ‘over centralization’ of the country, as one of the project's initiators calls it, seems to be a stumbling block. He explains: “As far as education is concerned, the system is highly centralized. Even if there are the beginnings of devolution, we have things that can only be managed at the central level, like the budget. Teachers are paid at the central level, so monitoring at the decentralized level and at the regional, departmental and local levels cannot be done because we cannot exercise sanctions on teachers at the local level.” The few months that have passed since the end of the project give us some indication of the evolution of the politico-administrative context of the education sector after the TOME project. The national consultations on the reform of the national education system, organized from 16- 20 November 2021, were a unique opportunity to see to what extent the project ’s ideas have impacted and shaped the field. The event brought together all the country’s key education players, with nearly 500 people in attendance. A similar exercise was carried out through “fifteen regional workshops for reflection and consultation on the main lines of reform of the education system in Mauritania, held simultaneously in all the country ’s wilayas” (ECODEV, 2021). The consultations were the product of a political program from the top of the State, as their Final Report explained (ECODEV, 2021): “the objective of these consultations is to establish, on the basis of society’s aspirations and the opinions of national actors (teachers and their unions, parents and their associations, political parties, civil society organisations, the administration, etc.), a shared vision that will serve as a reference framework for the 56 development of a coherent and operational strategy for the renovation and development of the education system in the short and medium-term”. The thirty-three pages of the General Synthesis Report deal with a range of issues, many of which, such as curriculum or language use, are outside the scope of TOME (MEN, 2021). Section III.12. deals directly with the governance framework. It refers to “the establishment of an appropriate coordination and consultation framework that empowers and involves actors and operators at all levels” (p. 31) and to “the establishment of an appropriate coordination and consultation framework that empowers and involves actors and operators at all levels ” (p. 31) and “transparent resource management” (p. 31). COGESs are also mentioned in the section on the rationalization of the school map. Still, it is clear that the element is new to many, the only action point relating to them is to “assign certain attributions in terms of management of the school map to COGESs, SPAs and local authorities.” (p. 20). The idea of increased community participation at the school level is not at the forefront of the synthesis report, which mentions the idea mainly in terms of strategies to improve pre-school and literacy. Overall, the national consultations confirm various hypotheses developed earlier in this report, including a slightly ambiguous situation typical of a potentially changing context. On the one hand, the ideas of social accountability seem to be discussed in the public arena, which is very positive in view of the previous situation, and TOME is probably no stranger to this: the principle of COGESs is accepted, for example, as is the idea of certain transparency, and sometimes even the idea of a fairly broad coalition of different actors in favor of education (parents, associations and local authorities are mentioned). On the other hand, the synthesis report also raises questions about what exactly to do with the COGESs. The general approach is not necessarily imbued with the idea of regular and increased participation and consultation either. The synthesis report mainly delineates public policy orientations, many of which are essentially, if not solely, a matter for the Ministry of Education. The space, therefore, seems to be opening, and progress is clear, but there is still a lot of work to be done to ensure that this opening up is sustainable, internalized by all, and real and tangible for all actors, including those on the ground in areas not covered by TOME. The New World Bank Funding At the level of the Mauritanian government ’s partners, the idea of COGESs and social accountability in schools in general also seems to be gaining ground. Thus, the tools and dynamics promoted by TOME have seen their immediate future assured through the second phase of the World Bank’s funding for basic education support (PASEB II) in Mauritania. It aims to improve primary education in the country between 2020 and 2025 ($40 million from the World Bank and a further $12 million from the Global Partnership for Education). Although the TOME project is only briefly mentioned in the PASEB II documents, it is very clear that the COGES and accountability actions have been developed as a follow-up to the Ecodev experience. The support covers many elements, including teacher assessment and the development of an education-focused information system, but a central aspect is the training and strengthening of COGESs in six regions. This strengthening includes, in addition to a series of activities directly inherited from the TOME project, the provision of a budget of up to $600 57 per year for each COGES (via mobile payment for some). 750 schools are targeted by the end of the project (which is being implemented incrementally). The new funding is not in itself a sustainable response. Still, it does demonstrate the success and importance attached to the TOME project by the World Bank and the Mauritanian government, which are choosing to make the ideas it has carried the cornerstone of their new program. Furthermore, PASEB II funding is potentially significant. It sets a time horizon of five years during which COGESs will be substantially supported, allowing for further learning and perhaps a shift to a fully sustainable (i.e., locally owned) system. New Areas to Explore Our interviews suggest areas where CSPEs and even more so COGESs could potentially forge strategic alliances to circumvent or mitigate some of the potential problems just mentioned. One ally of the schools mentioned on several occasions is the schools’ laureates, and alumni associations that will support the school in difficult times. As one headteacher explained: ‘Our main ‘allies’ as I told you are the National High School leavers and the Mauritanian parliamentary advocacy committee as well as Ecodev.’ In the same vein, the MP for Tidjikja also mentions the idea of a friendship that would support schools based on a model that he himself has set up. He explains the idea: “The aim of the association is to ensure that learning is validated and to seek solutions to some of the problems that may arise at the school. We may have to pay for studies, or sometimes even look at quality and price. It can facilitate exchanges between parents of pupils, or even solve political problems. I believe that this can serve as an ideal framework for consultation with the operators. As a member of the parliamentary group on education, I had promoted these associations in the National Assembly.” Summary and Conclusion The TOME project aimed to demonstrate the possibility of a new model of civic engagement and participatory governance in schools. With the enactment of the COGES law, but also through the strong interest of national (parliamentarians), local, and civil society actors (in particular with the National Federation of Parents [FENAPRIM]), it seems that a new field of possibilities has opened up in the education sector. The recent World Bank Education Sector Support Project in the five poorest wilayas of the country (PASEB II) has taken up the support for the COGESs, and incorporated the need for support to their budgets. This is, of course, an encouraging sign and should leave the possibility of the system piloted by TOME being implemented and monitored for a guaranteed five-year period (2020–2025). However, these positive elements should not obscure the fact that the social accountability dynamic promoted by TOME remains fragile and still requires support. The sector remains highly centralized and national discussions (such as the national consultations) address the subject of social accountability without making it a central issue. At the level of the people we met, people who had not been involved in TOME, both on the side of the partners and the administration, often struggle with the concept or even with the idea of COGESs. In terms of on-the-ground achievements (local, school and departmental), the enthusiasm of committee 58 members is certain, but the sustainability of the newly created institutions (in the form of the COGES or otherwise) also remains fragile. Even with a trained population, the lack of means (budget) to follow up on plans and decisions risks making committees not supported by partners - including those with a legal existence - empty shells or at least less central actors than conceived by the new laws (and by the TOME project). 9. Conclusions: Looking Back at the Theory of Action This study shows how the TOME project took advantage of a window of opportunity (an opening for innovation) in the context of the education sector in Mauritania to develop and implement a new approach that proposed to reform the education sector through increased collaboration, at all levels, between a broader range of actors. The idea of instilling more social accountability seems to work. At the local level, the buy-in from different actors is strong, even among those for whom joint decision-making means sharing power. Interviews and focus groups support a sense of substantial empowerment (and related expectations). At the national level, various initiatives, most notably the law on and then the launch of the School Management Committees (COGESs), which took up much of the essence of the (pilot) TOME project, show that ideas of social accountability are gaining ground in the country. Of course, as we show in the sustainability section, there is still a long way to go, and the successes are fragile. To achieve what is in many ways a success, the TOME project team was able to rely on a team that was able to read its context but also to adapt a project to a changing context (Grandvoinnet et al., 2015). It was originally conceived as a project in which the internet-based information exchange platform played a prominent role and was at the heart of the project. But, TOME developed a broader ambition, setting up institutions  - local, departmental, and national committees - to experiment and promote the participatory approach. This development came from an iterative learning process. It was influenced by discussions with the GPSA team, who introduced learning from neighboring countries (in particular, CARE International’s Integrating Social Accountability in Development Education [LEAD] project in Morocco 2015-2019). Ironically, perhaps, the end of the project was marked by a forced retreat to the platform due to the pandemic, but this time with actors already trained and sensitized to social accountability approaches. However, the experience has shown that the platform itself is not sufficient in the Mauritanian context, which is marked by a lack of familiarity with IT tools and data. What does the TOME project tell us about the GPSA Theory of Action? The general theory seems to be validated on many points. In terms of the hypotheses of what allows the dynamics to be put in place, we note the importance of (1) the willingness of the actors, including the government, to put in place a new dynamic, which in the Mauritanian case comes from the realization of the impasse and the difficulties of the education sector, which are an observation shared by all at the beginning of the TOME project; (2) the development of the project within a framework of institutional reform, which in the case of TOME is above all the development of the ideas of decentralization and putting the education sector at the top of the government’s agenda (and even more so after the Presidential election of 2019); and (3) the formation of a 59 broad coalition in favor of an alternative approach to the education sector. TOME is a project that is, very clearly, at the beginning of a possible broader change. Some of the actions being taken bear fruit and are inspiring. Various steps towards a medium-term effect are visible, including the scaling up of a key mechanism for accountability and the setting up of multi- stakeholder reflection processes. However, it is too early to speak of long-term effects or spontaneous collaborations. Things worked extremely well in the one school where the whole package of interventions could be delivered, but, so far, many school stakeholders see changes in processes and relationships rather than significant changes in school performance and learning levels. The TOME experience allows us to refine or formulate some additional hypotheses on the TOME Theory of Action, notably that: (1) the demonstration of the value and effectiveness of social accountability approaches can be done based on a small number of pil ot experiences  -  in many ways it is mainly the example of one school that TOME uses to push and promote its approach; (2) important questions about sustainability arise if the accountability and participation approach is without means: the stakes are de facto low as there are no real resources to manage (no budget) beyond the rational use of already very scarce material and human resources. The risk is then that enthusiasm and energy will run out; and (3) a condition for change is the ability to mobilize broadly (which the GPSA Theory of Action already talks about), but the TOME project shows how this ability and the discussion forums that are created should not all be seen in a purely formal sense - beyond committees and official meetings, what makes TOME different is the positioning of its staff between actors, for example between the national education administration (with contacts and a history of certain team members at the ministry) and the federation of parents’ associations, municipalities and teachers. Pushing further, it should be noted that what ‘saved’ the project and allowed for the full experience of CPSE/COGES to be implemented in one school is TOME’s ability to go beyond the actors initially targeted by the project and to mobilize a new actor not included in the initial strategy (the former pupils, who agreed to contribute to the budget needed to implement the idea of social accountability fully). 10. Recommendations To the Mauritanian government and its partners: • Give schools their own local budget. Without a proper budget (even a relatively small one), the action plans and resolutions of the COGESs (and even the COGES institution in itself) are, in part, futile. While there is room for improvement in some areas that do not require financial resources (such as teachers absenteeism, although the TOME reports suggest that this is not necessarily a huge problem), a management committee is only meaningful if there are issues and something to manage (Fox, 2015). The higher the stakes and the more weight the committee has, the more seriously the approach is likely to be taken. It is not for us to suggest a source for this local budget, but a report produced by TOME gives some starting ideas and points in particular to the continuation of the process of decentralization and devolution in the education sector. In the meantime, the PASEB II project provides an interim solution with funding a 60 school budget in five wilayas (the experience is useful and important, but not sustainable as it stands). • Continue to emphasize, explain and train on the principle of social accountability , including the COGESs. We found that newcomers to the sector are often unfamiliar with the TOME experience and the principles of social accountability. Without continued efforts to raise awareness and set the agenda, gains could quickly be lost. To the World Bank: • The TOME project and its evaluation show the limits of the flexibility of the funding obtained by Ecodev and its partners. A major element identified along the way - the need for CSPEs to have a budget to manage in order to practice social accountability, further develop their skills, and simply establish their legitimacy - remained unresolved throughout the project. For World Bank internal administrative reasons, it was not possible to adapt the funding - even though all parties agreed on the need for such an adaptation. Developing fully adaptive funding arrangements is essential to the success of experimental projects (such as TOME). • Rethink the time frame. Four years (after no-cost extension) is a long period for a development project, but it is still too short. Our report shows how most CSPEs had only just been trained when the project ended (a situation exacerbated by COVID-19). The start-up phase of this kind of project, which can be lengthy, is essential, as (1) there is a lot to learn and (2) it is necessary to build a whole infrastructure of human relations to set up a new institution like the CSPE/COGES. A longer project would allow not only to develop tools, which has been done, but also to test and refine them. There was only one test of the IT platform and the school development plans would have also benefitted from a back and forth to refine them. To the Mauritanian government and its partners in the implementation of the COGESs: • Developing support for COGESs. Following the launch of 1,000 COGESs and in the light of the TOME experience, it is clear that it will be important to continue to support COGESs over time. Legislation alone will not make them work. The TOME project suggests that a local approach, with the provision of an agent - for example, at the municipality level - with the role of facilitating the work of the COGES, would be valuable. At the departmental level, where there is not yet a structure provided for by law to continue the work of the CSPDs (unlike the COGESs, which can to some extent replace and extend the work of the CSPEs), this support is also necessary  - because the policy of devolution and decentralization also passes through this level. • Establish a system of continuous training. Members of local participatory institutions, like all education actors (including COGESs/CNP), change over time, and it is important to think now about a sustainable mechanism to ensure the training and competence of actors. Various complementary mechanisms are possible, including having a local facilitator, cascading training (with a member of the COGES specifically responsible for the training of their peers), or sharing of experience between committees 61 (which can be organized at the departmental or national level for departmental committees). To those involved in the development and promotion of social accountability: • Continue to develop positive examples. Actors in the field speak with great interest about the example of the one school supported more intensively by TOME. Documenting and sharing success stories, as TOME has done, is an excellent way to promote the approach and generate interest. The production of a short film or direct experience sharing could be key. • Reflect on the role of the IT tool. The experience of the platform is mixed; it is not clear that it is being fully used, and it would be worth considering in more detail whether this is a useful area to focus on at this stage given the context and the need first to build an understanding of the social accountability approach. If the answer is yes, it will be a matter of working on familiarization with the tool and the level of connectivity. In any case, it would be useful to also offer an offline experience via (digital or paper-based) display in the schools. 62 References Befani, B., & Stedman-Bryce, G. 2017. "Process Tracing and Bayesian Updating for impact evaluation." Evaluation, 23 (1), 42-60. https://doi.org/10.1177/1356389016654584 ECODEV. 2018. Analyse de la Faisabilite de l'introduction d'un Processus de Redevabilite Sociale en vue d'ameliorer l'apprentissage pour Tous les Eleves Mauritaniens ECODEV. 2019. Rapport des expériences et des leçons apprises. Projet TOME. Juin 2019 ECODEV. 2021. GPSA Grantee Final Completion Report Falisse, J.-B., & Ntakarutimana, L. 2020. "When information is not power: Community-elected health facility committees and health facility performance indicators." Social Science and Medicine, 265, 113,331. https://doi.org/10.1016/j.socscimed.2020.113331 Fox, J. A. 2015. "Social Accountability: What Does the Evidence Really Say?" World Development, 72, 346–361. https://doi.org/10.1016/j.worlddev.2015.03.011 GPSA. 2019. Monitoring, Evaluation, Research and Learning (MERL) Handbook. The World Bank GPSA. 2020. “The Global Partnership for Social Accountability: Theory of Action.” Version September 2, 2020. Washington, D.C. https://documents1.worldbank.org/curated/en/425301607358292998/pdf/The-Global- Partnership-for-Social-Accountability-Theory-of-Action.pdf GPSA. 2021. Annex E. Indicative Draft Results Framework of the GPSA. Revised Draft - July 21, 2021. The World Bank GPSA. forthcoming. “Monitoring, Evaluation, Reporting, and Learning Guide for GPSA Grant Partners and Consultants.” Washington, D.C. Grandvoinnet, H., Aslam, G., & Raha, S. 2015. "Opening the Black Box: The Contextual Drivers of Social Accountability." In World Bank Group (Issue 3). The World Bank. https://doi.org/10.1596/978-1-4648-0481-6 Lekweiry, M. 2018. Rapport de l’etude baseline du projet TOME. ECODEV Mansuri, G., & Rao, V. 2013. "Can participation be induced? Some evidence from developing countries." Critical Review of International Social and Political Philosophy, 16(2), 284 –304. https://doi.org/10.1080/13698230.2012.757918 McCoy, D. C., Hall, J. A., & Ridge, M. 2012. "A systematic review of the literature for evidence on health facility committees in low- and middle-income countries." Health Policy and Planning, 27 (6), 449–466. https://doi.org/10.1093/heapol/czr077 Ministère de l’Education Nationale et de la Réforme du système Educatif (MEN). 2021. Concertations Nationales sur la Réforme du Système Educatif National 63 Ministère de l'enseignement fondamental et secondaire (MEFS). 2006. Rapport d’Etat sur le Système Éducatif National Osmani, S. 2008. "Participatory Governance: An Overview of Issues and Evidence." Building Trust Through Civic Engagement (pp. 9–44). United Nations. Pawson, R., & Tilley, N. 1994. "What works in evaluation research?" British Journal of Criminology, 34(3), 291–306. https://doi.org/10.1093/oxfordjournals.bjc.a048424 Projet d’Appui à la Qualité de l’Enseignement Fondamental ( PAQEF). 2020. Manuel des procédures des Fonds d’Appui à la Qualité de l’Enseignement Fondamental (MP/FAQEF) World Bank. 2015. “Opening the Black Box: Contextual Drivers of Social Accountability” World Bank. 2018b. “Learning to Realize Education’s Promise.” World Development Report 2018. Washington, D.C. https://www.worldbank.org/en/publication/wdr2018 64 Annexes A. List of interviews Type Function 1 Maintenance President Ecodev 2 Maintenance Executive Secretary Ecodev 3 Maintenance Former head of the TOME project 4 Maintenance Parliament: President NPC 5 Maintenance Parliament: Member of Parliament, NPC 7 Maintenance President FENAPERIM 8 Maintenance Education Project Manager, World Bank 9 Maintenance Education Project Officer, WB 10 Maintenance Technical Advisor MENRSE 11 Maintenance Technical Advisor MENRSE 12 Maintenance DEF/MENRSE 13 Maintenance President COMEDUC 14 Maintenance Mayor of Tidjikja: CSPD Tagant 15 Maintenance IDEN Tidjikja: CSPD 16 Maintenance Director El Medina School 17 Maintenance Director School Zira 18 Maintenance School 3 Pupil’s parent (CSPE member) 19 Focus group El Medina School 20 Focus group Zira School 20 Focus group School 3 22 Maintenance Mayor: CSPD Riyadh 23 Maintenance District Inspector: CSPD 24 Maintenance Director Wejaha School 25 Maintenance Teacher representative Fodé Diaguily School 26 Maintenance Director Talha School 27 Focus group Wejaha School 28 Focus group Talha School 29 Focus group Fodé Diaguily School B. Interview Guide Interviews 1. Introduction – what is your role in TOME and how did you come to work on the project? 2. If you had to explain the TOME project in 3 sentences to a stranger, what would you say? 65 3. In what ways do you think TOME is similar to, or different from, the main projects in the field of education (this may be at any level, on process, theme, actors, etc.)? 4. From your perspective, what worked best in TOME? Why or why not? What are the elements (e.g., people, institutions, processes, contextual elements) that made it work? 5. From your perspective, what was the least successful part of TOME? Why or why not? What elements (e.g., people, institutions, processes, contextual elements) made it more difficult at this level? 6. The TOME project is based on the idea of transparency and discussion of information, among other things budgetary, is the Mauritanian education context conducive to this transparency and discussion? Why or why not? Give examples and concrete cases. 7. The TOME project also relies on a strong collaboration between parents, schools, administration and civil society. What have you seen of this collaboration? Give examples and concrete cases. Is the Mauritanian education context conducive to this collaboration? Why or why not? 8. In the TOME project, who were the main ‘allies’ in pushing the idea of social accountability in schools, and why? 9. In the TOME project, were any individuals or institutions more reluctant to support the idea of social accountability in schools, and why? 10. Depending on the level: have you been involved in the School Participatory Monitoring Committee/Departmental Participatory Monitoring Committee/National Advocacy Committee? If so, what was your role? What issues did you address? Did you make any progress on these issues? Which ones and why? 11. Who are the key actors to ensure that the gains of TOME are not lost with the end of funding, and why? Do you think that TOME’s ideas on accountability will be taken up? If so, by whom and how? 12. Any other aspects you would like to discuss? Focus Groups • To start with, we suggest that you make a collective drawing of the school, and place the people you think are most important in it. This could be people who are in the school or people or institutions who are not. We will then talk about these people. • Can you explain what information flows between these people and actors and how, and how often? Who talks or communicates with whom, and about what? • From your perspective, what worked best in TOME? Why or why not? What are the elements (e.g., people, institutions, processes, contextual elements) that made it work? • From your perspective, what was the least successful part of TOME? Why or why not? What elements (e.g., people, institutions, processes, contextual elements) made it more difficult at this level? • The TOME project is based on the idea of transparency and discussion of information, among other things budgetary, is the Mauritanian education context conducive to this transparency and discussion? Why or why not? Give examples and concrete cases. • The TOME project also relies on a strong collaboration between parents, schools, administration and civil society. What have you seen of this collaboration? Give 66 examples and concrete cases. Is the Mauritanian education context conducive to this collaboration? Why or why not? • In the TOME project, who were the main ‘allies’ in pushing the idea of social accountability in schools, and why? • In the TOME project, were any individuals or institutions more reluctant to support the idea of social accountability in schools, and why? • Who are the key actors to ensure that the gains of TOME are not lost with the end of funding, and why? Do you think that TOME’s ideas on accountability will be taken up? If so, by whom and how? C. Example of Participatory Drawing 67 D. Project Results Framework Transparency of the Mauritanian education budget Project Development Objective (PDO): To improve the transparency of the basic education budget by monitoring and evaluating its implementation and quality through social accountability tools in the target regions of Mauritania PDO Level of Performance Indicators* Cumulative Target Values**. Data source/ Responsibility for Description Core Unit of Measurement Baseline Frequency (%) data collection 2017 2018 2019 2020 2021 Methodology Indicator 1: Increased collaboration Number of consolidated reports 0 0 2 4 6 7 Annually Final evaluation Project Team between stakeholders at the local level to co-produced on budget Consultant generate information that will lead to the transparency and quality of public improvement of basic education in education service targeted schools. Number of decisions taken at 0 0 0 1 4 8 Annually Final evaluation Project Team Indicator 2: Increase in the number of national level that have an impact Consultant actions inspired by citizen monitoring to on budget transparency and the improve budget transparency and quality of the public education education services service INTERIM RESULTS Intermediate Outcome, Component 1: Capacity Building and Partnership Framework for Civic Monitoring of Primary Education Intermediate outcome indicator 1 Mastery Number of actors trained using 0 30 60 80 120 240 Annually Training report Follow-up Project Team + of social accountability tools social accountability tools report + Mid-term evaluation Consultant Intermediate outcome indicator 2: Number of actors structured and 0 60 80 100 140 250 Annually Memos for the Project Team + Level of structuring and engagement of involved in the different implementation of CSPE, Consultant stakeholders in the social accountability committees CSPD and CNP and process. partnership agreements Intermediate Outcome, Component 2: Generation of citizen feedback Generation of citizen feedback through the social accountability process and collaborative engagement with public sector institutions Intermediate outcome indicator 1: Number of consolidated reports 0 0 2 4 6 7 Annually SE reports + Mid-term Project team + Stakeholders co-produce information on co-produced evaluation consultant budget transparency and quality of public education service Intermediate outcome indicator 2: Number of decisions taken on the 0 0 0 1 4 8 Annually SE reports + Mid-term Project team + Reactions of government stakeholders basis of citizen feedback to evaluation consultant based on citizen feedback. improve budget transparency and quality of public education service Intermediate Outcome, Component 3: Facilitation of knowledge and learning to strengthen the effectiveness of social accountability interventions and project management Intermediate outcome indicator 1: Number of lessons learned to 0 1 2 3 4 5 Annually ES report + Mid-term knowledge and lessons learned during the improve project management Project team and evaluation implementation of the project Intermediate outcome indicator 2: Sharing Number of communications shared 0 0 1 3 5 10 Annually ES report + Mid-term Project Team + and dissemination of lessons learned. and widely disseminated evaluation Consultant E. Comment on the Results Framework This annex qualifies the Results Framework, providing contextual elements to understand levels of achievement. In general, we recommend that more detailed attention is paid to the pre-project definition of some of the indicators, especially in components 2 and 3. Intermediate Outcome, Component 1 (Capacity building and partnership framework for civic monitoring of primary education). These indicators focus, quite typically, on processes, so there is a strong (but, again, quite typical) assumption that the process actually leads to the intended outcome, e.g., that training leads to learning. • Intermediate Outcome indicator 1: Mastery of social accountability tools. o No particular comment: the training courses (unit of measurement) have taken place, as evidenced by the training reports. It is assumed here that the training leads to a "mastery" of the tools. • Intermediate Outcome indicator 2: Level of structuring and engagement of stakeholders in the social accountability process. o No particular comment: the project reports do suggest that the number of actors involved in the committees (unit of measurement) is even higher than initially foreseen by the project. It is assumed here that "presence" in the committees is also a "commitment". Intermediate Outcome, Component 2 (Generation of citizen feedback through the social accountability process and collaborative engagement with public sector institutions) • Intermediate Outcome indicator 1: Stakeholder-generated information on budget transparency and quality of public education service. o No particular comment: the unit of measurement is the number of reports produced - and these reports seem to have been produced. • Intermediate Outcome indicator 2: Responsiveness of government stakeholders based on citizen feedback. o Please see Section 6. There is a fundamental issue of attribution here. It is very difficult to attribute decisions to a single factor or project, and our suggestion is to consider different levels of attribution. The table below shows what we see as possible attribution. Responsiveness of Government Stakeholders Attribution to TOME Decree No. 2019-039 of 1 March 2019 on the creation in Direct, TOME has significant each public school of a School Management Committee influence / inspiration (sets up (COGES) precursors to COGES) Decree No. 2019-038 of 1 March 2019 regulating the Indirect, TOME probably absenteeism of teachers in public primary and secondary influenced because it is a topic schools directly addressed in the pilot schools Regulations giving school headteachers, in coordination Direct, TOME weighs in / with COGES, the power to sanction absent teachers and inspires (via its pilot experience to report to the IDEN (Inspections Départementales de with CSPE) l'Education Nationale). Establishment of a mechanism for monitoring the Direct, TOME weighs operational performance of COGES through the heavily/inspires (such measure absenteeism rate of primary school teachers already exists for CSPE) Decree No. 2019-040 of 1 March 2019 instituting a Indirect and probably weak. It is strategic review of the skills of primary school teachers in possible that TOME has an order to ensure a minimum level of competence of basic influence (very low level of school teachers evidence). Regular evaluation of the teachers has been established in Indirect and probably weak. It is order to determine the level of mastery by the teachers of possible that TOME has an the contents taught in Arabic, French and Mathematics, as influence (very low level of well as the pedagogical and didactic practices evidence). Expenditure on the education sector has increased by 10.6 These measures are not related percent compared to the 2019 budget, representing 23.9 to TOME (but to election percent of the total state budget. promises and the pandemic) but Finally, the President of the Republic announced on 2 it remains possible that the September 2020 the allocation of additional resources to TOME experience affects the education of MRU 40 billion as part of a new post breakdown (very low level of COVID-19 economic take-off plan. evidence). Government has set up an embryonic system of distance Measure that is directly related learning to the pandemic. It remains possible that the TOME experience affects this system (low level of evidence). Launch of the national program to set up 1000 COGES Direct, TOME weighs heavily / on 13 July 2020 by the Minister of Basic Education and inspires National Education Reform The Mauritanian government in collaboration with Indirect, funding is not caused Counterpart International and local partners including by TOME but it seems that some Ecodev, is implementing a five-year (2019-2024) $22.5 of the implementation is inspired million integrated school feeding and nutrition program by TOME (low level of entitled "Food for Education and Nutrition for evidence). McGovern-Dole Children - The Future Belongs to Us!" Intermediate Outcome, Component 3 (Facilitation of knowledge and learning to strengthen the effectiveness of social accountability interventions and project management). The indicators in Component 3 were achieved through an action research process. This learning, and the lessons it produced, is one of the exit strategies for the project to feed into the wider process of improving public education policies in the country. The most concrete example is the establishment of a fund to finance COGES in six of the country's fifteen Wilayas as part of the Basic Education Sector Support Project (PASEB II). This being said, it should be noted that the two indicators for this component are defined in very broad terms, namely: (1) number of lessons learned to improve project management and (2) number of communications shared and widely disseminated. As explained in Section 6 of the report, what matters here is the type of lessons learned by the project, and whether they are shared, rather than their number. It seems complicated to set a target in terms of lessons and, in the absence of a strict definition of what a lesson is, a quantitative assessment seems futile. 70