Maghreb Technical Note on Multi – Stakeholder Justice Survey Introduction In the Maghreb and beyond, the justice system has the mission of being a safety net protecting citizens’ rights and property, allowing normal and safe economic activity while at the same time preserving society’s culture and core values and protecting the vulnerable. Managing justice institutions to deliver on this mission requires measuring their performance, because what gets measured gets done. In the Maghreb and Malta, relying on administrative data to measure justice system performance historically means that the institutions are not being measured and managed in a way that primarily serves their users. Multi-stakeholder justice surveys in other parts of the world have proven to be a measurement tool that can contribute to providing the required multi-faceted performance picture. When it comes to justice and the broader governance environment, the countries in the sub-region are far from being a monolithic bloc as measured by a range of third-party indices. They perform differently on various measures of public sector governance (as measured by the Worldwide Governance Indicators), so the context in which the justice system operates differs from country to country. As an EU member country, Malta consistently outperforms its neighbors in North Africa across all governance aspects, but with a relative weakness in the control of corruption. As a country affected by fragility and conflict, Libya performs poorly on all governance dimensions. Algeria, Morocco, and Tunisia constitute the middle field, with Algeria’s governance scores consistently below those of the other two. Graphs 1-6: Governance dimensions in Algeria, Libya, Malta, Morocco, and Tunisia (Worldwide Governance Indicators 2019) Graph 1: Voice and Accountability Graph 2: Political Stability/Absence of Violence Graph 3: Government Effectiveness Graph 4: Regulatory Quality Graph 5: Rule of Law Graph 6: Control of Corruption Source: Worldwide Governance Indicators, available at https://info.worldbank.org/governance/wgi/Home/Reports. On various dimensions of justice system performance, cross-country evaluations shed some light on key strengths and weaknesses. They include timeliness, cost, quality of judicial processes, and judicial 1 independence. On the latter, Tunisia performs relatively well, but Morocco is assessed by the Bertelsmann Transformation Index with a score of 2 out of 10, lower than Libya (3/10) and Algeria (4/10). Graph 7: Judicial independence across MENA in 20201 Judicial independence (score 0-10), Bertelsmann Transformation Index 2020 Tunisia Lebanon Kuwait Jordan Iraq Egypt, Arab Rep. Algeria United Arab Emirates Qatar Libya Djibouti Yemen Saudi-Arabia Oman Morocco Iran, Islamic Rep. Bahrain 0 1 2 3 4 5 6 7 Source: Bertelsmann Stiftung, 2020. Available at https://www.bti-project.org/en/home. When it comes to timeliness, the differences are less stark. According to the World Bank’s Doing Business report and its indicator on Enforcing Contracts, Malta and Morocco are among the better performers in the MENA region, followed by Tunisia and Algeria with Libya being among the lowest performing. Graph 8: Timeliness of courts in enforcing contracts (Doing Business in 2020) 1 Malta is not covered in the data-set. 2 Time to enforce a contract (in days) 1200 1000 800 600 400 200 0 Source: Doing Business in 2020. Available at www.doingbusiness.org. The differences are even less significant when assessing the cost for users. While the share of court fees, lawyer fees, and enforcement fees varies, Algeria, Malta, and Tunisia, on the one hand, perform slightly better than Libya and Morocco, on the other hand. Graph 9: Cost of Enforcing Contracts (Doing Business in 2020) 3 Cost of enforcing contracts across MENA 40 35 30 25 20 15 10 5 0 Lawyer fees Court fees Enforcement fees Source: Doing Business in 2020. Available at www.doingbusiness.org. Under the Doing Business Quality of Judicial Processes Index, there is some variation in performance. The differences are largely due to the use of case management approaches and automation in those countries that perform better under this index. Given that even Malta’s comparatively high score of 10.5 out of 18 is still low in absolute terms, there remains a lot that can be improved across the Maghreb. Graph 10: Quality of Judicial Processes Index (Doing Business in 2020) 4 Quality of judicial processes index 12 10 8 6 4 2 0 Libya Algeria Tunisia Morocco Malta Court structure and proceedings Case management Automation ADR Source: Doing Business in 2020. Available at www.doingbusiness.org. This birds-eye view of justice system performance in the Maghreb and Malta indicates that despite areas of better performance there is room for improvement. That is where performance measurement and management become key and center. Even if a justice system is functioning in a stable, unchanging environment, it can always be improved. And in today’s rapidly changing society, justice systems need to adapt to their environment and to society as they change in order to provide fair and efficient services as their part of the social contract. A user-centric perspective focused on justice service delivery therefore requires a data generation effort that goes beyond traditional justice statistics and expert assessments to capture the justice journey of users while capitalizing on the insights of a broader variety of justice stakeholders to also reflect systemic constraints for service delivery. Box 1: How the Justice system contributes to economic and social development In the Maghreb as elsewhere, a well-functioning justice system protects the physical and material safety of the citizens it serves and gives them a guarantee that their rights, property, and values will be protected. An effective, fair and efficient justice system is also a necessary condition for successful economic activity and growth since it is the ultimate mediator in all business relations. Empirical studies have revealed the importance of law and legal institutions to enhancing growth, promoting secure property rights, and improving access to credit. After all, even in the presence of fair and stable laws, well-functioning justice institutions are required to uphold them. Laws establish the entire playing field on which firms operate, from the regimes used to access land, capital, and labor, to the rate at which they will be taxed. Predictability and a level playing field in the application of the law are critical for contestability and investor confidence. Key areas where courts are critical include helping firms resolve contractual disputes with other firms, dealing with insolvency, providing protections against unlawful regulatory changes and land appropriation, challenging administrative decisions on permits and licenses, and handling labor disputes. Strong and fair justice institutions are also a prerequisite for 5 leveling the playing field between unevenly powerful stakeholders. Courts need to be strong so they can resist pressure from influential parties. They need to be fair, because poor and marginalized populations, otherwise tend to be victimized by wealthier and better-connected parties to disputes and by the exercise of the state’s coercive power. Poverty is more than low income and social development focuses on the need to put people first in development processes. Courts play a key role here when laws are passed to advance social development. These policies will remain dead letters if the target populations are not willing or able to claim their rights and seek the protection of the courts against abuse based on contrary social norms and local power structures. Trust in justice institutions by poor and marginalized populations is therefore particularly important so the courts can play their role effectively. Importance and types of information The first step towards improvement of the justice system involves collecting data that can be used to produce information on performance. This can provide an overview on the functioning of the justice system, pinpointing areas for improvement and measuring the impact of changes. Some data about the functioning of the justice system are already available, others need to be collected for a specific purpose, for instance with surveys. Likewise, both types of information tracked over time can be used to measure the impact of specific changes and reforms, but the perspectives are often different. Typically, two types of information are needed: information from data registered by courts (and other entities) on their workloads and how they handle them; and information through surveys and similar methods, on how users, internal actors, and intermediaries experience and perceive the process. The remainder of this Note focuses on the second type of information. Such survey information is equally important in determining where improvement is most needed. For example, it can provide information on whether stakeholders of various types believe the results are, for instance, fair, equitable, influenced by corruption or political intervention. Moreover, it can indicate how citizens of all types prioritize apparent failings – what statistics indicate may not be where their greatest concerns lie. People-centered justice requires that these perspectives also be used, not only to identify problems but also to determine whether they have been resolved or at least reduced. Information about the functioning of the system is essential in identifying priority areas for reforms and in highlighting priority interventions. Data also provides information about the success or failure of interventions. Obtaining data before the reform to set a baseline, and then obtaining data again for comparison afterwards, will produce information to establish if, and to what extent, an intervention had the intended effect. Types of data Surveys are usually a necessary tool to collect data about where court users and future users want remedies. Statistics on court cases alone do not provide enough information for decisions about reforms. Already available data usually includes statistics about court cases or other services provided by the justice system. Courts usually record numbers of entries and dispositions in a certain time period as well as the number of pending cases. Similar records may be kept within all sector entities – prosecution, legal aid agencies, notaries, private bailiffs, ADR systems and sometimes also by traditional or customary 6 dispute resolution systems. In some countries, automated CMISs (Case Management Information Systems) will record more information on each case, thus allowing more detailed analysis. A much broader specter of data than that already available is needed for informed decisions about reforms in the justice system. For this reason surveys are usually necessary as a tool to collect this data. Surveys are a targeted tool, and they can be designed to collect necessary, targeted information. They can also provide focus on specific interesting areas of the justice system. Surveys are an adaptable tool essential for gathering needed information. However, surveys require extra resources. Targeted data collection efforts can range from very small and specific exercises like focus groups or individual in-depth- interviews in a specific region or court and focused on a very specific issue. Or they can cover one target group, for instance a general population survey on perceptions of the judicial system. Complex multi- stakeholder data collection efforts can cover a wide variety of issues and collect abundant data about multiple aspects of judicial system. 7 Why multi-stakeholder surveys A multi-stakeholder survey provides the broadest A group of blind men heard that a strange possible view on sector performance. This type of animal, called an elephant, had been brought to survey allows for capturing the views and the town, but none of them were aware of its experiences of a range of relevant types of shape and form so they sought to inspect it by stakeholders, including (potential) users (e.g. touch. The first person, whose hand landed on the citizens, businesses), service providers (e.g. judges, trunk said: "This being is like a thick snake". For court staff) and intermediaries (e.g. lawyers) Its another one whose hand reached its ear, it advantage is in the multiple perspectives on the seemed like a kind of fan. The third with his hand upon its leg said the elephant is like a tree-trunk. performance of the justice system it can provide. The one placing his hand upon its side said the Each stakeholder or potential participant of the elephant "is a wall". Another who felt its tail system has a specific view of the system and the described it as a rope. The last felt its tusk stating best way to understand the system as a whole is to the elephant is like a spear. take all of their views and combine them. Moreover, comparing views of those with and without experience with the justice system can provide a valuable insight into potential reform areas. Comparing views of justice service providers to those of justice service users on the same issue is another example of useful information that multi-stakeholder surveys provide. There are many other examples of specific viewpoints that can, together, build a comprehensive picture of the justice system. Multi-stakeholder surveys can also provide information that is specific for some stakeholders: like satisfaction with working conditions among service providers, or Only one in seven citizens is familiar with free legal aid. They availability of information for users. In the are actually outnumbered by those “knowing� there is none. process of designing instruments for multi- On the other hand judges estimate that two out of three citizens are informed about free legal aid. stakeholder surveys parts of instruments are Source: Georgia multi-stakeholder justice survey, 2019 always designed to cover the same issues (e.g., duration of court cases or perception of courts’ efficiency) while there are parts of instruments for each respondent group that are specific for that group (e.g., actual and optimal number of cases for prosecutors and judges, or awareness of free legal aid for citizens). Multi-stakeholder surveys also allow for specifically designed approaches to each group of respondents. Sometimes it may be face-to-face interview. Sometimes the optimal option may be web-based questionnaire or a telephone interview. All these elements are carefully considered and weighted during the development of a multi-stakeholder survey to maximize its usefulness. Finally, multi-stakeholder surveys raise the involvement and participation of all justice system stakeholders in the reform process, providing it with more leverage and support. 8 How to conduct multi-stakeholder surveys Implementing a multi-stakeholder survey of the justice system requires thorough preparation, careful execution and a well thought out definition of outputs. It is important to balance the cost of a survey with the resources available. This can be done in two ways: by tailoring the scope (or measuring area) of the data collection effort, or by reducing the cost of data collection effort itself by employing more “cost efficient� ways to collect data. A brief outline of the most critical implementation steps covers the three main stages of such a survey: preparation, execution, and reporting. Preparation The preparation stage is a necessary step to achieve consensus on survey goals and targets, obtain endorsement and support for the survey as well as to determine necessary resources for the survey. In the preparation stage it is crucial to identify and involve all key stakeholders. This should include representatives from the executive branch (ministry) and from the judiciary and the prosecution. Lawyers, notaries, and bailiffs should also be involved. Involving all stakeholders from the earliest stages has many benefits. It generates broad support for the survey, facilitates the execution of the survey and enables feedback from all groups leading to better and more comprehensive survey coverage of various issues. After involving all stakeholders, it is prudent to appoint a project manager: a person (or persons) who would facilitate the flow of ideas and decisions between stakeholders. A project manager should be more than just a “communication hub� however. This person can and should make certain decisions during the survey process autonomously to avoid a lengthy decision-making process for every detail. The mandate for such a manager should therefore be defined clearly. In addition to appointing a project manager, it is a good idea for each stakeholder institution to appoint a “contact person� within it to facilitate the flow of ideas, information, and decisions. The group of all appointed contact persons can also be considered as an advisory body to the project manager. Beside appointing a project manager, it is recommended to build a multi-disciplinary team for this exercise. As the minimum recommended such team should have a survey specialist and a legal expert that can serve as advisors and guides during all stages of the survey. In the preparation stage a consensus needs to be reached about the exact scope or objective of the survey. The question “What do we want?� needs to be answered. This may sound simple, but the answer to that question drives the preparation process forward to determining respondent groups, issues to be covered in the survey, and size of the survey and, very importantly, to identifying necessary resources for it. An especially important part of the preparation process is to identify potential partners in survey execution. It is not recommended to attempt to execute a complex multi-stakeholder survey using only existing staff employed in various judicial or ministry institutions. Multi-stakeholder justice surveys require a professional survey contractor with sufficient knowledge, experience, and implementation capacity. It is important to identify potential partners in the preparation stage because their existence (or lack of it) and capacity (or lack of it) may drive some decisions in the preparation stage regarding survey scope, complexity, and coverage. A survey process needs to be based on a professionally designed methodology that guarantees that collected data are sufficiently reliable, valid, and objective. Data needs to be collected using instruments also designed by professionals. Survey instruments need to be designed by professionals to ensure that 9 all respondents will understand them, understand them in the same way, and have no difficulties answering them in a way that is clear and unambiguous to those who will use the collected data. Finally, a data collection process needs to maximize honesty and thoroughness of the information provided by respondents. Data collection should also entail certain safety mechanisms preventing errors and manipulation during the data collection process. If available resources are scarce, it may be advisable to be modest in determining survey scope. The preparation stage should result in some outputs and decisions that are necessary to proceed to survey execution: - Support from various stakeholder groups - Appointment of Project Manager and definition of his/her mandate - A clear outline of the areas and issues that the survey should cover and questions it should answer - A clear idea about the scope of the survey and target respondent groups - A clear idea about the resources survey requires and how will they be obtained - Identification of potential partners for the survey. Execution The execution of the survey includes the selection of a survey contractor, preparation of instruments and methodology, their application in the field and analysis of collected data. After the successful completion of preparatory activities the survey can move to the execution stage. The first step in the execution stage would be the selection of partner / survey contractor responsible for technical implementation of the survey. The selected contractor needs to have knowledge, experience, and capacity to carry out a complex survey like this one. Once the contractor is selected, they will take the lead in developing the survey instruments based on the decisions made in the preparation stage. The contractor should also develop the methodology to collect data from multiple respondent groups. The process of developing the instruments and methodology is an iterative one and should include multiple levels of communication between the stakeholders and the contractor. When stakeholders and contractor are satisfied with the pilot instruments and methodology, the process moves to the testing phase (the pilot) where both the instruments and methodology are tested to determine their applicability, find potential errors, and identify necessary improvements. This testing process needs to be completed in this phase with care and attention. It should not be skipped or hurried. Proceeding to the main fieldwork with untested instruments or methodology can cost much more than the few days it takes to complete the testing properly. Upon pilot completion, the results should be discussed between the contractor and stakeholders, and decisions reached about eventual changes in the instruments or methodology. Final instruments and methodology should be developed and agreed upon. Upon finalization of the instruments and methodology the survey may move into the main fieldwork phase. Although this phase is mostly in purview of the contractor, certain actions from stakeholders may help with the fieldwork execution and make it smoother, faster, and more successful. The most important ones, soliciting or providing support letters from various institutions, can provide additional legitimacy to the survey and may increase the response rates among “hard to reach� populations (like judges or prosecutors). Such letters may be issued by government representatives (both executive and judicial 10 branch), professional associations (bars, bailiff and notary chambers, etc.), but also by various NGOs or international institutions. Once the data is collected it must be analyzed. It is recommended to define and agree in advance on the exact statistical procedures to be used as well as format and content of data analysis. Reporting There are two basic types of reports as final outputs of multi-stakeholder justice surveys. First, a technical (or survey) report captures basic statistical findings of the survey, analyzes differences between various (sub)groups but does not provide legal explanations or recommendations. Second, an analytical report is usually built on the foundation of the technical survey report and adds an analytical review of the findings, explanations and (sometimes) recommendations. The final outputs of the multi- stakeholder survey will include datasets, data tables and, of course, reports. As soon as the survey instruments are finished, it is particularly important to reach an agreement with the survey company on the format and content of all outputs before the data collection is finished. This may save a lot of time and resources in the final stage of the survey. Usually, the report structure follows the survey outline agreed upon in the preparation stage of the survey. It can be organized around pre-agreed “dimensions� of the justice system that the survey will measure (e.g., efficiency, quality, accessibility, and integrity) or it can follow some other pattern suitable to stakeholders. The technical (or survey) report is usually written by the survey contractor. Analytical reports, however, are usually beyond the capacity of survey contractors who generally specialize in fieldwork. They are usually written by legal expert(s) using the survey report as a base and building on it. Finally, once the survey outputs are ready, survey findings need to be disseminated. It is up to the stakeholders to determine how and to whom. As with many other tasks in this When a multi-stakeholder survey is done in several countries it allows for survey it is very prudent to comparison of data across countries. Here: citizens’ trust in justice system. Source: Multi-stakeholder justice survey, 2019 reach an agreement on the methods and audiences for survey output dissemination in the preparation stages of the survey. How, where and to whom the findings will be presented should not depend on the actual survey findings. How to avoid some common problems Such multi-stakeholder justice surveys have not been implemented in the Maghreb and Malta. Since the practice is well-established in other parts of the world, Algeria, Libya, Malta, Morocco, and Tunisia can build their efforts on lessons learned from previous survey exercises in other countries. Only the most salient lesson are briefly touched upon here. Quality, quantity, and level of detail in collected data are directly correlated to resources allocated for a survey effort. It is important to manage expectations and recognize what resources are available. The 11 survey scope must be set accordingly, before the survey even begins, rather than risking that corners will need to be cut during the various survey stages. Every data collection effort carries a certain level of error within it. It is important to employ all available mechanisms to minimize them. Sampling errors have their source in the mere fact that it is not possible to cover the whole population with some survey exercise. By selecting only a part of that population (a sample) to be surveyed (and not the whole population), no matter how hard we try to make this part representative, the results we get will always somewhat deviate from “real� values in the population. The estimate of such deviation is referred to as the sampling error. Other errors may occur during the survey process, like those due to self-selection or response bias or misinterpreted questions. These error sources can be reduced by adequate pre-testing of questionnaires and approaches (e.g., face-to-face, phone, or internet application) during the pilot phase with a comparable test population to make adjustments where possible. For example, while face-to-face interviews typically have higher response rates, they may intimidate some groups who may do better with online questionnaires. The above-mentioned additional measures can increase the time and resources needed but are necessary to maximize the quality of the data collection methodology, instruments, and collection process. Unexpected events are always possible, and it is important to have capacity to face them. The COVID- 19 pandemic had a huge impact on survey execution in many countries, for example on the method of survey administration. Such “force majeure� events are always a possibility, and this is another reason for engaging survey professionals in multi-stakeholder justice surveys, as they are the ones with the knowledge, resources, and capacity to deal with unexpected situations. Support and assistance from all stakeholders is important and welcome. All stakeholders are essential in an effort as complex as a multi-stakeholder survey. Multi-stakeholder support is needed at all survey stages, from the determination of the survey scope, construction of instruments and methodology, data collection, report writing and finally dissemination of results. Although some stakeholders may seem less important than others, previous surveys of this type demonstrate that support from all groups can be meaningful. It is strongly recommended to make every effort to include all stakeholders in this exercise from its very beginning. What to use findings for and whether to repeat the survey A properly done survey should provide us with the answers to the problems set out in the preparation stage. The survey’s main contributions may be roughly summarized into two categories: 1. Diagnostic value A multi-stakeholder justice survey, properly done, is a great tool to determine what elements of the justice system need reform most. In other words: which parts of the justice system would most benefit from some intervention. The survey can do more than just identify the area that would most benefit from reforms. It can also point to the directions for these reforms or interventions. 2. Baseline value 12 The survey also provides baseline data which can be used in the future to measure improvements (or lack thereof) after an intervention has taken place. It can be used as a tool to compare the initial situation with the situation after reforms and measure the impact of the reforms. Of course, to measure such impact, a survey would need to be repeated after certain actions were taken. It should be noted that a significant portion of resources required for a multi-stakeholder justice survey is spent at the preparation stage where the methodological approach and When a multi-stakeholder survey is done in several waves instruments are developed. But once they are tracking changes on selected dimensions of judicial system is possible. developed, they can be re-used at lower cost Source: Serbia multi-stakeholder justice survey, 2009 and 2013 and faster than during the baseline Perceptions of users of justice system – percentages of positive evaluations implementation. There are always adaptations and fine-tuning, but the main principles and the instruments’ “core� remain the same. Repeated waves of multi-stakeholder surveys can provide an answer about the success (or failure) of reforms and can also be used as a base for new interventions thus starting a new reform cycle. Conclusion Multi-stakeholder justice surveys would be a great asset for justice reform initiatives in the Maghreb and Malta. The shift in well-performing justice systems elsewhere towards a more user-centric approach focusing on justice service delivery to citizens and businesses requires the implementation of a tool that will generate relevant insights and data to capture justice dimensions that current approaches are unable to measure. The fact that this methodology has been tested and proven successful elsewhere offers an opportunity to build on such experiences and to ensure maximum impact with the objective of improving the performance of justice institutions in terms of service delivery and resource management. 13