72928 19/2012 Estimating Staffing Needs in the Justice Sector Dr. Heike Gramckow Table of Contents Foreword ..................................................................................................................................... ii About the Author ...................................................................................................................... ii Abstract ........................................................................................................................................ 1 1. The Evolution of Methods for Estimating Staffing Needs ....................................... 1 1.1 Improved Ways to Measure Staffing Needs...................................................................... 2 1.2. Experiences in Europe ............................................................................................................. 3 2. Methodologies Used for Developing Staffing Estimates ......................................... 5 2.1 Analytical Methods: Normative Analysis, Regression Analysis, Computer Modeling ................................................................................................................................................ 7 2.2 Delphi Method.............................................................................................................................. 9 2.3 The Weighted Caseload Method and the Development of Workload Models .. 10 2.4 Case Weighting Studies and Process Improvements ................................................. 13 2.5 Work Not Directly Case-Related: Moving to Full Workload Assessments ........ 15 3. Weighted Workload Studies: What is Measured and How? ............................... 16 4. Reflecting Workload Variations by Profession ...................................................... 24 5. Estimating Future Staffing Needs ................................................................................ 25 6. Efficiency, Quality, and Other Performance Measures ........................................ 26 6.1 Efficiency Considerations ..................................................................................................... 26 6.2 Public Trust and Confidence ............................................................................................... 27 6.3 Quality Services and Decisions ........................................................................................... 27 6.4 Access to Justice and Cost Effectiveness ......................................................................... 28 6.5 Establishing Standards .......................................................................................................... 28 6.6 Improving Performance Measures ................................................................................... 31 7. Conclusions ........................................................................................................................ 32 References ................................................................................................................................ 34 i Foreword by the Editorial Committee Across the globe, many justice sector agencies are facing an ever expanding workload and increasing pressure to reduce costs. Courts and other justice system agencies are thus finding themselves in need of better methods to estimate staffing requirements in order to justify budget requests for increased numbers of judges, prosecutors, and other court staff. Yet, estimating staffing requirements is not nearly as straightforward a process as might first appear, as the more uncomplicated methods—using the number of cases handled as a basis, for example—have proven to be highly inaccurate. It has thus become increasingly clear that in order to more accurately estimate the staffing needs of any justice sector agency, it is crucial to examine not only the varying complexity of the cases handled, but the amount of time each case actually requires from each justice sector staff member as well as time required for non-case ralted work—that is, the actual workload involved. This paper addresses the increasingly critical need for more precise measures of justice sector staff workloads, first by examining a variety of the newer, more sophisticated methods of estimating staffing needs, and second by presenting a deeper analysis of the most precise method, the weighted workload study, as well as a ―how to‖ on how such a study can be put into practice. About the Author Dr. Heike Gramckow is a Senior Counsel with the Justice Reform Group in the Legal Vice Presidential Unit of The World Bank in Washington, DC, where she advises on justice reform issues and conducts related research. She has been working on justice sector reform issues for over 25 years. Before joining the Bank, Gramckow was Deputy Director of the International Division of the National Center for State Courts and served as Director for Research and Development for the American Prosecutors Research Institute. During her career, Gramckow has worked with courts, prosecutors, and police in the United States and throughout the world, especially on justice system management and reform. She has directed several justice reform programs, conducted justice system assessments, and provided advice on justice sector reform issues to the governments of numerous countries, including Argentina, Bosnia, Bulgaria, Egypt, Haiti, Hungary, Indonesia, Kenya, Mexico. Mongolia, Morocco, Nigeria, Papua New Guinea, Romania, Saudi Arabia, Serbia, Syria, the United Arab Emirates, and the United States. She has worked not only with common law and civil law systems but also with Shari‘a courts and other traditional justice systems training judges, prosecutors, and court personnel on management, budget and strategic planning, reengineering, and particular substantive issues, such as victim assistance, drug cases, juvenile offenders, and domestic violence. She has also provided advice on establishing sustainable mechanisms for continuing education, introducing effective information technology and communications (ITC) ii solutions, and designing more participatory governance structures that support democratic and accountable agencies. Gramckow holds a law degree and a doctorate in law from the University of Hamburg, Germany. She has taught undergraduate and graduate courses on international criminal justice systems and juvenile justice at American University and the George Washington University in Washington, DC, and is widely published in the United States and internationally. iii iv Estimating Staffing Needs in the Justice Sector Dr. Heike P. Gramckow Abstract Justice system agencies around the world continue to seek adequate methods to estimate staffing needs. Especially when caseload rise and budgets are limited, the pressure is on to justify adequate staffing with solid data. The simpler approaches of basing staffing needs on number of cases filed or population numbers have proven to be imprecise at best and seriously flawed at worst. The search for better estimation measures first led to weighted caseload studies, which weighed the complexity and other special needs of different case types. As these methods still did not provide an accurate assessment, efforts continue to be made to develop a more precise measure of not just caseload but workload, a measure that factors in the time spent on managing the case and on the increasing amount of non-case-related work, such as administration, training, outreach, travel, etc. This paper describes the leading approaches (including the analytical, Delphi, and weighted caseload methods) used throughout the world for determining workload among justice sector employees, presenting the benefits and limitations of each. The paper then focuses on what is currently viewed as the more optimum method of the weighted workload study, and offers a step-by-step outline of how this kind of study can be developed and implemented. Also considered are the dual challenges of forecasting future staffing needs and incorporating performance measures to promote quality decision making and cost-efficient court procedures and services. 1. The Evolution of Methods for Estimating Staffing Needs Courts and other justice system agencies around the globe are struggling to develop a solid approach to estimating staffing needs for professional and support staff. Most justice systems are under constant pressure to reduce costs, often despite rising workloads and increasing demands for quality services. Accordingly, courts, prosecutors‘ offices, and other justice system agencies are taking a closer look at their operations, staffing, and other resource needs to identify whether more efficient use of staff time, staff reallocation across different activities, and perhaps automation could offset budget constraints. They are also looking for better methods to estimate staffing needs and develop solid justifications for budget request increases to hire needed judges, prosecutors, and other staff. 1 In an apparent effort to support judicial independence, some governments allocate a certain percentage of the overall budget to the courts,1 even setting a fixed number of judges and prosecutors to be allocated in a legal code, which then requires parliamentary action if adjustments are needed.2 As reasonable as fixing a budget percentage or a predetermined number of staff positions may sound in support of judicial independence and a well-functioning judicial system, this approach pays no attention to the actual staffing and resource needs or work fluctuations over time. It may also not be as helpful as envisioned for the judiciary. If staff and resource needs are greater than the preset number provided, the judiciary suffers; if the judiciary‘s needs are fewer, credibility might be jeopardized and courts can become an easy target of accusations that they are wasteful and unaccountable. Many countries have tried a range of methods to identify the number of judges and other positions needed in the courts and other justice sector agencies. Frequently, the number of cases processed or the size of the population served is used to estimate staffing needs. These are relatively easy measures, but they are also imprecise and can lead to serious under- or overestimates of the number of essential positions.3 Since the effort and time required to process a certain type of case varies according to its complexity (that is, the complexity of the case, not just the legal matter involved), one judge may be able to process a large number of simple cases in only one day, while another, who is handling a very complex case that involves multiple offenders and victims, cross-border issues, and complicated legal matters, may need to focus on just one case for days or even months. How much time a judge or prosecutor has to devote to a case will also depend on the staff support available, ease of access to information, information technology (IT) support, and a range of other factors, including what has been termed ―the local legal culture‖ (see Church 1986; Ostrom et al. 2005; Gramckow 2011). 1.1 Improved Ways to Measure Staffing Needs Better measures of an agency‘s workload are clearly required in order to determine staffing needs. For example, in the case of courts, this would mean that reliable information is available to Definitions estimate the number of judicial officers (judges, Caseload: the number of cases judicial assistants, and clerks) needed to complete all handled in a court or by a judge work and resolve all cases before the court in a timely Weighted caseload: caseload manner that does not compromise due process and the adjusted by case weights, i.e., quality of decisions. Such measures must take into the average time judges spent account the factors discussed above—the complexity on a specific case type of the cases, for example, and the fact that more Workload: caseload plus other work conducted weight should be given to cases that require more Weighted workload: workload time. adjusted by case weights 1 This phenomenon appears to have originated in Central America and is still in use in several countries there, such as Costa Rica and El Salvador, sometimes even backed by the consitution in those countries (McEldowney 2001; Popkin 2002). 2 This is, for example, still the case of Mongolia in 2011. See Gramckow and Allen (2011). 3 See section 3 for a larger discussion of the weaknesses of this simple case volume approach. 2 Building upon experiences from other sectors, more effective methods were developed to estimate workloads for justice sector staff, including judges and other court personnel, prosecutors‘ and public defenders‘ offices, and staff of corrections agencies. First to be instituted were weighted caseload studies, or studies that assess the time required by judges and other staff by weighing the complexity and other special needs of different case types. Today these kinds of studies are well established in the United States, with origins dating back to the mid-1970s (Ostrom et al. 2000, 3), and a rich database of U.S. research and other information related to such studies can be found (see, for example, Jacoby 1987; Flango et al.1993; Aikman, Pederson, and Roper 1998). Canada, the United Kingdom, and Australia also soon began to apply weighted caseload studies to the estimation of staffing needs for the judicial sector. As much of an improvement as this method was, today it is recognized that even estimating the weighted caseload may not be a precise enough way to estimate court staffing needs. Thus, efforts are increasingly being made to capture a truer ―workload‖ measure, that is, an estimate of the entirety of the work involved for judges and other justice sector staff in managing not just cases but also the often increasing non-case- related work, such as general administration, inter-agency coordination, community outreach, and so on. 1.2. Experiences in Europe The civil law countries in continental Europe did not begin to adopt these approaches until much later. Even in 2011, weighted caseload studies were still, with a few exceptions, not very well known and rarely applied there, and the understanding of workload continues to evolve (Lienhard and Kettiger 2011). Early examples of similar approaches have, however, been in Germany, for example, were as early as the 1960s, quite complex caseload and work analyses for all judicial and prosecutorial personnel were conducted (Lienhard and Kettiger 2011, 68). There, the Pensenschlüssel (position calculator) had been developed in 1961 and underwent a significant review in the mid-1970s and again in the late 1990s. This calculator combined a set of indicators that measured variations in judicial disposition rates as well as the time needed for administrative work, vacation and sick days, and training, and also provided for add-on rates to account for variations in state legislation, case mix, court settings, and population distribution. The development of this calculator involved representatives of the judiciary, prosecution, Ministry of Justice, academy, and private bar and was enormously time and resource intensive (Hirth 1997). This position calculator was adjusted throughout the years to reflect changes in legislation, work hours, and other elements. Not included, however, was a detailed assessment of the time actually needed to complete various processing steps for different types of cases in different locations (Hirth 1997). In 1996, the Ministry of Justice of the German state of Hesse failed to reach an agreement with the state‘s Judicial Council to adjust the number of positions needed, an adjustment that was triggered by a change in legislation according to calculations using the then current position calculator. At that 3 time, the debate over the national position calculator‘s ability to capture the relevant conditions in the German states intensified. As a result, almost 90 percent of the members of the judiciary in Germany supported a new time study that would mirror the more comprehensive methodologies applied in the United States. Using the experience of a similar study conducted in Austria, the German judiciary became intensely involved in developing the methodology for, collecting the data from, and analyzing the results of a three–six month study that required each judge selected for the study to record how he or she spent his or her time. This approach meant that judges were fully engaged and had a good understanding of the methodology and meaning of the data; it also meant that each judge selected for the study had to spend approximately 20–30 minutes daily completing time sheets (Deutscher Richterbund 1999). This was a significant commitment on their part. Nevertheless, the resultant data collection, while quite evolved, lacked some essential variables to reflect true case complexity rather than just legal complexity, as well as important local variations. Weighted caseload studies need to capture complexity in terms of the overall time needed to process a case from start to finish; greater legal complexity, however, is not necessarily equal to increased time requirements. Not every murder case is complex and time consuming, for example, while an otherwise straightforward case that involves many offenders and victims (especially child or special needs victims) or cross border issues will require more time. The chosen approach did not sufficiently consider these issues or the significant local differences. In the end, it was deemed that the German Pensenschlüssel, though useful, was developed with insufficient input from some key process participants, especially support staff, and was never linked to efforts to optimize operations. In 2004, a study conducted by Arthur Anderson led to a new approach to measuring staffing needs in Germany, the Personalbedarfsberechnungssystem (PEBB§Y, or Staffing Needs Calculation System) (Herrler 2004). While the PEBB§Y introduced a more sophisticated approach to measuring staffing needs in the justice sector, it also drew criticism from the beginning, particularly for not adequately reflecting local conditions, and follow-on studies continued to provide recommendations for changes (Deloitte Consulting 2005; Neue Richtervereinigung 2008). One of the challenges in Germany has been that the larger studies aimed at establishing staffing needs calculators focused on the national level, which, considering the significant variation in case mix, operations, and staff availability in different parts of the country, could not provide adequate estimates of local staffing needs. Similar efforts in the Netherlands involved the calculation of the average time needed for processing different types of cases based on time-log assessments. This reportedly resulted in changes in staff allocations, providing for more fairness and flexibility in the distribution of resources among different courts (Tragter-Schubert 2000). The gaps in the Dutch methodology, however, are comparable to those in Germany. The experience of both countries provides important lessons, including the following: 4  Even when the main laws or agency rules regulating the way cases are handled are the same throughout a country, case variations and local conditions vary so greatly that actual staffing needs can be determined only on the local level. A country-wide approach to measurement can and should be applied, but the results need to be ―fine tuned,‖ based on local data and experiences.  Even the most sophisticated case- and workload assessments require constant adjustment over time and can provide staffing guidelines and measures for justifying staffing needs only at the time the measures are taken. They contribute to but are not the only data source for establishing standard case disposition rate guidelines,4 and they can be only one of several indicators used for evaluating individual judge or prosecutor performance. It is not surprising that the ―science‖ of weighted caseload and workload assessments is relatively new to the civil law countries in continental Europe. It is only with the start of the 21st century that some European countries, especially the Netherlands, Denmark, Finland, and possibly Spain, focused on introducing modern case management approaches and thereby started looking at workload allocations in a different way. How new this approach still is in many parts of Europe was highlighted by a January 2011 article in the Utrecht Law Review, which stated that ―it is only recently that caseloads have again become a major subject of research [in Europe],‖ citing efforts by the Belgian Ministry of Justice to commission a series of caseload surveys and Switzerland‘s several caseload studies as examples of different initiatives towards good court management (Lienhard and Kettinger 2011). Recognizing the value of a solid approach to establishing staffing needs, weighted workload studies were conducted also in some evolving democracies as part of donor-funded justice reform programs. Such work was, for example, completed in Mongolia in 2003 and in Kosovo in 2009 (see NCSC 2003a, 2009). The following sections outline in more detail different staff estimation methodologies used in the United States and several other countries, including their benefits and limitations. It is notable that the more advanced methods can and have been used successfully for estimating position needs for lawyers and various support staff in courts and other justice sector agencies operating on a range of jurisdictional levels and in a variety of legal systems. The underlying principles apply across the systems, but their application requires adequate tailoring to the local conditions. 2. Methodologies Used for Developing Staffing Estimates As discussed above, identifying staff resource needs requires information on caseloads handled, specifically, the number and type of cases processed by the number of judges, 4 As mentioned, in some countries, judges or prosecutors are required to handle a set number of cases. This approach completely neglects the reality that case processing times differ greatly depending on complexity and other issues. It also has the negative effect of forcing staff to focus on simpler cases first to ensure that they are reaching the required set number, contributing to greater delays in the more complex cases. 5 prosecutors, public defenders, and other justice system staff in a particular location. Also needed, however, are data about the full workloads, namely, case-related work plus other administrative work, time spent on travel, training, and community outreach, and so forth, in relation to work time available (that is, work hours minus average vacation and sick time). Similarly, given the varying amounts of time required for handling different kinds of cases, measuring staff requirements according to the number of cases or offenders handled is not an accurate way of identifying the amount of work that has to be done by judges, prosecutors, and other lawyers and staff. Measures need to be available to estimate the different time requirements for the different case types. Even as new approaches were being developed in the United States, many U.S. offices continued to use less reliable methods largely because they were easier and less costly to carry out. In 1999, the Texas Office of Court Administration, in an effort to develop good methods for estimating staffing needs, undertook a survey of the most frequently used methods in the United States for determining the number of judges required. The study showed that just 27 of the 50 U.S. states had undertaken a more comprehensive weighted caseload or similar statistical analysis. The use of straightforward case activity data, population statistics, and even simply surveys of other states and political considerations was standard in the remaining states (See Texas Judicial Council 1999). This survey also inquired about the factors used in various states to assess the need for new trial judges. Table 1 displays the factors most frequently cited by survey respondents. Table 1. Common Factors Used for Estimating Staffing Needs in U.S. Courts in 1999 Factors Used for Estimating Staff Needs Number of States Number of Cases Filed 41 Number of Cases Filed Per Judge 41 Caseload Growth 39 Population Size 31 Number of Active Pending Cases Per Judge 30 Number of Active Pending Cases 29 Number of Cases Disposed 29 Number of Cases Disposed Per Judge 27 Case Types 27 Population Growth 26 Judges’ Travel Time 24 Weighted Caseload Indicators 23 Number of Jury Trials Per Judge 20 Source: Texas Administrative Office of the Courts (1999). The recognition of the value of using more appropriate measures for estimating staffing needs continues to grow. Today, weighted workload studies are much more prevalent. 6 The workload and resource assessment webpage of the National Center for State Courts shows that in 2012, such studies had been conducted in 37 states.5 Several of the simpler indicators listed above have also been used elsewhere and have often drawn significant criticism. For example, a report issued in the United Kingdom in 1998 criticized the use of prior caseloads because it did not address current needs and did little to reward efficiency or effectiveness. Courts that require fewer resources because they operate more efficiently may be penalized by using prior-year caseload and budget data instead of trend information. Caseload data that do not count all the work required were criticized, as well as methods that consider only time spent and not costs associated with the work. Population figures were also judged to be insufficient criteria for staff resource estimates; while an increasing population may affect the work of the justice system, it is demographic trends and related litigation and crime rates that are generally more relevant. The report further recognized that time assessments should be adjusted regularly to account for variations in time requirements due to differences in case complexity or staffing expertise (Lord Chancellor‘s Department 2000). Not mentioned in that report are another set of issues, such as the way in which legislative changes that introduce new case types or processing steps can also significantly impact justice sector workloads. These changes have to be reflected either in forecast models or in revisions to the data collection procedures (for more information about forecasting approaches, see below). What are the best means of determining a reasonable workload for justice sector employees? There is no solid consensus on this question. The dominant approaches used in North America, Australia, and several European countries are the analytical methods (normative and regression analysis and computer modeling), the Delphi method, and the weighted caseload method. Considering that each has its benefits and limitations (see below), using a mixed approach is often advisable. 2.1 Analytical Methods: Normative Analysis, Regression Analysis, Computer Modeling Normative Analysis. Courts have explored the relationships between the various data elements mentioned above (within single courts, in multiple courts in the same system, and across jurisdictional lines) through techniques such as normative analysis, which compares a court to a ―model‖ court. At its best this approach assumes that the model court is appropriately managed, equipped, staffed, and resourced, and also applies ―best practices,‖ assumptions that may or may not be correct. Also problematic is that the two courts may not be comparable, since their case mix may differ, there may be resource variations within the courts (such as different levels of support staff, IT) and outside the courts (prosecutors, private lawyers), and local legal cultures may also impact case processing. While looking at ―best practice courts‖ for lessons learned and benchmarking is appropriate, it is not sufficient for estimating staffing needs, due primarily to the variations in court environments across a country. 5 See National Center for State Courts, Workload and Research Assessments, at http://www.ncsc.org/topics/court-management/workload-and-resource-assessment/state-links.aspx. 7 Regression Analysis. Another analytical approach uses regression analysis, especially to forecast staffing needs based on the relationship between a certain business factor (such as caseload) and workforce size. Using statistical software, the analyst first creates a scatter diagram depicting the relationship between the business factor and workforce size, such as the number of students and the corresponding number of teachers needed. The software can then calculate a regression line, which cuts through the center of the points on the scatter diagram, assuming that the line then constitutes a standard to which courts should be compared. By inspecting values along the regression line, one can see how many employees are needed at each value of the business factor (Reference for Business 2011). Figure 1. Sample Diagram Workforce/Business Factor Relation 60 50 40 30 20 10 0 0 100 200 300 400 500 600 Source: Reference for Business (2011). This method is not without its flaws, however. If there were, for example, a direct relation between the number of cases filed and the number of court staff needed to handle them, such a calculation would work, but that is not the case. The problem is that court business factors are generally quite complex, while statistical methods for demand forecasting assume that the relationship between workforce size and the business factor remains constant over time or is at least predictable. If this relationship were to change unexpectedly, which is often the case for courts, the forecast becomes inaccurate. Such projections can provide general trend information related to overall caseload, for example, but not more. Computer Modeling. Some jurisdictions tested computer modeling, using sets of algorithmic formulas to predict probable outcomes under given circumstances. These formulas may or may not be repeated in real life, however. They are also based on various assumptions that may or may not be valid, and are applied to such data as are available—and whose validity and accuracy are always subject to question. Even in combination, these methods have led to weak measures for a court competing with other agencies‘ demands for limited resources, especially in the face of legislative, executive, 8 and popular indifference—or even disbelief—regarding the court‘s needs, requests, and justifications for staff increases (Caylor 2000). This is not to say that computer models cannot assist in predicting future staffing needs when legislative or process changes are proposed or in estimating the impact of staffing changes if solid information about current workloads exists (Flango and Ostrom 1996). IT solutions can often greatly assist in accessing and compiling the data necessary for estimating staffing needs. Courts with well-functioning case management software will have more precise information on case events and case types, and on factors that may make the handling of a case more complex and therefore more time consuming (such as multiple parties, interpretation needs, and so on). Advanced software will also be able to track a good portion of the time support staff and even judges are spending at their workstations and on the bench. Still, as outlined further below, this is only a fraction of the time judges and prosecutors especially are spending on case preparation, interviewing parties, and other core activities. 2.2 Delphi Method The Delphi research method is a technique for arriving at ―true estimates‖ by sampling expert opinion. This technique has been used frequently as a source of external validation of weighted caseload studies, and less often as a stand-alone process to determine the workload of the judiciary and other justice sector agencies (NCSC 2005). Still, Delphi studies are probably used far more frequently than can be documented since they are easy to conduct. The benefit of the Delphi method is its cost effectiveness and the relatively short time required to develop staffing estimates. Its shortcoming is that it is based on the assumptions and views of a limited number of ―experts‖ who can rarely represent the universe of jurisdictions and work situations that need to be reflected. While experienced judges, court administrators, and prosecutors may be able to estimate quite accurately how long it may take them and possibly their staff to process certain types of cases by major process step, few can accurately forecast the processing times required for all types of cases in all the various types of court levels throughout the entire jurisdiction, still less the entire country. While this method has significant weaknesses, some situations require its application. For example, in 1988, following an Arkansas Supreme Court decision invalidating the state‘s existing court of juvenile jurisdiction as unconstitutional, Arkansas had to establish an entirely new statewide juvenile court. Because the new system had to differ from the prior one, the existing juvenile court case records could only provide some information but could not be used to accurately determine the staff requirements for future events that would be needed to initiate and dispose of juvenile cases. As a result, the estimation process could depend only on ―informed guesses‖ using the Delphi technique. Judges and their support staff were asked to identify the events required for case initiation, adjudication, disposition, and review, and estimate the time required to accomplish each event. Case weights from these estimates were then 9 calculated based on the prior caseload to determine the number of judges that would be needed to staff the statewide juvenile court. Using this procedure it was estimated that it would require 180 minutes of judicial time to handle a typical juvenile case from initiation through closure. Using a six-hour case hearing day and a 220-day work year, Arkansas concluded that a judge could hear an average of 440 cases per year. The state used the results of the study to staff its new courts of juvenile jurisdiction and, from all accounts, participants have been quite satisfied with the results (Hurst 1999). The creation of administrative courts in the newly emerging democracy of Mongolia presented a similar situation. Since the operations in these courts were new, there were no measures for appropriately estimating their staffing needs. The Delphi method, as inadequate as it is, was the only way to arrive at somewhat reasonable estimates. Another difficulty was that no case data existed to predict the number and types of cases that would be filed in these courts and how the filing trends would develop. Using information from similar civil cases for which a weighted caseload study had been completed was the only way to develop initial staffing estimates.6 Such an approach was not perfect but still provided the best possible estimates. There can also be another useful application for a Delphi study. Since weighted caseload studies capture the time judges in different locations spend on different case types, the resulting information will likely show differences in the time expended. Although this is important information for exploring why such differences occur (that is, whether it is due to case mix, resource differences, etc.) it does not provide an answer as to which time requirements are reasonable and sufficient to deliver quality. This is when a Delphi study can be added to provide a reasonable and justifiable answer, an approach that was used, for example, in Florida in 2000 (see Ostrom et al. 2000). 2.3 The Weighted Caseload Method and the Development of Workload Models A stronger technique for estimating the need for judges and other staff is a weighted caseload study, which can be applied using two approaches. The least-favored option involves the use of court caseload data in combination with the Delphi method to build case weights based on the expert opinion of judges, attorneys, clerks, litigants, or other court system stakeholders as to the length, frequency, variability, and probability of particular processing events in particular case types. Although usable information can be obtained by this method, it is not self-validating, and it may be unpersuasive to funding authorities, since the resulting analysis is based on the opinions of interested parties. The more defensible approach is to actually measure those same event attributes (length and frequency of processing activities, completed by case type) by conducting an empirically derived, weighted caseload study. The strength of this approach (when done correctly) is that the results obtained lead to a reasonably accurate and comprehensive picture of how long it would take a group of judges, prosecutors, or other justice systems staff to process a given body of cases and tasks. 6 For infomation on the weighted caseload and workload study conducted in Mongolia, see NCSC (2003a). 10 This method measures how much staff time is spent at a given court, prosecutor‘s office, or other justice sector agency to process cases from intake to disposition. Data are collected on the number of cases being filed by type, as opposed to simply the total number of cases filed, in an effort to control for the difference in processing time for the various classes of cases (for example, murder, traffic cases, disputes over pasture rights, divorce). Because aggregate case counts cannot provide information on the amount of time it takes to dispose of different case types, they are not a very good basis for determining staffing requirements. The weighted caseload approach shifts the emphasis away from caseloads to a more appropriate measure of workloads. A weighted caseload study addresses two key issues that are essential when assessing staffing needs: 1) How much judge, prosecutor, or support staff time, on average, is required to handle each type of case to disposition, and 2) The amount of time available to a typical judge, prosecutor, or other staff member to handle cases. Simply stated, the number of judges and other staff needed is determined by dividing the amount of staff time required to dispose of all cases by the total amount of time judges and others are actually working (that is, regular working hours per year minus vacation and sick time). Weighted caseload studies are particularly appropriate for geographic areas where court jurisdictions vary significantly in size, population, or case mix, since they provide an objective means of measuring the relative need for judges and court support staff while allowing for the differences. Accounting for these variations is critical in assessing how many judges and other justice positions are needed in different locations. For example, the amount of time available to handle cases may be substantially different in urban and rural jurisdictions. Higher caseloads in urban areas can often lead to increased expertise and more efficient processing in those locations; rural jurisdiction judges or prosecutors might spend more time each day on administrative work and traveling to hearing locations, making the nature of their workload different from that of their urban colleagues. Limitations of the Weighted Caseload Method Despite the apparent value of the weighted caseload approach, some limitations do need to be considered. First, data collection for these studies is burdensome and expensive. Substantial information must be gathered from the courts, prosecutors‘ offices, and other justice system agencies involved on the steps and time required to handle the key events that are part of processing particular case types. All staff need to keep time logs, which must be monitored on a regular basis to ensure that correct information has been entered. Other criticism of the weighted caseload approach reflects more the type of data included rather than the methodology used. For example, the assessment system in the United 11 Kingdom was changed in 2000 after it was argued that in measuring case volume per year, the number of cases filed should be counted rather than cases completed, since this reflected a truer indicator of work. In jurisdictions with serious backlogs, this is a valid argument. Moreover, the weighting of cases was deemed to be insufficient, as it represented the time required for processing cases, rather than the cost of the work involved (Lord Chancellor‘s Department 2000). These arguments indicate that the weighted caseload model as it was applied in the UK example mentioned above still gave inadequate consideration to the actual case complexity mix. As discussed earlier, case complexity is driven not just by case type but often by other aspects of the case, such as the need for expert opinions, or the number, type, and location of offenders, victims, and witnesses. When cross-border issues are involved, for example, time and cost increase dramatically. These are generally not measures that even standard weighted caseload studies capture. Advanced caseload studies will include separate complexity indicators, but this still points to the need for much more detailed case type and workload assessments (and possibly information about other cost elements), which might not be needed for every jurisdiction. Although the results of good weighted caseload studies provide a standard measure for staffing needs across jurisdictions, this clearly requires adjustment if case complexity or other workload measures are higher or lower in an individual location. Keeping the weights current is another critical consideration for ensuring that the information gained through weighted caseload measures leads to credible results over time. In the United Kingdom, critics pointed out that the same weighting was used for eight years, despite significant changes in legislation, policy, procedure, or the nature of the workload for 97.8 percent of cases (Lord Chancellor‘s Department 2000). Unfortunately, updating the weights can be both time consuming and expensive; not updating periodically, however, can potentially undo all the hard work and resources dedicated to creating the weighted caseload system in the first place. In the end, despite the shortcomings, weighted caseload studies are generally considered to be the most appropriate method for determining staffing needs in the judicial sector (Texas Judicial Council 1999). To be even more precise, however, additional measures need to be added to better reflect actual workload in individual locations. Weighted Caseload Studies and More Accurate Staff Allocation As outlined above, when the work of courts, prosecutors‘ offices, and other justice sector agencies is described just by traditional measures of work volume (that is, number of cases), the different effort involved in processing different case types cannot be captured. These measures plainly do not provide reliable information for estimating the staff resources needed and are thus likely to provide a highly distorted picture of staff resource allocation, possibly leading to significant inefficiencies throughout the entire judicial process. 12 The following example, using actual data from a U.S. prosecutor‘s office,7 highlights this point: Table 2. Cases Disposed by Offense Type and Prosecutor Time Offense type Number Average Case and Workload Percent Distribution of Cases hours spent per case Case Workload Percent Distribution hours Caseload Case Workload Felonies 4,870 2.7 13,317.4 25.4 82.3 Violent 784 4.3 3,285.2 4 20.3 Property 2,852 2.0 5,704 14.9 35.3 Drugs 494 2.3 1,136.2 2.6 7 Other 760 4.2 3,192 4 19.7 Misdemeanors 14,274 0.2 2,854.8 74.6 17.7 All cases 19,144 0.9 16,172.2 100 100 Source: Jacoby (1987). Looking at the results in table 2, it becomes obvious that examining caseloads only can be misleading for estimating needed staff. If the total number of cases were the only indicator of required staff time, one would assume that each case took 0.9 hours per prosecutor. Yet, the data show the significant variation of prosecutor time needed for different case types. Across an entire country, this diversity is less important for estimating how many positions are needed nationally, since the generally smaller number of complex cases make up a smaller portion of the total at the national level; in a local jurisdiction, however, the greater the diversity of cases the greater its impact on staff requirements. These data also suggest that over 80 percent of staff time (and positions) should be focused on felony prosecutions and less than 20 percent on misdemeanor cases. If caseload alone had been considered, 75 percent of the staff would have been assigned to prosecuting misdemeanor cases (Jacoby 1987). As shown in table 2, the difference between these two factors (number of cases vs. case workload) is significant when it comes to staff allocation and is a result of differences in their meaning. Using simply the caseload as the primary measure for estimating staffing needs assumes that all cases require the same effort; using case workload as the primary criterion recognizes the variations in staff effort required for handling different types of cases. 2.4 Case Weighting Studies and Process Improvements Modern case management approaches stress the need to identify how long it takes certain types of cases to progress through the system, where bottlenecks are, and where processes may be streamlined. Case weighting studies can assist in detecting and explaining some but not all process inefficiencies, and it is essential to understand the 7 This office was staffed with 20 prosecutors and the chief prosecutor. 13 difference between the time it actually takes a prosecutor or a judge to conduct all the work required to dispose of a case (that is, from filing to final court decision) and the overall time that passes between case intake and disposition. The time that prosecutors, judges, and other justice system staff each spend on a case is only a fraction of the total time the case requires to be moved through the system. That total time is a combination of time spent by the many individuals who work on a case and the considerable amount of time during which the case is idle. Some of this idle time is due to scheduling requirements, as the parties, lawyers, prosecutors, and the court need to have some time in between hearings to prepare. Other idle time may be due to the difficulty in finding a hearing date on which the judge and other parties can be available, a lack of courtroom space, or other issues that point to system inefficiencies, such as the frequent postponement of hearing dates because parties are not prepared, witnesses do not appear, and so on. The time staff spends on each case step will increase if the processes are not well managed. For example, if hearings are frequently postponed, staff will spend more time on this particular process step, since they need to reschedule and rearrange their work accordingly. These are the areas where effective staff allocation and efficient procedures interconnect. The unavailability of staff will delay the process, and inefficient procedures will absorb additional staff time—which in turn leads to increased need for staff. The difference between staff time captured by weighted caseload studies and overall case processing time is illustrated in table 3 below. An actual example from a U.S. prosecutor‘s office, involving a felony case, is used to demonstrate the process and time calculations involved. Table 3. Estimating Staff Time vs. Case Processing Time in Sample Criminal Proceedings in a Typical U.S. Court Work Step Total Time Calendar Time* Prosecutor Work between Work Time** Steps Filing to preliminary hearing 6 days 5 day 5 hours 3 hours Preliminary hearing to Grand 26 days 15 days 7.5 hours 0.5 hours Jury Grand Jury to arraignment 8 days 7 days 7 hours 1 hour Arraignment to motions 31 days 30 days 2 hours 6 hours Motions to jury trial 51 days 45 days 3 hours 45 hours Disposition to sentence*** 31 days 30 days 6 hours 2 hours Totals 153 days 145 days 0.5 hours 62 hours * Calendar time is the time between processing steps minus the time the prosecutor is active. ** Prosecutor time is the actual amount of time a prosecutor spends on completing the specific work step. *** The U.S. system bifurcates criminal court hearings to first establish guilt and in a second hearing determine the appropriate sentence. Source: Jacoby (1987). Table 3 shows that 62 hours of prosecutor time would be needed to process this sample case type. The remaining 145 days and 0.5 hours between case filing and disposition may 14 involve actions required by administrative staff, judges, and defense attorneys, but mainly reflect scheduling time frames from one step to the next, essentially time to allow the court and parties to be ready for the next step. The longer the ―empty‖ scheduling time in relation to prosecutor and other staff time needed, the less efficient the process. The above example demonstrates the importance of identifying both the time spent by justice sector staff and the time it actually takes to complete each step when estimating staff resource requirements, allocating staff to process steps, and identifying where process adjustments and staffing changes could speed up the disposition of cases. The interconnectivity of these elements also shows that the first response to a weighted caseload study that indicates insufficient availability of staff should not be ―more staff is needed,‖ a solution that will be difficult to address in many cash-strapped jurisdictions, but rather, ―how can processes be made more efficient to free up staff time.‖ 2.5 Work Not Directly Case-Related: Moving to Full Workload Assessments Understanding the impact of different case workload on staff allocation and process efficiency is essential, but this still does not consider another important element: the fact that many tasks performed by prosecutors, judges, and other justice system personnel are not directly related to a case. Different from case-weighting studies, true workload assessments capture the variety and complexity of the work done by judges, prosecutors, and other justice system staff. This includes the above-mentioned difference in case complexity that can be independent of case type and its impact on time and effort required; it also means that non-case-related work is captured. Many practitioners in the justice system still tend to think of their work only in terms of the cases they handle or clients they are assigned to represent. Yet, this does not reflect their full workload. A certain amount of non-case-related administrative work, preparatory tasks that may or may not lead to a court case (for example, prosecutorial advice to the police department), a court‘s screening of a case filed that will not go forward due to incomplete information provided, other activities that are not related to individual cases (such as general interdepartmental communications about processing approaches, policies, and so on), or time spent away from the office on non- case-related business (for example, in training sessions) have always been part of justice sector work. Similarly, handling general citizens‘ complaints, reaching out to the public, and working with victims are only some of the activities that prosecutors and judges attend to in addition to handling various types of non-case-specific administrative tasks, managing their staff and operations, or being involved in training efforts. None of this would be captured in agency statistics and is frequently overlooked in weighted caseload studies. With increasing public demands for responsive services, and the recognition that alternative approaches—such as alternative dispute resolution (ADR) and conflict resolution, community justice, diversion to alternative non-court responses and a strong prevention focus—can have a more profound long-term impact on reducing caseload, societal conflicts, and crime, the proportion of work not related to individual 15 cases increases. For example, in U.S. jurisdictions that stress community-oriented work, the percentage of time that police, prosecutors, judges, and corrections officers spend in community meetings and on organizing volunteer groups adds considerably to the time spent on their core responsibilities, i.e. handling cases. The more comprehensive workload assessments provide information about workload for different case types plus accnts for work that is not related to individual cases. How this is achieved will be outlined below. At the same time, it has to be understood that even the most credible resource assessment technique, including the weighted workload assessment model, cannot determine the exact number of judges required to manage caseloads. As stated by Margaret Guidero and Suzanne Tallarico, two well-known and experienced U.S. court consultants, ―No single quantitative resource assessment model can do that. A workload model should be used in concert with the budget, population trends, and court-specific factors that affect judicial resources‖ (Guidero and Tallarico 2011). 3. Weighted Workload Studies: What is Measured and How? The importance of measuring caseload vs. workload leads to the question of how the complex work of judges and prosecutors can be translated into measurable elements. As outlined above, well-tested methods for such studies have been applied in many countries, mainly for courts (Jacoby 1987; Flango and Ostrom 1996; Deutscher Richterbund 1999, Tragter-Schubert 2000, NCSC 2009) and continue to evolve (NCSC 2010). However, within courts, they have been applied to judges and other staff, and similar studies have also been applied in other justice sector agencies, especially prosecutors‘ and public defenders‘ offices (NCSC 2003a, 2007; APRI 2002). In the United States, such studies have repeatedly been conducted for courts, prosecutors, and public defenders at the same time to ensure that staffing estimates are based on same-year case data. While the work of justice system employees in different agencies requires some adjustments in the data elements that need to be collected (mainly the work steps involved), the overall methodology remains the same. In order to measure the time and effort required, distinguishable work steps to dispose of various types of cases are identified, and the time for processing different case types at each required step is measured, along with the number of cases and available staff time. The following sections outline how weighted workload studies are developed and implemented. Step 1: Identifying the scope of the study Different jurisdictions handle a broad range of case types, and the mix of cases and their complexity influence staffing needs as much as—if not more than—simply case volume. 16 Ideally, detailed information is available on all case types handled. Since weighted workload assessments are time and resource consuming, reducing the effort required by combining cases into a few major categories or even focusing just on the most predominate case types handled is often the only feasible approach, particularly if resources are scarce. For example, if the majority of cases handled in a court are civil cases, developing well-based estimates for staffing resources for the civil courts may be preferable to relying only on less-valid methods, such as the Delphi method, for all cases. A review of case-type data allows for the identification of the major types of cases processed in order to determine which case types can be reasonably merged into major categories—or if a focus on the majority of cases is sufficient. Even in highly automated courts that electronically track detailed information about the time it takes for different case types to progress from one step to the next, the inclusion of all case types and some sampling may still be needed, since important complexity information is often not tracked and staff time collection will still need to be conducted separately. For some support staff positions, the IT system may actually provide the time effort required by various persons at each step, but additional time collection is generally inevitable. The study may include only the work at a certain court or agency level (such as, first instance court, appeals court). It also may need to be confined to representative sample locations that include urban and rural jurisdictions in all relevant regions of a country, include general jurisdiction courts only, or involve special jurisdiction courts (for example, commercial, juvenile). These issues need to be determined beforehand, together with the time period to which the study should be applied. To develop reliable estimates of staffing needs, at minimum, case volume data by case type for one year prior to the start of the study need to be available. Even better, access to such data for at least three years is useful to get a real sense of the changes in case volume and will be essential if any forecasting is envisioned. In addition, information about the noncase workload (that is, work not directly related to the case) has to be collected. As mentioned above, what this comprises differs across jurisdictions and positions and will fluctuate over time. For example, if a country goes through major changes in its legislative framework, judges and others will likely spend a greater amount of their time in training and other educational events. This would have to be considered either when case assignments are made or possibly in staff allocation over the succeeding few years. Step 2: Defining the case count It may appear easy to define what constitutes a case for the purpose of a workload study and how to count these cases, but in practice, courts and other justice agencies often apply various practices for recording cases. The primary issue is: how does one count cases with multiple offenders or multiple charges? In the United States in 1989, the National Center for State Courts and the Conference of State Court Administrators instructed court administrators to count each defendant and all charges involved in a 17 single incident as a single case (NCSC and COSCA 1989). In developing its standards for the staffing needs of defenders‘ offices, the National Advisory Commission on Criminal Justice Standards and Goals in 1973 defined a case as ―a single charge or set of charges concerning a defendant (or other client) in one court in one proceeding‖ (Standard 13.12., see National Legal Aid and Defender Association 1995). The former approach is sufficient and often more feasible but both are valid options, though it is important that this decision is made before data are collected and that the chosen option is applied throughout the study. For any assessment that involves more than one court or other justice system agency, it is critical to count cases using a uniform definition, something that may add an additional challenge to the study if the underlying agency systems count cases differently. Especially for criminal cases but also for other case types, it is essential to count cases by offender or litigant, independently of how court and other justice agency statistics are generally collected. The defendant is the one who is tried, convicted, or otherwise processed, and the litigant the one to whom the court responds. Cases with codefendants and multiple litigants should be counted separately because each individual may require different staff time, and dispositions may occur separately for each. Other questions arise when an offender has multiple charges or counts. On the one hand, multiple charges resulting from one offense should be counted as one case identified by the most serious charge, since the work required is only for one actual case. Yet, how multiple case counts are handled should, in the end, depend on the prosecution strategy or how the court handles the case, since this determines if multiple actions are generally necessary, and the time needed to complete them has to be captured. If the multiple charges are handled as one, they should be counted as one. If additional time is required for handling multiple case counts because they result in different court actions, they should be counted separately. For example, if someone breaks into a house, steals valuables, and sets fire to the premises, all three counts are likely, resulting in one case under the most serious charge. If someone is charged with five counts of public drunkenness over a one-month period, these multiple cases may be handled by the court as one process and should therefore counted as one offense. If the charges are handled separately and multiple court hearings are required, they should be counted as separate cases. This distinction is essential for completing the time studies. Ideally, this case definition reflects how the court or other justice sector agency collects case statistics, although experience indicates that this is not the case in many countries. As a result, before a case weighting study is started, it is crucial to identify how cases are recorded and reflected in the agency‘s statistical data collection and that the approach chosen for counting what constitutes one case can be matched to the information available on case volume. Step 3: Identifying processing steps for cases To determine work steps related to cases and noncase work, a flowchart identifying each of the process steps where work occurs has to be developed. This flowchart then has to be 18 linked to the volume of cases that progresses from one step to the next. The information needed to depict each processing step is:  The first event that can be related to a specific case entering the process  The number of cases moving into the process  The number of cases moving from one step to the next  The number and type of dispositions at each process step conducted by various court or agency staff The process steps need to be identified by actual decision steps that are performed by judges (and other staff), not just what is indicated in the law. Figure 2 outlines a hypothetical case flow for prosecuting a criminal case in a standard common law system. 19 Figure 2. Example Criminal Case Processing Flow Chart for Prosecution Review of Screening Filing in 1st Peparation and trial investigation for filing instance court 3,140 4,190 3,950 3,680 Investigation Cases dismissed Cases dismissed denied – 240 270 by court 540 Prison Not guilty sentence finding 1,510 Appeal Appeal 400 Fine: 200 denied Prison: 500 100+450 Fine 1,230 Prison review Appeal 1,010 + 450 granted 100+ 50 Source: Adjusted from Jacoby (1987). Step 4: Compiling case data 1,010 Once the major procedural steps are identified, the volume of cases by type that are processed from one step to the next has to be determined. Annual case data for at least one full year—preferably the year prior to the start of the study—are needed to represent a relatively reliable picture of the cases handled (Jacoby 1987). Three years (or more) of data are helpful to understand case trends over time and to project staffing needs into the future. Naturally, not all cases progress through to the very last step, as some cases are dismissed, withdrawn by the parties, or otherwise settled. To adequately assess staff requirements, it is important to understand this change in volume to calculate workloads, though this can be particularly difficult since it requires access to good agency and court statistics by case types and by major process steps. While the former may be available, the latter often are not. The difficulty frequently lies in having access to sufficiently reliable case disposition data for each identified processing step for each major case type, and these data need to be available in a comparable manner across different courts. The more decentralized the court and prosecutor systems are, the higher the likelihood that detailed case data are available only in individual jurisdictions and that the data will differ in definition and detail across locations. As a result, the needed data may not be readily available and may need to be newly calculated from the agency‘s databases or even through case file reviews, which greatly increases the time and effort required. 20 As mentioned above, the cases should be grouped into common core case categories beyond misdemeanor and felonies. Reporting categories commonly used include violent crimes, property crimes, drugs, other felonies, misdemeanors, small value and high value civil cases, family cases, land issues, and commercial cases as defined by the relevant codes. Cases that are handled less frequently (for example, murder or disputes over patent rights) should be included in the major categories, unless they pose particular staffing issues that need to be captured. If they are included, the higher staff time requirements should be dealt with through separate complexity indicators. Complexity indicators are methods of collecting information about certain case elements that may add to staff time, such as the involvement of children as witnesses, multiple offender cases, cross-border cases, or cases that involve individuals who require an interpreter. The collection of such information ensures that the additional time requirements are adequately reflected, since they can explain potential time variation between different courts despite seemingly similar caseloads. A flow chart for each of the selected case categories has to be developed to distinguish variations in steps and time needed. Step 5: Preparations for and collection of staff time In order to conduct the workload assessment, a time log has to be developed and tested that reflects the relevant processing steps, case types, and complexity indicators.8 At that point, the time study period has to be agreed upon. The time study has to cover at least a representative time frame for fully capturing the major case types and steps and all general types of non-case-specific work (if there are certain times of the year set aside for training or other judicial responsibilities, such as election supervision, such time spent may not be captured via time logs but as separate time estimates). This generally requires at least six weeks of data collection for first instance courts. The earlier referenced study conducted in Germany in 1999 involved data collection for at least three months in lower-level courts (Amtsgericht) and five–six months from higher-level courts (Landgericht) to assure that all types of cases processed through all case steps were captured without giving undue weight to rare events. Whether to include or avoid special time periods during a year that influence case filing and staffing levels also must be considered. These can include primary vacation times or seasonal variations in case filing or crime rates (such as variances due to holidays, harsh winter conditions, or high population concentration during special events or festival seasons). If the data collection cannot involve every location, the participating courts and prosecutors‘ offices need to be carefully selected to ensure representation of urban and rural jurisdictions and other local variations and specializations that influence workloads. While it is important that the data collection be anonymous (or timekeeping can be 8 For examples of such time logs developed for courts see for example NCSC 2010. 21 compromised), the verification and completion of entries must be assured. This means that the log sheet has to include initial identification that will be deleted after entry verification and that a reliable staff member must collect and see that the forms are complete. Before the time study begins and after the data collection instruments have been tested, all participating judges and staff members have to be trained in the proper completion of the time logs. Further, to ensure the accuracy and proper implementation of the study, a study committee or task force composed of representatives from each relevant entity should be created to provide input into the development of data collection instruments, to identify data and possible collection issues, and to later assist in the analysis and verification of the results. In the United States, the collection of staff time is frequently assisted by online data collection instruments. To ensure accuracy, participants are generally asked to record, on paper, the time study information throughout the day, preferably as they complete each activity. They are then asked to transfer this information to the online form, a task that can be completed for them by an assistant, if available. Those who are participating in these studies also then have access to an online help desk to ask questions related to the data collection. The web-based data entry form also allows for the data to be immediately downloaded, compiled, and reviewed by analysts, which provides for another data quality check and significantly reduces time for data entry and analysis (NCSC 2010). Step 6: Calculating case weights Staff effort is the time judges and others spend on each processing step. For each major case category, the sum of time spent by all judges or prosecutors in each major case step is calculated. The total number of minutes spent is then divided by the total number of cases recorded for each step, providing the average time spent for each major case step in each major case category. The average time spent is then multiplied by the number of cases processed in each step, resulting in an average workload by case type. Table 4 provides an example for developing case weights (such as the average amount of time required to process a case from beginning to end) for prosecutors. 22 Table 4. Annual Workload by Case Type and Process Step Case Type Process Step No. Cases Avg. Time in Workload in Processed Hours Hours Violent Felony Review of 4,190 0.2 838.0 investigation Screening for filing 3,950 0.5 1,975 st Filing in 1 3,680 0.2 736 instance court Preparation and 3,140 2 6,280 trial Appeal 700 2 1,400 Prison review 1,500 0.2 300 Misdemeanor Review of 34,856 0.1 3,485 investigation Screening for filing 34,110 0.2 6,822 st Filing in 1 32,900 0.1 3,290 instance court Preparation and 32,850 0.5 16,425 trial Appeal 15,350 0.5 7,675 Total weighted hours 49,226 Source: Adjusted from Jacoby (1987). Step 7: Calculating staff numbers In order to identify how many judges and other staff are needed, the annual workload expectation for each case type has to be calculated. This is created by multiplying the case weight by the number of each case type filed (or expected to be filed) in a year. The result is the total number of minutes or hours expected to be taken up by each individual case type. The number of attorneys and other staff needed to process cases is then calculated by dividing the weighted total hours by the actual work hours available for all judges, prosecutors, and other staff. Actual work hours available are the average hours per year worked, considering holidays, vacation time, and sick days. In the United States, this is usually equal to about 220 work days per year at eight hours per day per full-time judge, prosecutor, or other staff member, or a total of 1,760 hours per one full-time employee, or FTE. Next, the total number of weighted hours has to be divided by work hours available per FTE. Using the U.S. example shown in table 4, this means a total of 49,226 workload hours divided by 1,760 hours available per person, which equals 28, meaning that 28 full- time prosecutors are needed to handle the total case-related workload. If applied to misdemeanor and felony cases separately, the calculation would show that 6.5 FTEs are needed to handle misdemeanor cases and 21.5 for felonies. 23 In addition, the non-case-related workload has to be added and included in the required FTE calculation. 4. Reflecting Workload Variations by Profession The workload assessment process described above has been used in the United States, Canada, and other countries to assess staffing levels for courts, prosecutors‘ offices, public defender agencies, and probation departments. In order to develop an appropriate data collection mechanism that is specific to each agency and profession, it is important to recognize that even when these professions participate in the same processing step, such as the trial, for example, the work required by the judge, prosecutor, defense attorney, or court clerk varies not only by role and substance but also by the time expended. While the work of judges focuses predominantly on activities that occur in the courtroom (exceptions exist in cases where judges have investigative responsibilities and other significant preparatory, research, and writing tasks to fulfill), the majority of the work conducted by prosecutors, defense attorneys, and support staff occurs outside the courtroom. For example, in addition to preparing for the court hearing and participating in the trial, Defense attorneys  Meet with clients to explain the proceedings before the clients appear at their detention hearings  Investigate the circumstances of the alleged offenses  Learn about the accused persons‘ ties to their families and communities  Develop dispositional plans that may be preferable to institutional confinement  Follow up with clients during dispositional reviews  Monitor placement problems that may arise regarding needed services or conditions of confinement Prosecutors  Respond to police requests related to the inquiry and investigation  Review cases presented by police for legal sufficiency for filing in court  Meet with police, witnesses, and the defendant to hear their statements  Review evidence presented by police and possibly request additional investigative activities  Engage in jury selection, where such are used  Engage in plea negotiations, where allowed by law  Prepare appeals processes  Review detention and incarceration decisions  Conduct prison reviews The workload of a prosecutor‘s office, for example, can be significantly higher than that of the corresponding criminal court, since prosecutors handle many police matters that never result in a court case. They may also have other responsibilities, such as 24 participation in parole decisions or the review of prison operations. For their part, courts can also have additional responsibilities, such as notification and enforcement activities, that add to their core case work. As a result, case data requirements and time logs need to reflect the actual work environment of each profession while applying the same general methodology. 5. Estimating Future Staffing Needs Despite their benefits, one of the criticisms of workload studies is that they are based on current procedures and staffing arrangements, which sometimes include inefficient practices. To estimate how many staff positions are actually needed, it is argued, one should first look at how processes can be made more effective, and then estimate the workload based on improved operations. This is a valid point, and as outlined earlier, workload studies can greatly contribute to identifying inefficiencies in court and other justice sector operations—if combined with a proper operations assessment. The combined information from both studies can then be used to estimate the impact on staffing resulting from process or policy changes. Furthermore, forecasting is challenging since many factors affect future case and workloads, and all forecasts rely on information about past workloads, to which assumptions about increases and decreases are applied. The reliability of forecasting declines as the knowledge about future changes declines, and the forecasting becomes more volatile where small units are involved (Lord Chancellor‘s Department 2000). Significant changes, such as the number and type of cases investigated by police and filed by litigants, the type and complexity of cases filed, adjustments in agency operations and resources, new technology, and recent legislation, are important indicators that are difficult to predict and require that workload assumptions and established case weights be adjusted over time. Population growth or changes in demographics (for example, increasing numbers of young people), as well as a straight-line projection of past caseloads (if reliable and sufficiently detailed multiyear information is available), can be used to approximate coming changes, but they are not very reliable in the long run. Since cases result from different types of matters (criminal, civil, commercial, administrative, and so on), forecasts should not be made for overall caseloads but should instead consider changes at least by major case types. If new legislation is pending, projections need to determine its potential influence on case and workloads. Such projections frequently have to be based on expert assumption only, since no other data are available. If staffing levels in one justice sector agency change, it may be possible to project the impact on caseloads in all affected agencies. For example, increased numbers of police officers generally result in more investigations and arrests, leading to higher workloads for prosecutors, judges, and corrections staff. More difficult to project is the impact of population and societal changes on caseload, even though these may have the largest impact. Population increases are likely to impact case numbers, but the impact may not be at the same rate as the population numbers. Furthermore, as societies become more complex and less integrated, disagreements are 25 more likely to occur and less likely to be resolved informally (Krislov 1995). At the same time, public trust in the justice sector, as well as its accessibility and efficiency, will influence the extent to which the public will turn to the justice system to resolve those conflicts. For practical purposes, all jurisdictions have to rely on the somewhat limited information available to make estimates about changes in workloads over time. At a minimum, information on caseload trends as well as weighted case and workload data has to be available to allow for projections, using expert opinions and applying Delphi methods as well as computer models, to assess the impact of new legislation and future population changes. 6. Efficiency, Quality, and Other Performance Measures As mentioned, an important challenge to estimating an appropriate staffing level is that the information collected may be based on current inefficiencies in the system. The collected data also may not reflect the quality of decisions made or other less tangible court and agency goals, such as transparency of operations or public trust, which can have an effect on the time needed for case processing, decision making, and overall workload. The goal of weighting various categories of caseloads is to identify how much proportionate effort is applied to different case types. It does not, however, provide information on how much time judges and other staff should spend to come to a fair decision in a timely manner, or how much time should be spent on other agency services (Lord Chancellor‘s Department 2000). Addressing these additional performance measures, especially for more than one court, requires that workload assessments be combined with methods for establishing performance standards for the court or justice sector agency, along with related studies to measure the performance achievement. The following sections briefly outline the relationship between workload assessments and the most predominant measures used by courts and other justice sector agencies across the globe to assess, both alone and in combination with additional studies, these other factors. 6.1 Efficiency Considerations Although the time studies underlying workload assessments capture only the effort required using the existing procedures, the information gained can nevertheless identify which worksteps are currently particularly work intensive and thus provide some information on whether staff resources are being appropriately assigned. The results can also point to areas of inefficiencies. While staffing needs across courts or prosecutors‘ offices cannot usually be directly compared since the case mix and other elements may differ from one location to the next, information from courts with a similar caseload, case mix, and operating environment can indicate which court and prosecutor‘s office may be 26 operating more or less efficiently than others. In combination with workflow assessments and resource allocation data, workload studies can assist in pinpointing where and what type of additional information should be collected to identify the efficiency gaps without compromising quality or service delivery. 6.2 Public Trust and Confidence Justice agencies that have focused strongly on increasing public trust and confidence have realized that this requires not just quality and timely decisions, but also other judge and staff skills and a range of information, outreach, and engagement activities that involve time and additional resources. Research in several countries has shown that the ability of judges, prosecutors, and other justice sector staff to explain their procedures and decisions well is critical to projecting procedural fairness, another important goal. Studies in the United States, Germany, and other countries concluded that a participant‘s perception of a court proceeding‘s fairness is not limited to whether he or she wins or loses (Vidmar 1990), or, in the case of criminal trials, by the length of the sentence. Crucial also is the perception of the judge‘s fairness in conducting the trial—the extent to which the defendant felt he had an opportunity to tell his side of the story and the judge appeared to listen without prejudgment (Tyler 1984, 1997). In a study conducted in New Mexico, for example, it was the judge‘s courtesy to trial participants that shaped their perception of the fairness of the process (Greacen 1999). Still, it is difficult to establish how much more time should be reasonably spent on such communication efforts. Additional outreach and information services can be captured by time studies, but the ability of a judge or prosecutor to communicate well with parties in a case and explain decisions in a comprehensible manner is not simply a matter of more time but of skillful communication. A court‘s goal of enhancing procedural fairness may be an argument for allowing for more time, but identifying if this goal is actually being achieved—and therefore whether it justifies the additional funding—requires further study. Clearly, proper communication with parties, other court users, and the public is essential for building public trust and confidence. Yet, responding to court user needs may require additional staff to provide information or explanations of procedures, which can compound time and staff requirements. Moreover, even as courts and other agencies are asked to foster public trust and confidence and make themselves more user friendly, they are also often being forced to cut services or reduce the amount of information offered to the public due to personnel shortages (Rivera 2000). 6.3 Quality Services and Decisions One valid criticism of workload studies is that they can capture which courts and agencies process cases with more or less staff effort, but not how different staffing levels may influence the quality of decisions. Part of the problem is that the precise meaning of ―quality‖ with regard to court and other justice system agency decisions and operations is 27 debatable and can be highly subjective. It is also multidimensional and complex, as it includes the quality of the services provided by the system as a whole, by individual judges, prosecutors, and staff, and in individual cases. Moreover, ―quality‖ may be viewed in terms of such highly diverse factors as legal correctness, fairness, integrity, effectiveness in settling disputes, efforts at crime prevention versus punishment, and service-oriented, user-friendly, and independent decision making. Today there is general agreement that the earlier-mentioned procedural fairness is an important element of quality decision making and operations, but only one component (Court Review 2007– 08). Difficult to measure and define, the impact of these quality indicators on procedures is not easily established, and they need to be balanced against other important goals and performance measures such as timelines or cost effectiveness. For example, speedy disposition can compete with the need for good case preparation, solid evidence collection, and an in-depth assessment of a defendant‘s need for rehabilitation. Similarly, cost effectiveness may conflict with the need to provide access to the justice system in remote areas and service delivery in several languages. 6.4 Access to Justice and Cost Effectiveness Access to justice too has multiple meanings, such as geographic access, affordability, language and other factors that affect understanding, and equal treatment. Each of these is influenced by appropriate staffing levels or the lack thereof, and at the same time, the need to respond to these access issues influences staff time. For example, if judges need to travel to remote locations to attend hearings, this will increase the time needed to respond to such cases. Similarly, cases in which interpretation is required not only trigger the need for the additional relevant staff, but also require more time due to the interpretation process itself, which is why the need for interpretation is one of the above- mentioned complexity indicators. Cost effectiveness can only be defined by how well all the other performance measures are achieved and is directly influenced by the allocation of staff, which tends to be a significant cost factor for each agency. This demonstrates well how the different performance measures influence and need to be balanced against each other—and that staffing and budget decisions have a direct affect on the extent to which the different performance goals can be achieved. 6.5 Establishing Standards In order to attempt to assess the impact of these often competing performance goals on court and other agency operations and staffing needs, performance targets and ultimately standards for a single court or agency should be set in a particular region or country. This would help to establish common parameters for defining and measuring ―quality‖ and other similarly intangible goals across comparable jurisdictions, and it would assist in understanding how and where staff allocations need to be adjusted to ensure the standards are met. 28 Performance standards for judicial agencies usually describe requirements for operations in terms of core court and agency goals. They are linked to but distinctive from performance standards for individual judges, prosecutors, public defenders, and others that describe minimum requirements for competent conduct. Agency and staff standards are important but are just one set of data elements needed to assess staffing needs in relation to goals to achieve. Performance standards for courts and other agencies as well as for individual judges, prosecutors, and other staff have been implemented in several countries to establish basic performance requirements. For example, in the United States, standards for defense lawyers were developed that require, among other duties, that attorneys maintain contact with the client, conduct a factual investigation, examine the complaint for legal sufficiency, file appropriate motions, and conduct discovery (National Legal Aid and Defender Association 1995). A framework of judicial abilities and qualities introduced in the United Kingdom in 2008 by the Judicial Studies Board is intended to identify the knowledge, skills, behaviors, and attitudes that judges are expected to demonstrate in performing their judicial role (Judicial Studies Board 2008). These kinds of standards that provide a framework for assessing individual performance are becoming more widely understood and implemented in courts and other justice sector agencies across the globe. Standards for agency performance, on the other hand, are a more recent development. They are no less important, however, especially for courts and other justice sector agencies that want to demonstrate that current or increasing numbers of judges, prosecutors, and other staff are necessary to maintain quality operations, decisions, and services that achieve all agency goals. Already in the late 1980s, courts across the United States recognized the need to develop agency performance standards to respond to pressures to more effectively justify budget requests, and to demonstrate to other branches of government and the public their seriousness about delivering high performance—if adequate funding is available. This led to the development of the ―Trial Court Performance Standards‖ (NCSC 2003b), which define the desired results of court activities in core performance areas, such as access to justice, expedition and timeliness, equality, fairness and integrity, independence and accountability, and public trust and confidence. A series of pilot applications of these Trial Court Performance Standards in several U.S. courts produced a number of suggested measures and tools for courts to use to determine how well they accomplish their goals. This development followed an increasing trend among governments across the globe to embrace a Total Quality Management (TQM) philosophy,9 which requires translating the 9 TQM is a management approach that originated in the Japanese private sector in the 1950s. Since the early 1980s, its relevance for the public sector has been increasingly recognized. Total quality means that the culture, attitude, and organization of a company or agency strives to provide customers, clients, or users with products, decisions, and services that address their needs. Such organizational culture requires quality in all aspects of its operations. Initially, eight core elements were identified to be essential for successfuly implementing TQM: ethics, integrity, trust, training, teamwork, leadership, recognition, and 29 state‘s activities into a series of related performance measures. These performance measures then had to be implemented through clearly defined agency programs. Even before this trend evolved in the justice sector, performance-based budgeting had been introduced in government operations in the United States, New Zealand, and a significant number of other countries of the Organisation for Economic Co-operation and Development (OECD).10 This required that the state appropriations process be linked to performance measures when deciding how to allocate scarce public resources. The TQM and similar approaches provided a helpful basis for developing appropriate performance measures to link to the budget process. Although TQM or comparable methods have been adopted by growing numbers of governments around the world, some agencies have been reluctant to embrace this trend toward the objective measurement of institutional and individual performance. Judges and prosecutors especially sometimes oppose this kind of empirical measurement of their activities (DiIulio et al. 1993; Keilitz 2000). Their view is often that the most important product of their work is ―justice,‖ which simply cannot be satisfactorily defined or measured. While this is true to some extent, meaningful performance measures have been developed and applied and have greatly assisted courts and other agencies in improving their own operations, justifying budget and staff allocation requests, and demonstrating good performance to the public. In the end, the very detailed Trial Court Performance Standards developed in the United States could still not adequately address the ―quality‖ or ―justice‖ aspect of judicial work. They also turned out to be too resource intensive for most courts in the United States. They were a step in the right direction, however, as they provided agreed-upon definitions for core goals and measurement approaches that could be applied across different jurisdictions (see NCSC 2003b). Already in 2000, a study conducted by the National Center for State Courts looked further at the relationship between procedural efficiency and quality of outcome. The study indicated that ―the subjective working conditions of attorneys in expeditious courts are more conducive to effective advocacy, due process, and quality than the conditions in less expeditious courts‖ (Ostrom and Hanson 2000). This study and similar others linked efficiency to good management practices that also tend to increase the achievement of other justice sector goals, such as transparency, user satisfaction, and public trust and confidence, mirroring similar experiences in the private sector. These lessons learned eventually led to the development of CourTools, a means of providing a core set of performance measures in a less resource-intensive manner that could be more easily implemented by courts of different sizes and with limited outside communication (see Padhi 2010). This private sector definition has since been transfered and adjusted to better reflect the needs of various public sector agencies and users. 10 Through National Security Act of 1949, the U.S. Congress established performance-based standards for the newly formed Department of Defense; the Government Performance and Results Act extended this requirement to all federal agencies in 1993 (see 103 P.L. 62; 107 Stat. 285). Similar requirements have since been introduced in all U.S. states (see Young 2003), and are used in the majority of all OECD countries (see OECD 2007). 30 assistance or resources. Similar work has been undertaken in Singapore, Australia, and various countries in Europe. For example, the work of the European Commission for the Efficiency of Justice (CEPEJ), which was established in 2002, has moved forward on the process of benchmarking, establishing common approaches to data collection and measurement tools for courts across European countries. 6.6 Improving Performance Measures As the experiences in the United States have shown, the CEPEJ also recognizes that the development and application of meaningful performance measures for courts and other justice sector institutions is a process that continues to evolve. Today, particularly when government funding is scarce, justice sector agencies understand the importance of performance measures that enable them to demonstrate that they are serious about efficient, quality operations. Courts across the globe are looking for well-established tools that are credible and accepted measures of performance. More and more courts and other justice sector agencies are seeking International Organization for Standardization (ISO) accreditation and are participating in local or regional quality measurement regimes, such as the Malcolm Baldrige National Quality Award in the United States, the European Foundation for Quality Management Award, the Singapore Quality Award, or the Abu Dhabi Award for Excellence in Government Performance. Although such measurement approaches developed for a broader user group are prestigious and helpful, they cannot adequately reflect the special needs and operations of courts and other justice sector institutions and most, in any case, are too complex and resource intensive for the majority of courts and other agencies to take part. In 2009, recognizing the need for quality measures that can be applied by courts around the globe, experts from the United States, Europe, Australia, and Singapore, inspired by court quality models used in in several of these countries,11 formed the International Consortium for Court Excellence with the goal of developing a tool that would fill this gap. Building on lessons learned from all over the world, the Consortium developed an International Framework for Court Excellence.12 This Framework assesses a court‘s performance against seven areas of excellence, and provides guidance for improving that performance. It utilizes recognized organizational improvement methodologies while taking into account the special issues that courts face. By 2011, courts from 18 countries had become members of the Consortium. Their combined experience will be important for further adjusting the Framework, developing new and better tools, and advancing measurement approaches. Most recently, the application of this Framework is being tested in court agencies in a number of countries, including Australia, the Philippines, and Ukraine, and prosecution agencies are expressing an interest in testing an adjusted tool for their own purposes. 11 Several jurisdictions in the United States, Australia, and the United Kingdom, as well as courts in Finland, the Netherlands, and Singapore, for example, are using different quality models. 12 The website of the Consortium, which also includes the Framework, can be accessed at http://www.courtexcellence.com/. 31 7. Conclusions Increasing demands by the public for quality services amid ever limited government resources continue to challenge justice sector agencies across the globe to more effectively estimate staff and resource needs in a manner that captures case complexity and local variations. Today, jurisdictions in the United States, Europe, and elsewhere regularly revise the methods for estimating staffing requirements and also closely study mechanisms to forecast future position and budget needs, processes that will continue to evolve as technology advances. Governments are also testing measures to evaluate quality of service, a difficult but important factor in assessing staffing needs. Thus, today, as weighted caseload studies become a more common way to assess the justice sector workload and related staff resource requirements (Lienhard and Kettiger 2011), such studies are also increasingly linked to more detailed quality measures to align staffing requests not just with more comprehensive performance measures but also with budget requirements. Efforts in the United States and other countries have resulted in relatively good measures to estimate current staffing needs. They have also provided a number of important lessons:  Easy and quick methods to assess staffing needs provide little reliable information.  The most reliable methods are relatively complex and require time and resources to implement.  Even the best techniques require adjustments over time.  Even relatively good methods for estimating staffing needs provide information only on positions needed under the present conditions and thus have limited reliability over time; methods to forecast staffing needs require additional data and add another layer of complexity to the assessment.  Staffing assessments need to be conducted on the national, regional, and local levels to serve the different purposes of the courts and other justice sector institutions. National- or regional-level assessments are essential for the development of annual budget requests and budget forecasts and for efficiency assessments across similar courts. In order to identify the actual staff needs in each location, local circumstances need to be reflected.  Ultimately, estimates of staffing needs have to be combined with ways to measure the quality of the services and decisions delivered by the justice system. A very well- functioning, cost-effective system alone does not automatically equate with a just and fair system. It is essential for courts, prosecutors‘ offices, and other justice system agencies to identify and project their staffing needs to develop realistic budget requests and manage their operations efficiently. It is equally important to concede the limitations inherent in all methods applied and to always recognize that the ultimate goal is to staff the courts and other agencies adequately to deliver quality justice. The various measures and tools developed are important assets to assist courts and justice sector agencies in developing the information needed to support budget requests, assess performance, and estimate staffing needs. As much as this field has evolved only in the 32 past 20 years, to date, few courts have solid data collection approaches in place to establish staffing needs, especially in combination with other performance measures. The collection of experiences and good measurement approaches that are feasible for courts and agencies across the globe by organizations such as CEPEJ, the Consortium, and others is essential in the advancement of new, better, and hopefully less costly tools and applicable standards that can be applied in a wide variety of locations. 33 References Administrative Office of the Courts. 1999. Survey. Creating New Trial Courts. Texas Office of Court Administration. Austin, TX. Aikman, Alexander B., D. Kurt Pedersen, and Robert T. Roper. 1998. ―Designing a Judgeship Needs Process for Florida.‖ Gryphon Consulting Services, Eureka, CA. APRI (American Prosecutors Research Institute). 2002. ―How Many Cases Should a Prosecutor Handle?‖ American Prosecutors Research Institute, Alexandria, VA. Berlit, U. 1998. ―Überlegungen zu ‗Gerichtsreform, Justizkritik und richterliche Unabhängigkeit‘‖ (―Responses to ‗Court Reform, Critique of the Judiciary and Judicial Independence‘‖). Hannover, available at http://www.richterverein.de/j2000/berlit.pdf. Caylor, Steven. 2000. ―Measuring the Need for Judges: Rationalizing the Allocation of Judicial Resources.‖ Institute for Court Management, Court Executive Development Program, Phase III Project, National Center for State Courts, Williamsburg, VA. Church, Thomas. 1986. ―Examining Local Legal Culture.‖ American Bar Foundation Research Journal 10 (3): 446–518. Court Review. 2007–08. ―Special Issue on Procedural Fairness.‖ Court Review: The Journal of the Amercian Judges Association 44 (½): 1–84. DiIulio, John J., Geoffrey P. Alpert, Mark H. Moore, George F. Cole, Joan Petersilia, Charles H. Logan, and James Q. Wilson. 1993. ―Performance Measures for the Criminal Justice System. Discussion Papers from the BJS-Princeton Project.‖ Office of Justice Programs, Bureau of Justice Statistics, U.S. Department of Justice, Washington, DC. Deloitte Consulting GmbH. 2005. ―Endgutachten. Erarbeitung eines Systems der Personalbedarfsberechnung für alle Berufsgruppen des richterlichen und nichtrichterlichen Dienstes in den Fachgerichtsbarkeiten der Bundesrepublik Deutschland - Projekt PEBB§Y Fach‖ (―Final Report. Development of a System to Estimate Staff Positions for all Judicial and Non-Judicial Services for the German Special Court System‖). Duesseldorf, available at http://www.thueringen.de/imperia/md/content/text/justiz/final_endbericht.pdf. Deutscher Richterbund.1999. ―Stellungnahme zu einer Richterarbeitsplatz-Untersuchung (Response to Study on judicial staffing).‖ Berlin. 34 Flango, Victor E., Brian J. Ostrom, Carol R. Flango. 1993. “How do States Determine the Need for Judges?‖ State Court Journal 17 (3): 3–11. Flango, Victor E., and Brian J. Ostrom. 1996. Assessing the Needs for Judges and Court Support Staff. Williamsburg, VA: National Center for State Courts. Gramckow, Heike. 2011. ―The Challenge of Making Courts More Efficient: Recent Experiences of the Western Balkan States.‖ In Law in Transition 2011. Towards Better Courts, 60–71. London: European Bank for Reconstruction and Development. Gramckow, Heike, and Frances Allen. 2011. ―Justice Sector Reform in Mongolia: Looking Back, Looking Forward.‖ Justice and Development Working Paper 16, World Bank, Washington, DC. Greacen, John. 1999. ―How Fair, Fast, and Cheap Should Courts Be? Instead of Letting Lawyers and Judges Decide, New Mexico Asked its Customers.‖ Judicature 82 (6): 287. Guidero, Margaret, and Suzanne Tallarico. 2011. ―Judicial Workload Study for the Superior Court in Yuma County, AZ.‖ The Court Manager 25 (2): 15–25. Herrler, E. 2004. ―Die neuen Pensen.‖ Deutsche Richterzeitung September: 229–32. Hirth, W. 1997. ―Pensenschlüssel: Ein Pyrrhussieg in Hessen.‖ Mitteilungen des Hamburgischen Richtervereins 2, available at http://www.richterverein.de. Hurst, Hunter. 1999. ―Workload Measurement for Juvenile Justice System Personnel: Practices and Needs.‖ JAIBG Bulletin, November. Jacoby, Joan. 1987. Caseweighting Systems for Prosecutors: Guidelines and Procedures. Washington, DC: U.S. Department of Justice. Judicial Studies Board. 2008. ―Framework of Judicial Abilities and Qualities.‖ Judiciary of England and Wales, London. Keilitz, Ingo. 2000. ―Standard and Measures of Court Performance.‖ Measurement and Analysis of Crime and Justice 4: 559–593. Krislov, Samuel. 1995. ―Caseloading the Balance.‖ In Conference on Assessing the Effects of Legislation on the Workload of the Courts: Papers and Proceedings, ed. F. Magnum, 33– 46. Washington, DC: Federal Judicial Center. Lienhard, Andreas, and Daniel Kettiger. 2011. ―Research on the Caseload Management of Courts: Methodological Questions.‖ Utrecht Law Review 7 (1): 66–73. 35 Lord Chancellor‘s Department. 2000. ―Review of Magistrates‘ Courts Revenue Grant Allocation Formula.‖ Lord Chancellor‘s Department Consultation Paper, Lord Chancellor‘s Department, London. McEldowney, John. 2001. ―Developing the Judicial Budget: An Analysis.‖ Paper presented at the World Bank Conference, Co-Hosted by the Government of Russia. Saint Petersburg, July 8–12. National Legal Aid and Defender Association. 1973. Standards for the Defense. Washington, DC: National Legal Aid and Defender Association; available at: http://www.nlada.net/sites/default/files/nac_standardsforthedefense_1973.pdf ———. 1995. Performance Guidelines for Criminal Defense Representation. Washington, DC: National Legal Aid and Defender Association. NCSC (National Center for State Courts). 2003a. ―Mongolia Judicial Reform Program. Annual Report 2003.‖ National Center for State Courts, Williamsburg, VA. ———. 2003b. Trial Court Performance Standards Desk Reference Manual. Williamsburg, VA: National Center for State Courts. ———. 2005. ―Commonwealth of Massachusetts—Administrative Office of the Trial Courts Staffing Study.‖ National Center for State Courts, Williamsburg, VA. ———. 2007. ―A Workload Assessment Study for the New Mexico Trial Court Judiciary, New Mexico District Attorneys‘ Offices, and New Mexico Public Defender Department.‖ National Center for State Courts, Williamsburg, VA ———. 2009. ―Kosovo Justice Support Program Annual Report June 2008–June 2009.‖ U.S. Agency for International Development, Washington, DC. ———. 2010. ―Oregon Court of Appeals Judicial and Staff Weighted Caseload Study.‖ National Center for State Courts, Washington, DC. NCSC and COSCA (Conference of State Court Administrators). 1989. State Court Model Statistical Dictionary. Williamsburg, VA: National Center for State Courts. Neue Richtervereinigung. 2008. ―Stellungnahme der NRV—Questionnaire for the 2008 CCJE Opinion Concerning the Quality of Judicial Decisions.‖ Bundesvorstand, Berlin, available at http://www.nrvweb.de/main.php?id=171&stellung_id=60&lv_id=88&fg_id. OECD (Organisation for Economic Co-operation and Development). 2007. Performance Budgeting in OECD Countries. Paris: OECD. 36 Ostrom, Brian J., and Roger A. Hanson. 2000. Efficiency, Timeliness, and Quality: A New Perspective from Nine State Criminal Trial Courts. Williamsburg, VA: National Center for State Courts. Ostrom, Brian J., Roger Hanson, Charles W. Ostrom, and Matthew Kleiman. 2005. ―Court Cultures and their Consequences.‖ Court Manager 20 (1): 14–23. Ostrom, Brian J., Charles W. Ostrom, Daniel Hall, William E. Hewitt, and Timothy Fautsko. 2000. ―Florida Delphi�based Weighted Caseload Project: Final Report.‖ National Center for State Courts, Williamsburg, VA. Padhi, Nayantara. 2010. ―The Eight Elements of TQM.‖ iSixSigma, Ridgefield, CT, available at http://www.isixsigma.com/methodology/total-quality-management- tqm/eight-elements-tqm/. Popkin, Margaret. 2002. ―Efforts to Enhance Judicial Independence in Latin America: A Comparative Perspective.‖ In Guidance for Promoting Judicial Independence and Impartiality, 100–132.Washington, DC: Office of Democracy and Governance, U.S. Agency for International Development. Reference for Business. 2011. Encyclopedia of Business, 2nd ed, available at http://www.referenceforbusiness.com/encyclopedia. Rivera, Eliada. 2000. ―Staffing Levels in the Clerk‘s Office, District Courts Division, 11th Judicial Circuit, Miami-Dade County, Florida.‖ Fostering Excellent Court Performance Through Adequate Staffing Levels, Executive Summary, National Center for State Courts, Williamsburg, VA. Texas Judicial Council. 1999. ―Assessing the Need for New Trial Courts/Judgeships: National Survey of States.‖ 1998–1999 Texas Judicial Council Annual Report, Committee on Judicial Performance Measures Staff Reports, Texas Judicial Council, Austin, TX, available at http://www.courts.state.tx.us/tjc/ar99/report.htm. Tragter-Schubert, Martina. 2000. ―Neue Steuerungsmodelle an niederländischen Gerichten oder der lange, steile Weg zur Qualität‖ (―New Control Models for the Dutch Courts – or the long road to Quality‖). Betriff Justiz ,16: 218; available at http://www.betrifftjustiz.de/texte/frame.php?ID=Tragter- Schubert.html&wb_2947_session_id=oimoeach5l2ponif2ruu9284b1. Tyler, Tom R. 1984. ―The Role of Perceived Injustice in Defendants‘ Evaluation of Their Courtroom Experience.‖ Law and Society Review 18: 51–74. ———. 1997. ―Procedural Fairness and Compliance with the Law.‖ Swiss Journal of Economics and Statistics 133: 219. 37 Vidmar, Neil. 1990. ―The Origins and Consequences of Procedural Fairness.‖ Law and Social Inquiry 15 (4): 887–92. Young, Richard D. 2003. ―Performance-Based Budget Systems.‖ Public Policy and Practice (USC Institute for Public Service and Policy Research) January: 11–24. 38